Advice

I understand that:

  • await some_async_function() doesn't yield control - it just calls the function synchronously
  • await some_io_function() DOES yield control to the event loop

But what's the actual mechanism? When I write:

async def foo():
    await bar()  # Doesn't yield
    
async def bar():
    await io_function()  # Yields here

How does the Python executor know that bar() should execute synchronously but io_function() should yield? What makes io_function() special?

Is there some internal flag or object type that signals "this is actual I/O, yield now"?

Improve this question

5 Replies 5

Vladimir
•

await in Python is just point of possible switching context. It used in asynchronous functions to pause execution until the awaited task is complete, allowing other tasks to run at the time while another coroutine is waiting.

Improve this answer
Klaus D.
•

It all comes out to what your coroutine actually does. To be truely asynchronous it has to give back control to the event loop at some point. If it doesn't it will act more or less like a normal synchronous function. At the point of the await coroutine() call, the interpreter does not handle them differently.

Improve this answer
Kapuca
•

I thin this chapter explains it best:

https://docs.python.org/3/howto/a-conceptual-overview-of-asyncio.html#await

(the crucial snippet) ---

await is a Python keyword that’s commonly used in one of two different ways:

await task
await coroutine

In a crucial way, the behavior of await depends on the type of object being awaited.

Awaiting a task will cede control from the current task or coroutine to the event loop.

...

Unlike tasks, awaiting a coroutine does not hand control back to the event loop! Wrapping a coroutine in a task first, then awaiting that would cede control. The behaviour of await coroutine is effectively the same as invoking a regular, synchronous Python function.

--- (I recommend the link, it gives good examples, but I don't wanna take too much credit for copy-pasting)

The only other thing to keep in mind is that io_function() is actually a task!

How I see it with my monkey brain if you are a thread (or CPU), when you get to the await instruction, if you are awaiting a task (something that is not your job to process but rather somebody elses job to do) then you better go do something else. But since you need this await instruction in order to continue straight forward you instead take another job from the event loop and continue there... but if you are awaiting a corutine (this is your job to process) then you go do the corutine (the await key word in that case just tells you that this is most probably tedious function that will make you await a task down the line)

Improve this answer
deceze
•

There's basically nothing you can do in pure Python that would ever truly suspend your code and cede control back to the event loop, making it wait for something to complete. Because your Python code will pretty much always just be CPU bound, and not wait for anything else.

You need to "step outside Python" to encounter something that needs actual awaiting. Like, making a network request and waiting for its response. Or asking the operating system to open a file and read its contents; depending on where that file is being read from, that can take a relative eternity, compared to all the things Python could be doing on the CPU in the meantime.

Things like these rely on underlying operating system infrastructure for the most part. E.g., to send a network request and awaiting its response, you'd open a port, send a request to some target, and then do nothing until something comes back on that port. The operating system coordinates this kind of thing, and has the necessary hooks for you to say "wake me up when anything happens here".

There's no direct interface to these things from pure Python; this is happening in the underlying C libraries. Python exposes some primitives in the asyncio module like create_server, and higher level abstractions like streams.

So, if you await some async Python code, that only coordinates causality. Meaning, code after this will only continue once this function has completed. But if that function never actually calls out to one of the primitives described above, then that mostly has no effect. Only if somewhere at some point you are calling one of those primitives which truly suspend Python processing, awaiting completion of some external event coordinated by the operating system, does it really have an effect. But even if that call is ten levels deep, your entire chain of Python code before that needs to be properly async and awaited, if you want to coordinate asynchronous causality throughout your program.

Improve this answer
VPfB
•

An await is basically a yield from (please read the "yield expression" docs if you haven't already) and a coroutine is basically a generator. A ganerator can either yield a value or return a value as its last statement (It becomes a StopIteration argument, that's not important now). In asyncio, the return is used to return a result to the caller and the yield is used to signalize that the function cannot continue because a result is not available yet. The returned value becomes the awaited value. The yielded value is an empty container called a Future where the now missing value is expected to be later filled-in. If the value will be a result of some I/O, it is expected that the coroutine made necessary steps to start the I/O. The yielded Future then goes through all nested awaits to the other end of a bi-directional pipeline. There is the scheduler.

Improve this answer

Your Reply

By clicking “Post Your Reply”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.