python asyncio: allow user to choose fire-and-forget vs blocking-group-of-functions

Question

I'd like to send an http request, and then operate on its response:

import requests

s=requests.Session()
....
(session setup...)
....
(url and params setup...)
....
r=s.get(url,params=params)

print('response json:'+str(r.json()))

In that case, the request should be blocking, so that the print line doesn't happen until a response has been received.

At other times, I'd like to send fire-and-forget requests - the response is never used, so, I don't care how long it takes to be received:

s.post(url,data=data,timeout=0)

What are the options for providing for both cases (blocking vs. fire-and-forget) with an intermittent network connection?

'retry' (with exponential back-off timing or such - as in the tenacity module) works in a blocking manner, but, this question isn't for a web-scraping or large-number-of-requests context.

I do have a functioning queue-and-thread technique in place: submit the request to a Queue instance, and then set a threading.Event to tell the worker thread to start working through the queue, with a delay between non-successful iterations, with each queue item not being removed from the queue until a valid response is received. This works nicely for fire-and-forget, but, it doesn't work in the blocking case, because the timing of the valid response can't be known, so the main thread has likely moved on before the response is available.

asyncio (instead of threading) could be made to either fire-and-forget or block, but I'm trying to find the technique that would be the most clear for the downstream coder.

E.g. should the default be blocking, or, should the default be fire-and-forget?

It's easy enough to specify blocking-or-fire-and-forget with an argument, but the downstream coder could be blindsided when they expect it to be one way and it's actually the other.

Is there a way to specify a 'group' of lines of code that should be a 'blocking set'?

While it would be great to be able to tell if the return value is "used", so that the code could 'automatically' determine if a call should be blocking or not, the answers to this previous question point out that it's not really possible to do so reliably.

Sri Ram · Accepted Answer · 2025-07-21 19:27:46Z

-1

To support both blocking and fire-and-forget HTTP requests clearly, the best approach is to use Python’s asyncio with an explicit flag like fire_and_forget=True. This way, the default behavior remains blocking—ensuring the response is available before continuing—while still giving developers the option to run non-blocking background tasks when needed. Using asyncio.create_task(...) allows you to fire off tasks without waiting for them, and wrapping this in a simple HttpClient class makes the code clean and easy to use. It’s also a good idea to document this clearly so that others know what to expect and don’t get surprised by silent failures in fire-and-forget mode.

answered Jul 21 at 19:27

Sri Ram

1

Sign up to request clarification or add additional context in comments.

2 Comments

David Maze Jul 21 at 19:51

Can you edit this answer with some details about what exactly the fire_and_forget option should actually do? How should the asker change their code?

Community Jul 21 at 22:15

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.

Collectives™ on Stack Overflow

python asyncio: allow user to choose fire-and-forget vs blocking-group-of-functions

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related