-
-
Notifications
You must be signed in to change notification settings - Fork 835
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use the existing runloop for sync calls in async handler #179
Comments
FWIW; I tried the following in a vein hope that it would do what I wanted:
But alas, no success.. this the end of the stacktrace.
|
In the meantime; I'll just use Line 625 in 66754ad
|
Hi to use backend parameter, it has to be derived by ConcurrencyBackend. |
Yeah, I figured that after I got the exception. I figured it would handle a standard runloop. |
you can check sample class from Line 19 in 66754ad
|
Yeah, unfortunately I can't modify my existing runloop to meet the requirements of the |
Maybe |
There is no API to provide the event loop yourself, however there is a We should consider adding this as a feature though. |
I'm not sure about the context you're working with but there's some subtleties around using the sync API in an async application. The sync API internally uses the loop's |
Okay, that's a problematic thing to be doing. If you're running in the context of an event loop, but then making blocking style network calls you're going to block the event loop. You don't want to be doing that. We've got a relevant issue here about how we describe that most clearly in any raised exception, but it's a fundamental property of the differing concurrency model, rather than a bug (at least as I read it here) |
I'd also echo what @yeraydiazdiaz's comment: if you're running in an async context, then you need to be using the async client. (Tho it might be useful to understand why you're not doing that, in order to provide better support for other users who try the same thing) |
@yeraydiazdiaz Yeah I figured; I was just trying to make the api client library idiot proof; and also give them the option to use a sync client even in an async application (for whatever reason that might be). I totally agree; staying in async land is definitely where I want to be too; but I can't assume that everyone is in async land; in fact my experience is that most aren't in the python world, unfortunately. |
@tomchristie yeah I completely agree & understand; it was more a question of is it possible rather than what am I doing wrong. I understand how its working; but was seeing if the library could behave the way I was hoping based on the way I'm building out the api client library. No big deal though; I can just revert to the
Then any following calls would either need to be using |
That's not really the right resolution here. httpx provides a sync client, for sync usage, and an async client, for async usage, so you have both cases covered. The error you're seeing only occurs when you try to call into the sync interface from within an async context which a broken thing to do for your users, regardless of if that's implemented in There are technical ways we could make it not error in (*) Unless you're switching over to sync portions of the codebase within a seperate thread, in which case |
I guess this is the core bit. Making the library robust here ought to mean raising a nice clean, explanatory error when it's used in a broken way, rather than allowing that to pass silently. |
Exactly! Trying to make an API both sync and async is an attractive nuisance. It does not even help end users who write applications. On a related note, here's one way for httpx to avoid using asyncio and its event loop (and avoid this error):
But I don't know if it's worth it or not. |
Hey, seems like this issue has drifted away. :) Is there anything actionable we could do about it?
I’m in an issue closing mood today, and I think this one could just be considered as a constraint and documented as such (ie option 1). |
I'd say documented constraint. There's a million and one other broken blocking operations that a user could do, that won't autodetect. Unless there ends up being some backed in support in the stdlib guarding against this (unlikely) then this really just comes down to "if you're using async you need to be attentive to avoiding blocking operations". |
I've just encountered exactly the same issue. In my understanding, there are cases where it makes sense to make sync calls inside a coroutine. Let's say that you have a command-line tool that reads a JSON file from a url containing a
Of course I can make the sync requests outside of
|
I think your comment is mistaken here, for two reasons:
In your case, it seems you have to use async (as most users do when they're using HTTPX in an async context), e.g. by wrapping all the code inside the I think this issue is a duplicate of #508 (or vice versa). I'll go ahead a push a PR that sync calls are not supported in an async environment; as discussed earlier this should at least address this PR. |
I completely agree, however, in the case of a simple script that needs to get some information from a url, before using that information to distribute it across multiple coroutines, blocking the event loop does no harm at all. In my example, the sync call happens when there is only one coroutine running on the event loop, so it makes no difference whether I make blocking calls or not.
Oh trust me, I'm not worried about that overhead, which, when you account for the I/O stuff (network requests), is most likely will not going to be noticeable ever. Just to be clear, I'm not alleging that you should break the philosophy of the library, because it makes sense so for a limited number of cases (although in web scraping that could be a common scenario). I'm just saying that it should bee made explicit 😄 For me, the issue was assuming that |
Hi, this library looks awesome and I really want to use it; but I seem to have hit a blocker and can't seem to find a solution in the docs.
TL;DR: Is there any way to provide our own runloop when using the
sync
calls as looking through the source the sync calls are actually using the async request call.Basically I'm writing a client api that provides 2 interfaces; an async and sync interface.
The issue that i am having is that if I try to use the sync methods with an existing runloop (provided by Sanic in this scenario), it complains with the exception:
Cannot run the event loop while another loop is running
.Essentially I'm just calling:
note that nothing here is async; apart from the fact that the
.get_some_url
call is made from within an async function like so:This works fine when using the async calls like this:
So I was just wondering if its possible to provide our own runloop when using the sync calls? Sanic uses uvloop internally, would be nice to be able to just pass that straight in somehow when constructing the
httpx
object, or setting it as a property after import.Thanks!
The text was updated successfully, but these errors were encountered: