gevent compatibility

[zj8487]

Hi, I want to make grpc client and server code to work with gevent, is grpc compatible with gevent?

[vinays]

@zj8487 did you get gRPC(python) working with gevent.monkeypatch? Could you please share the work-around.

[zj8487]

@vinays
it is sad, it is not work with gevent.
i do it as follow:
a thread(not monkey patched) to deal with the grpc and send or receive message from a message queue which was listen or loop in a gevent to process the request or response.

[mayflaver]

grpc make async io well， why do you want to make grpc client and server code to work with gevent? they do the same thing.

[vinays]

Thanks @zj8487

@mayflaver in our case 'grpc' and 'gevent' are transitive dependencies of gcloud-python and gunicorn respectively in our legacy app. If I was writing an app from scratch I would agree with you, but in this case these dependencies are not enforced by me. :(
To best of my knowledge--gunicorn with gevent as worker_class monkey patches a lot of standard library functions including threading. This conflicts with the use of "futures" in grpc and hangs the process.
Since, gunicorn(w/ gevent) is quite commonly used in the industry and anyone who wanna use gcloud-python(or other grpc based libs) in their app will likely run in to the same issue. I am open to any suggestions that might help solve this issue without changing the dependencies too much. Maybe gRPC and gunicorn teams should talk and figure out a solution?

[soltanmm]

Reopening as this seems to warrant investigating.

[vinays]

@soltanmm any resolution on this problem? It will be really help if we can get a rough timeline for it.

[soltanmm]

@vinays There's no timeline for it at present (and without in-depth gevent knowledge we're not exactly leaping into action on it), but as there appears to be mounting community pressure I guess I'll take a quick look tomorrow. Feel free to proactively provide a minimal reproduction; it might speed things along.

[soltanmm]

@vinays We have a hard time justifying interoperability with a library feature that seems to be unable to work with standard library elements in Python 3 because it literally changes the standard library and incompletely so.

That said, it's worth documenting this, and I don't know much about gunicorn. What's wrong with the other options listed here for your use case?

[soltanmm]

@vinays Actually, this issue comment might be of relevance to you: benoitc/gunicorn#927 (comment)

It appears that this problem arises with Django as well and what the gunicorn folks end up doing might be the fix you need.

EDIT: Hmm, nevermind. Reading further down the thread I see a face-heel-turn...

[vinays]

Thanks @soltanmm for looking into the issue.

@mickgarvey @grourk might have more insight into why we did not choose other options in the list. From what I know, since our app is IO bound 'sync' worker does not scale well because of blocking IO.

@soltanmm did your investigation conclude a root cause of the issue? Do you know what part of standard library patching was causing the hang in Python 2.7? Would excluding 'thread' from patch_all, fix the issue? I can test that myself, but I am looking for any side-effects you might know and most likely won't surface itself in a simple test. Any other information from your investigation might also help.

[soltanmm]

@vinays I'll have to guiltily admit that after reading about what gevent monkeypatching actually is and checking the docs on known hangs interacting with library elements we appear to use, I face-palmed and stopped before attempting a reproduction. I'll take a look again on Monday and get some first-hand results if that'll allow everyone to sleep a little better.

I have a strong feeling it's just going to be the usual, "Green-threads cannot be cooperative if they block on something that isn't cooperative," sort of deal, though (where 'something that isn't cooperative' could be anything the gevent folks missed while monkeypatching [which I have a feeling includes concurrent.futures] and every blocking gRPC core operation).

[soltanmm]

It's basically that. gRPC Python uses multiple threads for a server (e.g. one for the server receiving messages, one to pass the call data along to the application). These trip over each other when they don't cooperatively release polling to gevent (which at least one of them can't do because it's in gRPC core code and the core is obviously not based on gevent).

Short of getting in-depth knowledge of gevent (and I'm not a gevent dev, so, nope), I can't come up with much more than saying, "Excluding thread from monkey-patching seems to work."

I'm not going to close this as it's possible that there exists a way to force the use of standard Python threads despite monkey patching where they are necessary, or that some other hacky-solution-against-a-hacky-solution exists, but we will not attempt to support gevent's monkeypatching for the time being.

[nathanielmanistaatgoogle]

I'm still coming up to speed on what's being asked here, but from what I've gathered so far, I suspect it will be a lot: one of the core design assumptions of gRPC Python is that a reasonable number of threads will always be available to be devoted to blocking operations such as C extension calls and calls into application code. Rolling back that assumption and moving forward with a different structure are going to be substantial work, if they're something that we ever do at all.

I feel like I'm still missing quite a bit. What's the minimum sample program you could write that you would expect to work but that doesn't?

[vinays]

@soltanmm thanks for the explanation, it resonates with my understanding.
@nathanielmanistaatgoogle if your questions was pointed at me, I can write a sample script later today to reproduce the problem.

[nathanielmanistaatgoogle]

@vinays: yes, a code sample would be delightful - something that is enough to illustrate a use of gRPC Python, how you expected gRPC Python to behave, and how gRPC Python actually behaved.

[vinays]

@nathanielmanistaatgoogle here is some sample code to illustrate the issue. Let me know if you have any questions.

[Globegitter]

Just another use-case we have been running into. We are using graphql (https://github.com/graphql-python/graphene) as our public API and grpc for internal communication.

The graphql library supports running resolving requests in parallel using gevent but the problem is, that any grpc calls we are making (in different resolve methods) are run serially, which now is the cause of quite a few performance issues.

I have just started to investigate to find any potential solutions or workarounds so the grpc requests can run in parallel, but good to see there are other people who are working towards interoperability.

[soltanmm]

@nathanielmanistaatgoogle Thoughts on gevent/gevent#786? We could leave an unsupported (private, but documented) hook for the underlying thread pool executor somewhere in the gRPC implementation and allow users to swap it out post-import/pre-absolutely-anything-else if they want to at their own risk (facilitating the 'be selective' approach @jamadden mentioned).

[soltanmm]

Better yet: we could do that and attach warnings (similar to the deprecation warnings) to it in terminal emboldened firetruck-red all-caps font that the users have entered monkey territory and shouldn't be surprised if their hats get stolen and everything hurts twice as much.

[nathanielmanistaatgoogle]

Thank you @jamadden for your investigation.

Use of the same thread, with program control passing from application to gRPC Python to the gRPC Core and then to a blocking system call is an explicit part of gRPC’s design, is the basis for many of the performance improvements that we’ll be making in the next few months, and would be very hard to change. In particular major alterations would have to be made to the Core to enable its completion queues to be used in the manner gevent would want to use them. I’ve spoken with @ctiller about it and this work may be scheduled engineering effort if it is forecast to be valuable enough to be prioritized above other goals.

There may be a certain recipe for monkeypatching that happens to work today but we wouldn’t be able to offer proper support (with test coverage and a promise not to break in the future) for anything based around selectively patching thread pools used internally within gRPC Python. In particular we have near-term plans to scale back how widely we use thread pools. If you want today to do something based on selectively patching it may work fantastically for you and then break in a future release. I would celebrate your bravery and wish you fun.

[nicolasnoble]

@scorpioWalker: I think this is something you should create a new issue about. This one issue here is more integrating grpc with gevent. That's different from simply being able to run next to it.

[zero-master]

Google Datastore with Gunicorn blocks forever in app engine flexible.
I am not sure if it's because of this issue.

gevent==1.2.2
google-api-core==0.1.1
google-auth==1.2.1
google-cloud-core==0.28.0
google-cloud-datastore==1.4.0
google-gax==0.15.16
google-resumable-media==0.3.1
googleapis-common-protos==1.5.3
greenlet==0.4.12
grpcio==1.7.0
gunicorn==19.7.1

[lita]

Hello! Would love to hear the status of this!

[AndreCimander]

We could use this as well, spawning processes is a waste of memory and cpu for heavy-I/O bound tasks...

[Revolution1]

any update?

[kpayson64]

I'm hoping to have a PR out this week for gevent support.

[smetj]

hi @kpayson64 & devs,
I understand this issue is difficult or doesn't get any priority but would you pls communicate any concrete progress on this?
The issue is open for +2 years. It's totally fine if this doesn't get solved/delivered but if that's the case can we pls close this issue so ppl can move on to an alternative?

[codesuki]

@kpayson64 I saw in your repo that you are still working on this. Thanks! Does it seem to work?

[kpayson64]

@smetj
@codesuki
Sorry about the delay. There were a few high priority issues the popped up that I needed to address, and I was on vacation for a week so this got pushed back a bit.

This is very much at the top of my stack right now. I'm currently tackling some lingering test failures, but the bulk of this seems to be working.

[codesuki]

Sounds great! Thanks :) I hope we can get our gRPC working with locust soon.

[smetj]

Super excited this is moving forward! Thanks for the feedback ...
Can't wait starting to use Google cloud pubsub with Gevent + Python!

[ekhaydarov]

is aiohttp a viable alternative?

[lita]

Same! I am excited to try this out.

[raybotha]

Would be great to get #14561 merged in

[kpayson64]

With #14561 in, you should now be able to use gevent.

Note that you have to explicitly monkey patch Python, as well as enabling the gevent loop for Python (Before doing anything with gRPC)

from gevent import monkey
monkey.patch_all()

import grpc.experimental.gevent as grpc_gevent
grpc_gevent.init_gevent()

This will be included in the 1.11 release.

I'm closing this master issue, and separate issues can be created with any issues encountered using gevent.

[mitar]

For those who want asyncio support, can #7910 be reopened or should we open a new issue?

[mitar]

Or is #6046 that?

[charyorde]

Finally. Testing away!

[codesuki]

@kpayson64 thanks, great job!

[kpayson64]

@mitar
Feel free to open a new issue. I'd expect that we need to add some new surface APIs for asyncio support, so its probably a good idea to have a separate issue.

[mitar]

I opened #14876. Based on existing gist (linked in the issue) it seems this could be easily done.