Is eventlet compatible with ContextVars?

[pgjones]

Looks like gevent had to add a greenlet compatible implementation of ContextVars, issue and fix. Does eventlet need to do the same?

[temoto]

@pgjones I've put up a (probably wrong) test real quick and it seems that eventlet also should patch contextvars.

# check.py
import eventlet
import random
import contextvars

ctx = contextvars.ContextVar('foo')
ctx.set(0)


def w(id):
    for _ in range(10):
        eventlet.sleep(random.random()/10)
        print(f"{id}: set")
        ctx.set(id)
        eventlet.sleep(random.random()/10)
        print(f"{id}: get", ctx.get())

t1 = eventlet.spawn(w, 1)
t2 = eventlet.spawn(w, 2)
t3 = eventlet.spawn(w, 3)
t1.wait()
t2.wait()
t3.wait()

# observed
3: set
2: set
2: get 2
3: get 2  # so eventlet greenthread #3 reads global singleton contextvar set by #2, should be `X: get X` for all w id
...

Thanks for educating on this new library.

[pgjones]

@temoto thank you :).

[kamalmarhubi]

Some useful context here: greenlet added support for contextvars in 0.4.17 released in September. This is a nicer solution than adding a monkey patch, and is more in line with the intended use of contextvars as I understand it.

I modified @temoto's script to assert on failure and sleep for 0 seconds to simply yield to another greenlet. This lets me run it for far more iterations; see below.

I tested with a pretty old eventlet, 0.21, as well as the current version, In both cases, with greenlet 0.4.16, the script fails almost immediately. With 0.4.17 or 1.0.0 it succeeds.

I think this issue can be closed with a clarification somewhere that greenlet 0.4.17 or 1.0.0 (just released yesterday) is used.

---

My modified script for easier testing:

# check.py
import eventlet
import contextvars

ctx = contextvars.ContextVar('foo')
ctx.set(0)


def w(id):
    for _ in range(10000):
        eventlet.sleep()  # yield
        ctx.set(id)
        eventlet.sleep()  # yield
        assert ctx.get() == id, "FAILED %d != %d" % (id, ctx.get())

t1 = eventlet.spawn(w, 1)
t2 = eventlet.spawn(w, 2)
t3 = eventlet.spawn(w, 3)
t1.wait()
t2.wait()
t3.wait()

[davidism]

If I'm understanding this right, it means that even though eventlet.patcher.is_monkey_patched("contextvars") still returns False, they'll work correctly? Is there a way that we can detect this support?

[temoto]

As gentleman above says, you just have to install recent greenlet and no patching is needed.

…

On Tue, Feb 9, 2021, 23:54 David Lord ***@***.***> wrote: If I'm understanding this right, it means that even though eventlet.patcher.is_monkey_patched("contextvars") still returns False, they'll work correctly? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#663 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAGTMJTPRQLXVZFFI34S53S6GOHFANCNFSM4TEBJLKQ> .

[davidism]

What's the status of this? It's still open, but the last comment seems to indicate that eventlet works with contextvars despite reporting them as not patched. We are dropping Python 3.6 support in Werkzeug, are moving from thread locals to contextvars, and would like to drop our compat code for eventlet if possible.

[temoto]

status

greenlet>=0.4.17 is compatible with contextvars. Eventlet does not put such version limit on greenlet. You may want to list greenlet dependency separately, to ensure recent version.

[davidism]

PyPy comes with greenlet included, and it's not possible to pip install a more recent version. The version they have, even in their latest releases (PyPy 3.7.10 7.3.5), is equivalent to <0.4.17, so Eventlet will not work with contextvars on PyPy. Gevent includes its own contextvars support, so it works even with old versions of greenlet, and thus with PyPy.

This is an issue because Flask has been adding support for async, and as part of that has moved to using contextvars instead of threading.local so that data is local to each async worker. The popular Flask-SocketIO extension uses Eventlet, and thus if we drop our hacky contextvar compat, Flask with Flask-SocketIO will no longer work on PyPy.

I understand Eventlet is relying on greenlet to support contextvars, but it seems important to have this working on PyPy one way or another. Perhaps if you added your voice to https://foss.heptapod.net/pypy/pypy/-/issues/3408 it may be addressed.

[davidism]

Sorry for the confusion. Looks like PyPy 7.3.7, released last week, does have support for greenlet with contextvars, even though it still reports an old version number.