threading.current_thread does not work as expected for plain greenlets

[melwitt]

I'm running into an issue while using locks from the fasteners library [1][2] in an eventlet monkey patched application.

The fasteners library uses the threading.current_thread method to track which thread is holding a particular lock [3] and it works well for green threads of type GreenThread that are created by calling spawn [4]. However, the application I'm working with creates greenlets by calling spawn_n [5] and in this case threading.current_thread falls back to the standard threading.current_thread method [6] instead of calling eventlet.getcurrent.

Is this expected that the eventlet threading.current_thread method cannot be used with plain greenlets created by spawn_n?

Here is an attempt at sample code demonstrating the above:

import eventlet


eventlet.monkey_patch()


import threading


main = eventlet.getcurrent()


def f():
    print('current_thread = %s' % threading.current_thread())
    print('getcurrent = %s' % eventlet.getcurrent())
    main.switch()


print('\nCalling spawn() ...')
t1 = eventlet.spawn(f)
t1.wait()

print('\nCalling spawn_n() ...')
t2 = eventlet.spawn_n(f)
t2.switch()

print('\nDone')

Output:

Calling spawn() ...
current_thread = <_GreenThread(GreenThread-1, <GreenThread object at 0x7f42b7725880 (otid=0x7f42b788e080) current active started>)>
getcurrent = <GreenThread object at 0x7f42b7725880 (otid=0x7f42b788e080) current active started>

Calling spawn_n() ...
current_thread = <_MainThread(MainThread, started 139924526151488)>
getcurrent = <greenlet.greenlet object at 0x7f42b3c07b40 (otid=0x7f42b788e080) current active started>

Done

Any input would be appreciated.

[1] https://fasteners.readthedocs.io/en/latest
[2] https://pypi.org/project/fasteners
[3] https://github.com/harlowja/fasteners/blob/e1764c7/fasteners/lock.py#L119
[4] https://eventlet.net/doc/modules/greenthread.html#eventlet.greenthread.spawn
[5] https://eventlet.net/doc/modules/greenthread.html#eventlet.greenthread.spawn_n
[6] https://github.com/eventlet/eventlet/blob/8904a33/eventlet/green/threading.py#L125-L128

[temoto]

Right, spawn_n() looks inconsistent in big picture. I vaguely recall this decision, separation to greenlets and greenthreads was intentional, unsure if the distinction is still useful. spawn_n is intentionally lacking features for speed.

It could be possible to bend current_thread to raw greenlets, but there may be dragons as well. I'd most likely merge any relevant work that doesn't hurt performance; sorry, don't have time to actually code it.

So if spawn_n doesn't fully cover your task, use spawn. Distinction is negligible most of the time. I understand you may not control spawn caller. Your best option is to send them patch. Another option is patch eventlet.spawn_n=spawn, they should be one way compatible.

[melwitt]

Thank you very much for the reply, this info will definitely help me find a way forward. Much appreciated!

[melwitt]

Apologies that I forgot to mention this earlier ... but the scenario I was most interested in is when threading.current_thread is called from the main greenlet. Is it expected to not be able to call threading.current_thread from code running in the main greenlet of an eventlet monkey patched application?

After thinking about it, I realized that is the problem I'm running into.

[temoto]

@melwitt why no, current_thread works from anywhere and returns threading.Thread compatible object.

[melwitt]

It does ... I'm not sure whether what I'm about to say will make sense, but what I'm referring to is the fact that when called from the main greenlet, current_thread will return a native threading.Thread object and not an object unique to the greenlet. The native object is OK as long as spawn_n is not called from that main greenlet but if it is, the greenlet created by spawn_n will return the same native object as the main greenlet if current_thread is called within it, making the main greenlet and the spawned greenlet appear to be the "same thread" to callers of current_thread. This is the trouble in the scenario I'm looking at currently.

I did some testing and found that the issues do indeed disappear when I replace all spawn_n with spawn. Even though current_thread is still being called in code running in the main greenlet, no other spawned green thread will return the same object as the main greenlet (as there are no other plain greenlets), so there are no problems with two different threads appearing to be the same threading.Thread compatible object when current_thread is used.

Hope that makes sense. Either way, thank you for the helpful guidance.

[temoto]

@melwitt I understand the inconsistency, it's a shame. I don't have a fast solution though. Sorry.

This would be a very weird workaround crutch:

import eventlet
eventlet.spawn_n = eventlet.spawn
eventlet.convenient.spawn_n = eventlet.spawn
eventlet.monkey_patch()

wouldn't recommend running it in production with random other libraries set, which could depend on spawn_n specifics, unlikely.

[temoto]

It's somewhat redundant to call current_thread() from main thread, maybe that's why this issue wasn't reported years ago.

[melwitt]

Thank you @temoto, that all makes sense.

Meanwhile, I found something else interesting. I tried a local test where I replaced all use of spawn_n() with spawn() in the application I'm working with and found another way spawn_n() can be called indirectly. There is yet a different library being used by the app which creates a thread.Thread to do something and when start() is called on it, eventually start_new_thread() is called and the eventlet version of start_new_thread() calls spawn_n() [1] to create the greenthread. This means if anything running in that thread.Thread calls current_thread(), it will get the same native thread object. Not sure if that is expected also, that eventlet thread.Thread does not work with threading.current_thread().

[1] https://github.com/eventlet/eventlet/blob/v0.32.0/eventlet/green/thread.py#L72