Threading.current_thread does not change when using greenthreads?

[harlowja]

It appears that usage of threading.current_thread() does not change when the current greenthread is different. Was this intentional?

An example that creates a simple latch that uses the current_thread() information that doesn't work when the threading module is used due to this:

import eventlet
eventlet.monkey_patch()

import threading
import time

class CrappyLatch(object):
    def __init__(self):
        self._wait_for = []
        self._cond = threading.Condition()

    def append(self, identity):
        self._wait_for.append(identity)
        return identity

    def check_in(self, identity):
        with self._cond:
            try:
                self._wait_for.remove(identity)
            except ValueError:
                pass
            self._cond.notify_all()

    def wait(self):
        with self._cond:
            while self._wait_for:
                self._cond.wait()


def thing_to_do(latch, name):
    print("%s: %s" % (name, threading.current_thread()))
    time.sleep(0.5)
    latch.check_in(threading.current_thread())
    latch.wait()



latch = CrappyLatch()
threads = []
for i in range(0, 5):
    threads.append(latch.append(threading.Thread(target=thing_to_do,
                                                 args=(latch, i))))
for t in threads:
    t.start()
while threads:
    t = threads.pop()
    t.join()

When ran using threading.current_thread() the following (or similar) is output due to this current_thread update not happening...

0: <_MainThread(MainThread, started -136735040)>
1: <_MainThread(MainThread, started -136735040)>
2: <_MainThread(MainThread, started -136735040)>
3: <_MainThread(MainThread, started -136735040)>
4: <_MainThread(MainThread, started -136735040)>

This is obviously not right (since those are supposed to be different threads).

If instead you comment out the following two lines then this will work as expected:

#import eventlet
#eventlet.monkey_patch()

[harlowja]

Another example that has a similar issue:

https://review.openstack.org/#/c/138217/8/oslo_concurrency/lockutils.py (the ReaderWriterLock created there).

[vstinner]

threading.get_ident() is patched, but not threading.current_thread().

[vstinner]

threading.get_ident() is patched, but not threading.current_thread().

I'm wrong, I looked at eventlet/green/thread.py (thread), but I see current_thread() function in eventlet/green/threading.py (threading).

[jaketesler]

I may have found a super hacky fix to this issue. I want to emphasize that this is THOROUGHLY UNTESTED (and by that I mean it works for me but no guarantees to anyone else).

The code you have to change is in eventlet/eventlet/green/threading.py in the current_thread() function.

Lines 102-104 read as follows:

try:
    t = active[id(g)]
except KeyError:
...

Replace those lines (102-104) with the following code block:

try:
    if id(g) in active:
        t = active[id(g)]
    else:
        all_threads = patcher.patch_function(__import__('threading').enumerate)()
        all_idents = {th.ident: th for th in all_threads}
        t = all_idents[id(g)]

except KeyError:
...

You'll also need to comment out line 26 to be able to reuse patcher.

The _MainThread issue appears to arise from the active[] list not becoming populated with new eventlet-spawned threads (monkey-patched threading.Thread objects). Furthermore, I was unable to have the current_thread() return correct results from threading.enumerate() unless the enumerate() function was a) imported at runtime using the gross __import__() call and b) was hot-patched using patch_function().

AGAIN, I would like to emphasize that this is IN NO WAY a permanent fix, but as far as my testing goes, it's seems to be working for my setup.

[temoto]

@jaketesler thank you.

[strigazi]

It works for me, I tested it in an OpenStack project (OpenStack/Magnum) with the kubernetes client which uses multiprocessing. #147

[temoto]

@jaketesler @strigazi @harlowja Please, try this patch.

pip install https://github.com/eventlet/eventlet/archive/1d6d8924a9da6a0cb839b81e785f99b6ac219a0e.zip

[strigazi]

works for me :) with oslo.service, so I expect other openstack services to not be affected.

[strigazi]

@temoto Should we take this [0] in? Do you want to do more tests?

[0] 1d6d892

[temoto]

@strigazi yes, sorry for delay, I'll merge it.

[strigazi]

@temoto You will do a release too? xD

[aeon0]

Also solved my issues just to have another confirmation! Way to go fixing a > 3-year-old issue ; ) Thanks. A release would be cool.

[strigazi]

@temoto Apologies for pinging again. Eventlet is a very important dependency for OpenStack projects but not all projects suffer from this bug. The project I'm working needs this fix and I want to "sell" the release bump well before the next openstack release. Can we have a release? :)

[temoto]

Sure.

…

On Fri, May 4, 2018, 14:56 Spyros Trigazis ***@***.***> wrote: @temoto <https://github.com/temoto> Apologies for pinging again. Eventlet is a very important dependency for OpenStack projects but not all projects suffer from this bug. The project I'm working needs this fix and I want to "sell" the release bump well before the next openstack release. Can we have a release? :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#172 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABpsQLsiudbHhMPxjVr5p_EYuGXZbkNks5tvEHPgaJpZM4DEAB8> .

[temoto]

It's 0.23 on pypi.