New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix threading.Condition with monkey-patching on Python 3.3 and newer #187

Closed
wants to merge 3 commits into
from

Conversation

Projects
None yet
4 participants
@vstinner
Contributor

vstinner commented Jan 6, 2015

It fixes monkey-patching threading.Condition on Python 3.3 and newer:
fix issue #185.

@vstinner vstinner changed the title from Semaphore: handle negative timeout as blocking to Fix threading.Condition with monkey-patching on Python 3.3 and newer Jan 6, 2015

@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Jan 12, 2015

Contributor

Is it ok to change Semaphore.acquire() behaviour for negative timeout? Any issue with the backward compatibility? The idea is to have a behaviour to the Python stdlib.

Contributor

vstinner commented Jan 12, 2015

Is it ok to change Semaphore.acquire() behaviour for negative timeout? Any issue with the backward compatibility? The idea is to have a behaviour to the Python stdlib.

openstack-gerrit pushed a commit to openstack/oslo.messaging that referenced this pull request Jan 21, 2015

Port zmq driver to Python 3
With eventlet 0.16, it becomes possible to run Oslo Messaging tests on
Python 3 with eventlet.

This change ports the zmq driver to Python 3:

* encode the topic explicitly to UTF-8
* use a list comprehension instead of map() to also get a list
  on Python 3 (not a generator)

The following eventlet change is needed to run tests:
eventlet/eventlet#187

Related eventlet issue:
eventlet/eventlet#185

I will propose a different change to enable tests with eventlet enabled
when a release of eventlet including this fix will be available.

Change-Id: Ic8fec515cfa757e08ffb9604e3bfb2e87d08f3d8

openstack-gerrit added a commit to openstack/openstack that referenced this pull request Jan 21, 2015

Updated openstack/openstack
Project: openstack/oslo.messaging  ae009a4ed99caa414104bc06829ac5a4f4cc9d2c

Port zmq driver to Python 3

With eventlet 0.16, it becomes possible to run Oslo Messaging tests on
Python 3 with eventlet.

This change ports the zmq driver to Python 3:

* encode the topic explicitly to UTF-8
* use a list comprehension instead of map() to also get a list
  on Python 3 (not a generator)

The following eventlet change is needed to run tests:
eventlet/eventlet#187

Related eventlet issue:
eventlet/eventlet#185

I will propose a different change to enable tests with eventlet enabled
when a release of eventlet including this fix will be available.

Change-Id: Ic8fec515cfa757e08ffb9604e3bfb2e87d08f3d8

openstack-gerrit added a commit to openstack/openstack that referenced this pull request Jan 21, 2015

Updated openstack/openstack
Project: openstack/oslo.messaging  ae009a4ed99caa414104bc06829ac5a4f4cc9d2c

Port zmq driver to Python 3

With eventlet 0.16, it becomes possible to run Oslo Messaging tests on
Python 3 with eventlet.

This change ports the zmq driver to Python 3:

* encode the topic explicitly to UTF-8
* use a list comprehension instead of map() to also get a list
  on Python 3 (not a generator)

The following eventlet change is needed to run tests:
eventlet/eventlet#187

Related eventlet issue:
eventlet/eventlet#185

I will propose a different change to enable tests with eventlet enabled
when a release of eventlet including this fix will be available.

Change-Id: Ic8fec515cfa757e08ffb9604e3bfb2e87d08f3d8
@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Jan 23, 2015

Contributor

Ping? Is someone available to review this change? Is it ok?

Contributor

vstinner commented Jan 23, 2015

Ping? Is someone available to review this change? Is it ok?

@jstasiak

This comment has been minimized.

Show comment
Hide comment
@jstasiak

jstasiak Feb 12, 2015

Contributor

@haypo I'll review this tomorrow, sorry for the wait

Contributor

jstasiak commented Feb 12, 2015

@haypo I'll review this tomorrow, sorry for the wait

@jstasiak

This comment has been minimized.

Show comment
Hide comment
@jstasiak

jstasiak Feb 13, 2015

Contributor

@haypo The negative timeout value seems to be working already, do you have a test case showing a failure?

Contributor

jstasiak commented Feb 13, 2015

@haypo The negative timeout value seems to be working already, do you have a test case showing a failure?

@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Feb 18, 2015

Contributor

@haypo The negative timeout value seems to be working already, do you have a test case showing a failure?

My first commit changes the behaviour of eventlet.semaphore.Semaphore.acquire() when you pass a negative timeout. Before, Semaphore.acquire(timeout=-1) was non-blocking. After, Semaphore.acquire(timeout=-1) becomes blocking, same behaviour than Python 3 lock.

In Python 2, threading.Lock.acquire() has no timeout parameter. So RLock only calls acquire() (on its internal lock) with one parameter: blocking.

In Python 3, threading.Lock.acquire() got a new timeout parameter: def acquire(blocking=True, timeout=None). RLock calls aquire() (on its internal lock) with two parameters: blocking and timeout.

In Python 3, threading.Lock.acquire(timeout=-1) blocks, whereas eventlet.semaphore.Semaphore.acquire(timeout=-1) is currently non-blocking.

eventlet monkey-patching is not supposed to modify the behaviour of patched functions. I wasn't the case in Python 2, since Lock.acquire() had no timeout parameter. It's now the case, since Python 3 also uses the second timeout parameter.

In short, my changes fix two different Python 3 bugs:

  • fix monkey-patching on threading.Lock to have the same behaviour on Lock.acquire(timeout=-1)
  • fix RLock, it didn't work with monkey-patching on Python 3.3+
Contributor

vstinner commented Feb 18, 2015

@haypo The negative timeout value seems to be working already, do you have a test case showing a failure?

My first commit changes the behaviour of eventlet.semaphore.Semaphore.acquire() when you pass a negative timeout. Before, Semaphore.acquire(timeout=-1) was non-blocking. After, Semaphore.acquire(timeout=-1) becomes blocking, same behaviour than Python 3 lock.

In Python 2, threading.Lock.acquire() has no timeout parameter. So RLock only calls acquire() (on its internal lock) with one parameter: blocking.

In Python 3, threading.Lock.acquire() got a new timeout parameter: def acquire(blocking=True, timeout=None). RLock calls aquire() (on its internal lock) with two parameters: blocking and timeout.

In Python 3, threading.Lock.acquire(timeout=-1) blocks, whereas eventlet.semaphore.Semaphore.acquire(timeout=-1) is currently non-blocking.

eventlet monkey-patching is not supposed to modify the behaviour of patched functions. I wasn't the case in Python 2, since Lock.acquire() had no timeout parameter. It's now the case, since Python 3 also uses the second timeout parameter.

In short, my changes fix two different Python 3 bugs:

  • fix monkey-patching on threading.Lock to have the same behaviour on Lock.acquire(timeout=-1)
  • fix RLock, it didn't work with monkey-patching on Python 3.3+
@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Feb 18, 2015

Contributor

FYI I failed to find a call to Semaphore.acquire() which uses the timeout parameter in eventlet source code. In eventlet, the timeout doesn't look to be used.

The timeout parameter is documented:
http://eventlet.net/doc/modules/semaphore.html#eventlet.semaphore.Semaphore.acquire

The behaviour on negative timeout is not documented.

Contributor

vstinner commented Feb 18, 2015

FYI I failed to find a call to Semaphore.acquire() which uses the timeout parameter in eventlet source code. In eventlet, the timeout doesn't look to be used.

The timeout parameter is documented:
http://eventlet.net/doc/modules/semaphore.html#eventlet.semaphore.Semaphore.acquire

The behaviour on negative timeout is not documented.

@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Mar 4, 2015

Contributor

Ping?

Contributor

vstinner commented Mar 4, 2015

Ping?

@temoto

This comment has been minimized.

Show comment
Hide comment
@temoto

temoto Mar 5, 2015

Member

@haypo I've rebased it on recent master, now tests.patcher_test:Threading.test_threading fails with timeout. Could you look at it? https://github.com/eventlet/eventlet/tree/semaphore_timeout

Member

temoto commented Mar 5, 2015

@haypo I've rebased it on recent master, now tests.patcher_test:Threading.test_threading fails with timeout. Could you look at it? https://github.com/eventlet/eventlet/tree/semaphore_timeout

@temoto temoto referenced this pull request Mar 5, 2015

Closed

disable patcher.monkey_patch #209

@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Mar 6, 2015

Contributor

@haypo I've rebased it on recent master,

Cool!

now tests.patcher_test:Threading.test_threading fails with timeout. Could you look at it? https://github.com/eventlet/eventlet/tree/semaphore_timeout

It's not a regression of recent changes in eventlet. I'm not sure, but I guess that I used Python 3.3 to work on my patch, whereas Python 3.4 requires extra changes. Python 3.4 adds a new threading.Thread._tstate_lock attribute which doesn't work with monkey-patching, because it relies on private C code which releases the new lock at thread exit (when its Python thread state is destroyed).

Well, see the pull request #211 for the details, I proposed a fix.

Contributor

vstinner commented Mar 6, 2015

@haypo I've rebased it on recent master,

Cool!

now tests.patcher_test:Threading.test_threading fails with timeout. Could you look at it? https://github.com/eventlet/eventlet/tree/semaphore_timeout

It's not a regression of recent changes in eventlet. I'm not sure, but I guess that I used Python 3.3 to work on my patch, whereas Python 3.4 requires extra changes. Python 3.4 adds a new threading.Thread._tstate_lock attribute which doesn't work with monkey-patching, because it relies on private C code which releases the new lock at thread exit (when its Python thread state is destroyed).

Well, see the pull request #211 for the details, I proposed a fix.

vstinner and others added some commits Jan 6, 2015

Fix threading.Condition with monkey-patching
For the Python implementation of threading.RLock because the new C
implementation of threading.RLock of Python 3.3 is not compatible with
eventlet monkey patching.

Fix the issue #185.
@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Mar 6, 2015

Contributor

The changeset "Disable the thread state lock" fixes the threading support on Python 3.4 when eventlet is used with monkey-patching.

Python 3.4 introduces threading.Thread._tstate_lock to fix a race condition, see:

I'm not really proud of this change: patching a Thread instance after its creation is not the most elegant fix.

I tried to create a Thread subclass in eventlet.green.threading, but I got a lot of new issues. If the subclass is declared in eventlet.green.threading, the parent Thread class uses functions of the original threading module, instead of using patched functions by eventlet. It's unclear to me how symbols are added and patched in modules. I noticed that the threading module of the standard library is imported multiple times. The implementation of monkey-patching is really complex :-/

Currently, the best solution is to patch Thread instance in the patched thread.start_new_thread() function. The problem is to retrieve the instance. My heuristic checks if the wrapped function is the bounded Thread.run() method, and I retrieve the instance from function.self.

I'm not yet sure that I really understood the problem. At least, "tox -e py34-selects" pass.

Contributor

vstinner commented Mar 6, 2015

The changeset "Disable the thread state lock" fixes the threading support on Python 3.4 when eventlet is used with monkey-patching.

Python 3.4 introduces threading.Thread._tstate_lock to fix a race condition, see:

I'm not really proud of this change: patching a Thread instance after its creation is not the most elegant fix.

I tried to create a Thread subclass in eventlet.green.threading, but I got a lot of new issues. If the subclass is declared in eventlet.green.threading, the parent Thread class uses functions of the original threading module, instead of using patched functions by eventlet. It's unclear to me how symbols are added and patched in modules. I noticed that the threading module of the standard library is imported multiple times. The implementation of monkey-patching is really complex :-/

Currently, the best solution is to patch Thread instance in the patched thread.start_new_thread() function. The problem is to retrieve the instance. My heuristic checks if the wrapped function is the bounded Thread.run() method, and I retrieve the instance from function.self.

I'm not yet sure that I really understood the problem. At least, "tox -e py34-selects" pass.

@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Mar 6, 2015

Contributor

I rewrite the first commit: in fact, we don't need to support negative timeout, we only need to accept timeout=-1 for Semaphore.accept(). If the timeout value is None or -1, the timeout is ignored, only the blocking parameter is used:

  • acquire(timeout=None)/acquire(timeout=-1) is the same than acquire(blocking=True): blocking
  • acquire(blocking=False, timeout=None)/acquire(blocking=False, timeout=-1) is the same than acquire(blocking=False): non-blocking
Contributor

vstinner commented Mar 6, 2015

I rewrite the first commit: in fact, we don't need to support negative timeout, we only need to accept timeout=-1 for Semaphore.accept(). If the timeout value is None or -1, the timeout is ignored, only the blocking parameter is used:

  • acquire(timeout=None)/acquire(timeout=-1) is the same than acquire(blocking=True): blocking
  • acquire(blocking=False, timeout=None)/acquire(blocking=False, timeout=-1) is the same than acquire(blocking=False): non-blocking
if timeout == -1:
timeout = None
if timeout is not None and timeout < 0:
raise ValueError("timeout value must be strictly positive")

This comment has been minimized.

@temoto

temoto Mar 7, 2015

Member

Exception says "strictly positive" ( x > 0 ), while code allows timeout=0 just fine.

@temoto

temoto Mar 7, 2015

Member

Exception says "strictly positive" ( x > 0 ), while code allows timeout=0 just fine.

This comment has been minimized.

@vstinner

vstinner Mar 23, 2015

Contributor

Exception says "strictly positive" ( x > 0 ), while code allows timeout=0 just fine.

Hum... yes... I copied code from the C implementation of threading.Lock, to try to be as close as possible to threading.Lock.

Example:

$ python3
Python 3.4.1 (default, Nov  3 2014, 14:38:10) 
>>> import threading
>>> l=threading.Lock()
>>> l.acquire(timeout=-2)
...
ValueError: timeout value must be strictly positive
>>> l.acquire(timeout=0)
True
>>> l.acquire(timeout=0)
False
@vstinner

vstinner Mar 23, 2015

Contributor

Exception says "strictly positive" ( x > 0 ), while code allows timeout=0 just fine.

Hum... yes... I copied code from the C implementation of threading.Lock, to try to be as close as possible to threading.Lock.

Example:

$ python3
Python 3.4.1 (default, Nov  3 2014, 14:38:10) 
>>> import threading
>>> l=threading.Lock()
>>> l.acquire(timeout=-2)
...
ValueError: timeout value must be strictly positive
>>> l.acquire(timeout=0)
True
>>> l.acquire(timeout=0)
False
@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Mar 23, 2015

Contributor

temodo wrote "Exception says "strictly positive" ( x > 0 ), while code allows timeout=0 just fine." but I copied the error message from Python.

Do you expect that I change my patch, or are you ok to mimick Python lock?

Contributor

vstinner commented Mar 23, 2015

temodo wrote "Exception says "strictly positive" ( x > 0 ), while code allows timeout=0 just fine." but I copied the error message from Python.

Do you expect that I change my patch, or are you ok to mimick Python lock?

@temoto

This comment has been minimized.

Show comment
Hide comment
@temoto

temoto Mar 23, 2015

Member

Quite surprised that Python code has this code/comment mismatch. Yeah, leave it same as stdlib.

Member

temoto commented Mar 23, 2015

Quite surprised that Python code has this code/comment mismatch. Yeah, leave it same as stdlib.

@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Apr 4, 2015

Contributor

Quite surprised that Python code has this code/comment mismatch. Yeah, leave it same as stdlib.

Oh, so did you review my patch? Does it look ok? Should I change something?

Contributor

vstinner commented Apr 4, 2015

Quite surprised that Python code has this code/comment mismatch. Yeah, leave it same as stdlib.

Oh, so did you review my patch? Does it look ok? Should I change something?

@temoto

This comment has been minimized.

Show comment
Hide comment
@temoto

temoto Apr 5, 2015

Member

LGTM, running last round of tests now.

Member

temoto commented Apr 5, 2015

LGTM, running last round of tests now.

@temoto

This comment has been minimized.

Show comment
Hide comment
@temoto

temoto Apr 5, 2015

Member

I took the liberty of sqashing PEP-8 fixes into your commits.

Member

temoto commented Apr 5, 2015

I took the liberty of sqashing PEP-8 fixes into your commits.

@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Apr 5, 2015

Contributor
Contributor

vstinner commented Apr 5, 2015

@temoto

This comment has been minimized.

Show comment
Hide comment
@temoto

temoto Apr 5, 2015

Member

It's merged into master 390e71d thank you!
Please test if it works and drop a line.

Member

temoto commented Apr 5, 2015

It's merged into master 390e71d thank you!
Please test if it works and drop a line.

@jstasiak

This comment has been minimized.

Show comment
Hide comment
@jstasiak

jstasiak Nov 8, 2015

Contributor

@hardys @temoto Can this issue be closed now?

Contributor

jstasiak commented Nov 8, 2015

@hardys @temoto Can this issue be closed now?

@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Nov 8, 2015

Contributor

@hardys @temoto Can this issue be closed now?

A change fixing this issue was merged, so yes, the issue can be closed.

Contributor

vstinner commented Nov 8, 2015

@hardys @temoto Can this issue be closed now?

A change fixing this issue was merged, so yes, the issue can be closed.

@vstinner vstinner closed this Nov 8, 2015

@jstasiak

This comment has been minimized.

Show comment
Hide comment
@jstasiak

jstasiak Nov 8, 2015

Contributor

Thanks @haypo!

Contributor

jstasiak commented Nov 8, 2015

Thanks @haypo!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment