-
-
Notifications
You must be signed in to change notification settings - Fork 31.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
threading.Lock.acquire(timeout) should use sem_clockwait(CLOCK_MONOTONIC) #85876
Comments
The timeout for threading.Lock, threading.Condition, etc, is not using a monotonic clock — it is affected if the system time (realtime clock) is set. The attached program can be used to show the problem. It is expected to print "Took 2.000 s" repeatedly, but if run with permissions to set the system time, it prints: (the 1.6 s time can be explained by NTP correcting the clock) There are already a number of closed bugs for this and related issues: bpo-23428, bpo-31267, bpo-35747. This happens in Python 3.7.7 (ARM32, Yocto Warrior), Python 3.8.2 (AMD64, Ubuntu Linux 20.04) and Python v3.9.0rc1 (AMD64, Ubuntu Linux 20.04). |
bpo-23428 modified the pthread implementation of conditional variable to use pthread_condattr_setclock(&ca, CLOCK_MONOTONIC) is available: commit 001fee1. The change should be part of Python 3.8. What is your sys.thread_info value? Example on Fedora 32 with Python 3.8.5: >>> sys.thread_info
sys.thread_info(name='pthread', lock='semaphore', version='NPTL 2.31') Sadly, the semaphore implementation doesn't use monotonic clock. See glibc issues: |
sys.thread_info = sys.thread_info(name='pthread', lock='semaphore', version='NPTL 2.31') on my system. Looking at the source I think the semaphore implementation will be used on all modern Linux systems. In my tests it works as expected on a Macintosh (3.8.5 with lock='mutex+cond') and also if I force a Linux build to use the mutex+cond implementation by defining HAVE_BROKEN_POSIX_SEMAPHORES. Doesn't look like those glibc and Linux bug reports will get any attention anytime soon. I will find a workaround instead :-/ |
glibc v2.30 onwards provides sem_clockwait which can wait on either CLOCK_MONOTONIC or CLOCK_REALTIME. I failed to notice that https://sourceware.org/bugzilla/show_bug.cgi?id=14717 existed until today. :( |
On Unix, PyCOND_TIMEDWAIT() is implemented with pthread_cond_timedwait(). If pthread_condattr_setclock() is available, it uses CLOCK_MONOTONIC. Otherwise, it uses CLOCK_REALTIME. The glibc 2.30 adds pthread_cond_clockwait() which could be used to use CLOCK_MONOTONIC. But if pthread_cond_clockwait() is available (glibc 2.30 or newer), it expects that pthread_condattr_setclock() is also available. So I'm not sure that it's worth it to change PyCOND_TIMEDWAIT(). See the _PyThread_cond_after() function which computes an absolute timestamp (timespec) from a relative timeout in microseconds. |
vstinner wrote:
That's correct. The only time that pthread_cond_clockwait is essential is if you don't know which clock you're going to need to use for the wait at the time of condition variable creation. (This is true for a generic condition variable wrapper that can be passed absolute timeouts using different clocks like the C++ std::condition_variable.) However, sem_clockwait is the only way to wait on a semaphore using CLOCK_MONOTONIC, so it is worth using that rather than sem_timedwait if it's available. Similarly for pthread_mutex_clocklock. |
With sem_clockwait(CLOCK_MONOTONIC) on Fedora 34 (glibc-2.33-20.fc34.x86_64, Linux kernel 5.13.19-200.fc34.x86_64), time_test.py now works as expected: $ sudo ./python time_test.py
Took 2.000 s
Took 2.000 s
Took 2.000 s
Took 2.000 s
Took 2.000 s
Took 2.000 s
(...) |
Fixed bpo-12822 modified threading.Condition.wait(timeout) to use a monotonic clock: use pthread_condattr_setclock(CLOCK_MONOTONIC). bpo-23428, bpo-31267 and bpo-35747 are duplicates of bpo-12822. This issue is about threading.Lock.acquire(timeout) which has a different implementation. |
Jonas Norling: Thanks for the bug report! It's now fixed in 3.9, 3.10 and main branches. |
Using a system clock causes issues when it's being updated by the system administrator, NTP, Daylight Saving Time, or anything else. Sadly, on Windows, the monotonic clock resolution is around 15.6 ms. Example: python#85876
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: