_thread.LockType: Optimize lock deletion, acquisition of uncontested lock and release of lock.

[kristjanvalur]

BPO	15124
Nosy	@pitrou, @kristjanvalur
Files	lock.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-06-24.20:18:42.343>
created_at = <Date 2012-06-21.14:11:39.442>
labels = ['interpreter-core', 'performance']
title = '_thread.LockType: Optimize lock deletion, acquisition of uncontested lock and release of lock.'
updated_at = <Date 2012-06-24.20:18:42.342>
user = 'https://github.com/kristjanvalur'

bugs.python.org fields:

activity = <Date 2012-06-24.20:18:42.342>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2012-06-24.20:18:42.343>
closer = 'pitrou'
components = ['Interpreter Core']
creation = <Date 2012-06-21.14:11:39.442>
creator = 'kristjan.jonsson'
dependencies = []
files = ['26066']
hgrepos = []
issue_num = 15124
keywords = ['patch']
message_count = 10.0
messages = ['163335', '163348', '163388', '163404', '163406', '163408', '163411', '163425', '163455', '163844']
nosy_count = 4.0
nosy_names = ['pitrou', 'kristjan.jonsson', 'python-dev', 'sbt']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue15124'
versions = ['Python 3.3']

[kristjanvalur]

_thread.LockType contains code for sanity checking when the lock is released, and for unlocking it when it is deleted. Both operations involve actually try-lock operations that are costly. Use a flag on the log to keep track of its locking state instead.

Also, acquiring a lock now first tries a try-aqcuire without releasing the GIL, same as _thread.RLock(). This is done by moving logic from RLock.acquire to acquire_timed() so that both locks benefit.

Improvement, on a 64 bit windows machine:
Before:
D:\pydev\hg\cpython3\PCbuild\amd64>.\python.exe -m timeit -s "from _thread import allocate_lock" "allocate_lock()"
1000000 loops, best of 3: 0.725 usec per loop
D:\pydev\hg\cpython3\PCbuild\amd64>.\python.exe -m timeit -s "from _thread import allocate_lock; l=allocate_lock()" "l.acquire();l.release()"
1000000 loops, best of 3: 0.315 usec per loop

After:
D:\pydev\hg\cpython3\PCbuild\amd64>.\python.exe -m timeit -s "from _thread import allocate_lock" "allocate_lock()"
1000000 loops, best of 3: 0.691 usec per loop
D:\pydev\hg\cpython3\PCbuild\amd64>.\python.exe -m timeit -s "from _thread import allocate_lock; l=allocate_lock()" "l.acquire();l.release()"
1000000 loops, best of 3: 0.294 usec per loop

This amounts to 5% and 7% improvement, respectively.

[sbt]

On 32 bit linux in a VM I get

BEFORE
allocation 0.125
acquire/release 0.434

AFTER
allocation 0.109 (-13%)
acquire/release 0.346 (-20%)

[kristjanvalur]

Interesting, I need to see what is the reason for that. Possibly, sem_* primitives used there are so cheap that they don't matter.

[kristjanvalur]

I cannot repro. On my ubuntu virtualbox I get:

before:
kristjan@kristjan-VirtualBox:/cpython$ ./python -m timeit -s "from _thread import allocate_lock" "allocate_lock()"
10000000 loops, best of 3: 0.0768 usec per loop
kristjan@kristjan-VirtualBox:/cpython$ ./python -m timeit -s "from _thread import allocate_lock;l=allocate_lock()" "l.acquire();l.release()"
1000000 loops, best of 3: 0.209 usec per loop

after:
kristjan@kristjan-VirtualBox:/cpython$ ./python -m timeit -s "from _thread import allocate_lock" "allocate_lock()"
10000000 loops, best of 3: 0.069 usec per loop
kristjan@kristjan-VirtualBox:/cpython$ ./python -m timeit -s "from _thread import allocate_lock;l=allocate_lock()" "l.acquire();l.release()"
10000000 loops, best of 3: 0.182 usec per loop

I did this a few times with consistent results. But virtual machines aren't the most stable of beasts. Perhaps someone with a proper unix box can test it?

[pitrou]

64-bit Linux:

./python -m timeit -s "from _thread import allocate_lock" "allocate_lock()"
-> before: 0.0712 usec per loop
-> after: 0.0556 usec per loop

./python -m timeit -s "from _thread import allocate_lock;l=allocate_lock()" "l.acquire();l.release()"
-> before: 0.177 usec per loop
-> after: 0.168 usec per loop

So there's a small improvement on artificial benchmarks.

[kristjanvalur]

well, I'm thinking for example about threading.Condition, where Lock objects are being created and erased willy nilly.
Also, locks are normally uncontested. The unconstested case is the one to optimize for. As such, the test cases are not artificial at all.

The implementers of RLock in _threadingmodule obviously thought this was important, I'm merely bringing that same benefit to the Lock object.

[pitrou]

Also, locks are normally uncontested. The unconstested case is the
one to optimize for. As such, the test cases are not artificial at
all.

Ok. I don't have any problem with the patch in principle (I didn't read
it in detail). I was just a bit surprised that you cared so much about
performance of lock allocation.

If you want the patch to go in 3.3, you will have to commit it before
the beta release, which is very soon (this week-end I think).

[kristjanvalur]

Lock release (the silly saninty check) and acquisition of an uncontested lock.

I was waiting for that other windows lock stuff to go in before doing this, because I thought they might actually overlap in implementation, which they didn't.

I'll check this in later. Meanwhile look at the other, slightly more contentious patch, issue bpo-15139.

(condition variables are the whole reason I've been messing around with locks of late, since I had to make sure my threadpool-related HTTP requests on the PS3 performed adequately)

[python-dev]

New changeset dfc7fd24983a by Kristjan Valur Jonsson in branch 'default':
Issue bpo-15124: Optimize _thread.LockType deletion and acquisition when
http://hg.python.org/cpython/rev/dfc7fd24983a

[pitrou]

Closing as fixed.