Document better what happens on releasing an unacquired lock

[birkenfeld]

BPO	14502
Nosy	@birkenfeld, @vsajip, @pitrou, @bitdancer, @asvetlov, @sandrotosi, @akheron, @JimJJewett

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-10-13.08:24:35.607>
created_at = <Date 2012-04-05.07:06:56.322>
labels = ['docs']
title = 'Document better what happens on releasing an unacquired lock'
updated_at = <Date 2013-10-13.08:24:35.606>
user = 'https://github.com/birkenfeld'

bugs.python.org fields:

activity = <Date 2013-10-13.08:24:35.606>
actor = 'georg.brandl'
assignee = 'docs@python'
closed = True
closed_date = <Date 2013-10-13.08:24:35.607>
closer = 'georg.brandl'
components = ['Documentation']
creation = <Date 2012-04-05.07:06:56.322>
creator = 'georg.brandl'
dependencies = []
files = []
hgrepos = []
issue_num = 14502
keywords = []
message_count = 22.0
messages = ['157544', '157547', '157548', '157549', '157623', '157625', '157626', '157643', '157652', '157654', '157678', '157680', '157693', '157715', '157827', '157830', '157877', '157883', '157884', '157901', '162417', '199672']
nosy_count = 10.0
nosy_names = ['georg.brandl', 'vinay.sajip', 'pitrou', 'r.david.murray', 'asvetlov', 'sandro.tosi', 'docs@python', 'python-dev', 'petri.lehtinen', 'Jim.Jewett']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue14502'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']

[birkenfeld]

From docs@python.org:

"""
I recently ran into a situation where I could not be certain that a lock
was currently in the acquired state. I checked the documentation to
determine what would happen if I attempted to release a lock that was
already released, and saw an ominous warning of "Do not call this method
when the lock is unlocked."

Needing to know what would happen, I cautiously tested it out. I half
expected my computer to explode as I released a lock for the second
time, but was pleased to see it raise a 'thread.error' exception which
could be caught and handled.

I generally expect the documentation to tell me what will happen if I do
something invalid. In this case the documentation should indicate that a
thread.error will be raised if you release an unlocked lock.
"""

I agree: if we know that a ThreadError will always be raised in this instance, we should document it as such.

[sandrotosi]

On Thu, Apr 5, 2012 at 09:06, Georg Brandl <report@bugs.python.org> wrote:

I agree: if we know that a ThreadError will always be raised in this instance, we should document it as such.

I've already prepared a small patch for that (every supported release
has a different exception raised..) I'll be fixing it this evening at
home.

[birkenfeld]

What different exceptions are they? Note that thread.error == _thread.error == threading.ThreadError. The docs should always use the last one (ThreadError).

[birkenfeld]

Ah, and I missed that apparently on 3.3, _thread.Error is aliased to RuntimeError. In that case you should use RuntimeError of course :)

[python-dev]

New changeset efeca6ff2751 by Sandro Tosi in branch '2.7':
Issue bpo-14502: release() and unlocked lock generates a ThreadError
http://hg.python.org/cpython/rev/efeca6ff2751

New changeset acea9d95a6d8 by Sandro Tosi in branch '3.2':
Issue bpo-14502: release() and unlocked lock generates a ThreadError
http://hg.python.org/cpython/rev/acea9d95a6d8

New changeset c10a0f93544e by Sandro Tosi in branch 'default':
Issue bpo-14502: merge with 3.2
http://hg.python.org/cpython/rev/c10a0f93544e

[jimjjewett]

At least put the information inside some disclaimers about "normally"; even the stdlib has some fake locks that let you release a lock someone else holds. (I think I found them in in workarounds for threading not being available, such as the dummy_* modules, but still, it is possible.)

[pitrou]

At least put the information inside some disclaimers about "normally";
even the stdlib has some fake locks that let you release a lock
someone else holds.

Not sure what you're talking about. The doc patch is about unacquired
locks, not locks that someone else (another thread) holds.

Indeed the standard Lock object (but not the RLock) does allow releasing
from another thread. It's a feature (which makes it serve as a binary
semaphore).

[jimjjewett]

On Thu, Apr 5, 2012 at 5:38 PM, Antoine Pitrou <report@bugs.python.org> wrote:

Antoine Pitrou <pitrou@free.fr> added the comment:

Not sure what you're talking about. The doc patch is about unacquired
locks, not locks that someone else (another thread) holds.

Isn't one common reason for not being able to acquire a lock that
someone else was already holding it?

[python-dev]

New changeset 068a614e9d97 by Sandro Tosi in branch 'default':
Issue bpo-14502: it's RuntimeError on 3.3
http://hg.python.org/cpython/rev/068a614e9d97

[pitrou]

On Thu, Apr 5, 2012 at 5:38 PM, Antoine Pitrou <report@bugs.python.org> wrote:
> Antoine Pitrou <pitrou@free.fr> added the comment:

> Not sure what you're talking about. The doc patch is about unacquired
> locks, not locks that someone else (another thread) holds.

Isn't one common reason for not being able to acquire a lock that
someone else was already holding it?

We're talking about *releasing* an (un)acquired lock, not acquiring it
again...

[jimjjewett]

On Fri, Apr 6, 2012 at 5:57 AM, Antoine Pitrou <report@bugs.python.org> wrote:

Antoine Pitrou <pitrou@free.fr> added the comment:

> > Not sure what you're talking about. The doc patch is about unacquired
> > locks, not locks that someone else (another thread) holds.

> Isn't one common reason for not being able to acquire a lock that
> someone else was already holding it?

We're talking about *releasing* an (un)acquired lock, not acquiring it
again...

Right, but I thought the original motivation was concern over a race
condition in the lock acquisition.

    lock.acquire()
    try:                    # What if something happens here, during
try setup?  Leak?
        foo()
    finally:
        lock.release()

vs

try:
    lock.acquire()
    foo()
finally:
    lock.release()               # But what if the acquire failed?

-jJ

[bitdancer]

It doesn't matter *how* you get to the situation where you are releasing a lock that hasn't been acquired, the point is to document what actually happens when you do the release. And just yesterday I needed to know this, since I have a lock that may or may not be currently held when I release it, and now I know I can just catch RuntimeError in that case.

[jimjjewett]

I have a lock that may or may not be currently held when I release it,
and now I know I can just catch RuntimeError in that case.

Only if you're willing to make assumptions about the threading model and the source of locks. And I fear the current change overpromises.

For example, the LockType from _dummy_thread raises an error not based on RuntimeError, and has comments suggesting it might stop raising entirely. I believe I have seen other Lock-emulation code which also does not raise an error, though the closest I can come to finding it right now is logging_releaseLock() when the import of either _thread or threading failed.

Starting with http://hg.python.org/cpython/file/acea9d95a6d8/Doc/library/threading.rst

I would prefer to change to following two sentences:

If an attempt is made to release an unlocked lock, a :exc:\`RuntimeError\` will be raised.

...
When invoked on an unlocked lock, a :exc:`ThreadError` is raised.

in any of the following ways:

(a) Change "will be"/"is" --> "may be", so it isn't promised:

If an attempt is made to release an unlocked lock, a :exc:\`RuntimeError\` may be raised.

...
When invoked on an unlocked lock, a :exc:`ThreadError` may be raised.

(b) Clarify that it is implementation-specific

If an attempt is made to release an unlocked \_thread.lock, a :exc:\`RuntimeError\` will be raised.

...
When invoked on an unlocked _thread.lock, a :exc:`ThreadError` is raised.

(and add to the caveats)
Locks provided by other modules may have slightly different behavior, particularly when an an operation fails. For example, unlocking without first acquiring may raise a different error, or may not raise at all.

(c) Clarify that alternatives are buggy (and fix those in the stdlib)
If an attempt is made to release an unlocked lock, a :exc:`RuntimeError` will be raised.
...
When invoked on an unlocked lock, a :exc:`ThreadError` is be raised.

(and add to the caveats)
Historically, many Locks have followed a slightly different contract, particularly when an an operation fails. For example, unlocking without first acquiring might raise a different error, or might not even raise at all.

[bitdancer]

I, on the other hand, would prefer if it were made part of the API contract that an error is raised, and to fix any stdlib implementations *of that API* that don't conform to that. (That is, locks from other modules may well not follow that API, and their documentation should cover their API.)

[jimjjewett]

On Fri, Apr 6, 2012 at 10:32 PM, R. David Murray <report@bugs.python.org> wrote:

R. David Murray <rdmurray@bitdance.com> added the comment:

I, on the other hand, would prefer if it were made part of the API contract that an
error is raised, and to fix any stdlib implementations *of that API* that don't conform
to that.  (That is, locks from other modules may well not follow that API, and their
documentation should cover their API.)

Do you consider it reasonable that all stdlib Locks follow that API,
and change to raise either RuntimeError or a subclass?

I don't feel comfortable declaring that (not even only for future
feature releases), but if you do, or Guido does, or ... etc ... I'll
submit patches for at least dummy_threading and logging.

-jJ

[bitdancer]

I think dummy_threading should be fixed (but only in 3.3, just in case it causes any backward compatibility issues with someone's code).

Logging I'd leave to Vinay to decide about.

I'm assuming that if any of the others devs nosy on this issue disagree with me that they will speak up :)

[vsajip]

Re. logging, logging._acquireLock and logging._releaseLock are not part of the public API and are undocumented at present. The case when _releaseLock does not raise an error is when threading couldn't be imported, so the _lock variable is None. I don't see the need for adding any documentation for this.

Logging doesn't use dummy_thread: if threading isn't available, all lock acquisition and release operations become no-ops.

[jimjjewett]

Vinay,

The current question is what contract locks should follow, and whether
all locks should follow it. Would it be acceptable for
logging._releaseLock to raise a RuntimeError if the lock hadn't
previously been acquired? In other words, would it be acceptable to
replace the current None with a counter (and to note in comments that
it should be safe from race conditions because it is only used when
threading isn't available).

-jJ

[pitrou]

The current question is what contract locks should follow, and whether
all locks should follow it. Would it be acceptable for
logging._releaseLock to raise a RuntimeError if the lock hadn't
previously been acquired?

I don't see the point of this discussion. We are talking about
threading.Lock (and, possibly, multiprocessing.Lock), not every lock API
under the sun. Especially when it's a private API...

[birkenfeld]

Agreed. Jim, I think you're trying to get consistency where none is required.

[akheron]

The docs of 2.7 and 3.2 still first say that RuntimeError is raised, and then that a ThreadError is raised:

...
If an attempt is made to release an unlocked lock, a RuntimeError
will be raised.

...

Lock.release()
    ...
    When invoked on an unlocked lock, a ThreadError is raised.

In 2.7 and 3.2, ThreadError is not a RuntimeError, so this is wrong.

[birkenfeld]

This is now fixed for 2.7 (see bpo-15829); no fix needed for 3.3+.