Document atomic operations on builtin types

[dimaqq]

BPO	25343
Nosy	@rhettinger, @terryjreedy, @bitdancer, @dimaqq

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2015-10-08.15:43:06.061>
labels = ['type-feature', 'docs']
title = 'Document atomic operations on builtin types'
updated_at = <Date 2020-09-11.22:17:49.770>
user = 'https://github.com/dimaqq'

bugs.python.org fields:

activity = <Date 2020-09-11.22:17:49.770>
actor = 'brett.cannon'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2015-10-08.15:43:06.061>
creator = 'Dima.Tisnek'
dependencies = []
files = []
hgrepos = []
issue_num = 25343
keywords = []
message_count = 10.0
messages = ['252545', '252559', '252561', '252575', '252646', '252649', '252651', '252654', '252665', '252672']
nosy_count = 5.0
nosy_names = ['rhettinger', 'terry.reedy', 'r.david.murray', 'docs@python', 'Dima.Tisnek']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue25343'
versions = ['Python 3.6']

[dimaqq]

Please document what builtin type operations are actually atomic.
For example, what set() operations are atomic?

(There are some blogs / tutorials online, but information is outdated and not authoritative)

[brettcannon]

We actually don't have any guarantees written down because we have never formalized them. It was discussed at the PyCon language summit this past year -- https://lwn.net/Articles/640177/ -- but it didn't lead to anyone writing a proposal to formalize the memory model.

[bitdancer]

I wonder...personally I prefer to program in asyncio style rather than threading style, where one doesn't have to worry about atomicity. Maybe Python shouldn't make any atomicity guarantees.

[rhettinger]

what set() operations are atomic?

The language doesn't make any guarantees about set operation atomicity. Different implementations such as PyPy, Jython, and IronPython are free to make different choices than CPython. In general, users should make no assumptions about atomicity unless explicitly documented and tested. The wise course of action is to use mutexes when there is any doubt.

FWIW, it is difficult to make blanket statements about the methods on sets because the atomicity depends on the objects looked up or stored in the sets rather than the set itself. Aside from trivial calls to __sizeof__ and __len__, most set methods potentially call __hash__ or __eq__ on the set elements either of which could make a callback into pure python code. Likewise, any reference count decrement can potentially make a callback as well.

[terryjreedy]

This question has been asked multiple times before. I think it should be documented that as far as the language goes, there is no answer. Raymond's answer is a start. Dima, where would you expect to find such a disclaimer (other than in the FAQ)?

[dimaqq]

Ideally I'd like 2 sources:

1. a whole section on atomic operations in language and CPython implementation

2. annotation of standard library methods, e.g.:
   set().add(element) [atomic] or [CPython: atomic()]
   () assuming basic types, note about what custom methods would break this guarantee; the reference could be to 1.

[bitdancer]

I think what Terry was asking was, where would you expect to see the disclaimer that *no* operations are guaranteed to be atomic? That's what we're inclining toward (though we'll probably need a signoff from Guido).

[dimaqq]

To clarify, Python language disclaimer can be in the general atomic operations or multithreading section.

What I'd really like to see is documented, practical CPython and stdlib behaviour.

I'm under the impression that there's quite a bit of code out there that relies on what's actually atomic in CPython.

d.

[bitdancer]

I think you are correct, and I wouldn't be surprised if there is some in the stdlib as well.

[terryjreedy]

Yes, I was asking where to put the disclaimer. The thread docs would be approriate if there is nothing already.

Guido has been very reluctant to put any performance guarantees in the language reference. I believe he said that O(f(n)) info even for CPython should be in the wiki -- and taken as a guideline, not a iron guarantee.

Further discussion might be better directed to python-ideas, after a search of the pydev and python-ideas archives (most easily done with the gmane mirrors, I believe).