Language reference does not clearly describe modern operand coercion

[ncoghlan]

BPO	39302
Nosy	@ncoghlan

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-01-11.08:41:52.195>
labels = ['type-feature', '3.8', '3.9']
title = 'Language reference does not clearly describe modern operand coercion'
updated_at = <Date 2020-01-11.08:41:52.195>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2020-01-11.08:41:52.195>
actor = 'ncoghlan'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2020-01-11.08:41:52.195>
creator = 'ncoghlan'
dependencies = []
files = []
hgrepos = []
issue_num = 39302
keywords = []
message_count = 1.0
messages = ['359788']
nosy_count = 1.0
nosy_names = ['ncoghlan']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue39302'
versions = ['Python 3.8', 'Python 3.9']

[ncoghlan]

While reviewing ISO-IECJTC1-SC22-WG23's latest draft of their Python security annex, I found a description of operand coercion that was based on the legacy coercion model described at https://docs.python.org/2.5/ref/coercion-rules.html

That's still the second highest link if you search for "Python operand coercion", while the highest link is this old, very brief, summary from Python in a Nutshell: https://www.oreilly.com/library/view/python-in-a/0596001886/ch04s05.html (still based on the old semantics where the inputs were coerced to a common type before calling the slot method, rather than giving the method direct access to the original operands).

The third highest link at least goes to PEP-208 (https://www.python.org/dev/peps/pep-0208/), which correctly describes the modern semantics, but it describes them in terms of the CPython C slot API, not the Python level special method APIs.

https://docs.python.org/3.7/reference/datamodel.html#emulating-numeric-types does technically provide the required information, but it's implicit in the description of the numeric arithmetic methods, rather than being clearly spelled out as a clear description of "Python operand coercion". (There are also some oddities around operand coercion for three-argument pow() that I'm going to file as their own issue)

https://docs.python.org/3/library/constants.html#NotImplemented references https://docs.python.org/3/library/numbers.html#implementing-the-arithmetic-operations which describes defining new numeric ABCs in a coercion-friendly way, but still doesn't spell out the operand precedence and coercion dance.

We could likely improve this situation by adding a new "Special method invocation" subject at the end of https://docs.python.org/3.7/reference/datamodel.html, moving the existing "Special method lookup" subsection under it, and then adding a second subsection called "Operand precedence and coercion".

That new subsection would then cover the basic principles of:

• for unary operands, there is no ambiguity
• for binary operands of the same type, only the forward operation is tried (it is assumed that if the forward operation doesn't work, the reflected one won't either)
• for binary operands where the type of the RHS is a subclass of the type of the LHS, the reflected operation is tried first (if it exists), followed by the forward operation if the reflected call does not exist or returns the NotImplemented singleton
• for binary operands of unrelated types, the forward operation is tried first (if it exists), followed by the reflected operation if the forward call does not exist or returns the NotImplemented singleton
• for ternary operands (i.e. 3 argument pow()), the behaviour is currently implementation defined (as the test suite doesn't enforce any particular behaviour, and what CPython does isn't obviously useful)

Other specific points to be covered would be:

• any argument coercion that occurs is up to the individual method implementations
• logical short-circuiting expressions (and, or, if/else) only call the equivalent of bool(expr)

While the corresponding reflected operations for the binary operators are covered in the documentation of the forward operations, it would also likely be worthwhile including a summary table in this new subsection of exactly which special methods support reflection, and what the reflected method names are.