Why isn't "in" called a comparison operation?

[wimglenn]

BPO	28617
Nosy	@freddrake, @rhettinger, @terryjreedy, @ncoghlan, @bitdancer, @serhiy-storchaka, @wimglenn, @miss-islington
PRs	bpo-28617 Fixed docs inaccuracies about the types that support membership tests #9086 [3.7] bpo-28617 Fixed docs inaccuracies about the types that support membership tests (GH-9086) #9171 [3.6] bpo-28617 Fixed docs inaccuracies about the types that support membership tests (GH-9086) #9172
Files	patch.diff: suggested changes to stdtypes documentation newpatch.diff: alternate proposal which corrects language but does not classify 'in' as a comparison.

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-01-09.00:56:03.443>
created_at = <Date 2016-11-04.20:36:00.006>
labels = ['type-feature', '3.7', 'docs']
title = 'Why isn\'t "in" called a comparison operation?'
updated_at = <Date 2020-01-09.00:56:03.442>
user = 'https://github.com/wimglenn'

bugs.python.org fields:

activity = <Date 2020-01-09.00:56:03.442>
actor = 'wim.glenn'
assignee = 'docs@python'
closed = True
closed_date = <Date 2020-01-09.00:56:03.443>
closer = 'wim.glenn'
components = ['Documentation']
creation = <Date 2016-11-04.20:36:00.006>
creator = 'wim.glenn'
dependencies = []
files = ['45359', '45360']
hgrepos = []
issue_num = 28617
keywords = ['patch']
message_count = 16.0
messages = ['280080', '280083', '280085', '280086', '280090', '280093', '280094', '280128', '280625', '282472', '306412', '325015', '325037', '325042', '325044', '325047']
nosy_count = 9.0
nosy_names = ['fdrake', 'rhettinger', 'terry.reedy', 'ncoghlan', 'r.david.murray', 'docs@python', 'serhiy.storchaka', 'wim.glenn', 'miss-islington']
pr_nums = ['9086', '9171', '9172']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue28617'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6', 'Python 3.7']

[wimglenn]

Regarding

https://docs.python.org/3/library/stdtypes.html#comparisons

There is a line at the bottom claiming:

Two more operations with the same syntactic priority, in and not in, are supported only by sequence types (below).

The claim is incorrect because in and not in are also supported by non-sequence types such as sets, mappings, etc for membership testing.

Is there any good reason why we don't include them in the table of comparison operations, and say that there are ten comparison operations in python? They do support comparison chaining in the same way:

    >>> 'x' in 'xy' in 'xyz'
    True
    >>> 0 in {0} in [{0}]
    True

[bitdancer]

It should really say "only by types that support iteration". They are not comparison operations, they are containment predicates.

[wimglenn]

Well, that wouldn't be true either. Because you can easily implement objects which support membership tests but don't support iteration.

[wimglenn]

I want to add that the grammar explicitly mentions them as comparisons

https://docs.python.org/3/reference/grammar.html

And they are also listed in the comparisons section of the expressions documentation

https://docs.python.org/3/reference/expressions.html#comparisons

So the docs seem to be contradicting themselves here.

[rhettinger]

At a grammar and implementation level, the "in" and "not in" operators are treated like comparisons (same precedence and opcodes), but at a semantic level, I concur with David Murray and don't think of these as being comparisons at all. Accordingly, I prefer the current presentation
and agree with David that the note should be revised to say "only by types that support iteration".

[wimglenn]

Perhaps it's better to call a spade a spade here - if they're implemented as comparisons, then why not document them as comparisons?

A colleague has mentioned one point that sets in and not in apart from the other comparisons in the table: comparisons are generally made between objects of the same type (with the exception of numbers). But membership "comparisons" are made between different types (with the exception of substring checks).

Here is an alternate patch which leaves the table alone, but corrects the inaccuracy in the note.

[freddrake]

"in" and "not in" are not comparisons, regardless of implementation mechanics (which could change).

They aren't really dependent on iteration, though they often correlate with iteration.

I'd rather see them described as "containment tests" or something similar.

[rhettinger]

newpatch.diff looks fine.

[terryjreedy]

"Two more operations with the same syntactic priority, in and not in, are supported ..."

could be changed to

The containment tests in and not in have the same priority as comparisons and are supported ..."

[wimglenn]

1 month later.. is newpatch.diff OK to merge or any further improvements needed? thanks

[ncoghlan]

Also see https://bugs.python.org/issue32055 regarding the prospect of bringing the implementation into line with the intuitive semantics, and preventing implicit comparison chaining for containment tests.

[serhiy-storchaka]

I think that "types that are :term:`iterable` or implement the :meth:`__contains__` method" is too low-level for this section.

In this section we tell about builtin types. From those the types that support in and not in are list, tuple, dict, set, frozenset, str, bytes, bytearray, memoryview, and iterator types. We can just enumerate them or use general word. Calling the "sequence types" is not correct, but fortunately all of them are iterable, so that we can say just "iterable types described below". Or enumerate categories of types, as they are named in the below subsections: "iterator types, sequence types, str, binary sequence types, set types and dict", with links to corresponding subsections.

[miss-islington]

New changeset 08bcf64 by Miss Islington (bot) (wim glenn) in branch 'master':
bpo-28617 Fixed docs inaccuracies about the types that support membership tests (GH-9086)
08bcf64

[miss-islington]

New changeset 3e648f8 by Miss Islington (bot) in branch '3.7':
bpo-28617 Fixed docs inaccuracies about the types that support membership tests (GH-9086)
3e648f8

[miss-islington]

New changeset 889f080 by Miss Islington (bot) in branch '3.6':
bpo-28617 Fixed docs inaccuracies about the types that support membership tests (GH-9086)
889f080

[serhiy-storchaka]

Humm, why the bot merges in the master branch?