Intersection of dict view with iterator returns empty set

[serhiy-storchaka]

BPO	38210
Nosy	@rhettinger, @vstinner, @ambv, @serhiy-storchaka, @corona10
PRs	bpo-38210: Remove intersection operation with empty set #16602 bpo-38210: Fix compiler warning in dictobject.c #16611

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-10-06.11:29:41.769>
created_at = <Date 2019-09-18.07:10:06.945>
labels = ['interpreter-core', 'type-bug', '3.9']
title = 'Intersection of dict view with iterator returns empty set'
updated_at = <Date 2021-07-15.04:01:41.293>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2021-07-15.04:01:41.293>
actor = 'corona10'
assignee = 'none'
closed = True
closed_date = <Date 2019-10-06.11:29:41.769>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2019-09-18.07:10:06.945>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 38210
keywords = ['patch']
message_count = 8.0
messages = ['352705', '354033', '354034', '354045', '354047', '354072', '397494', '397523']
nosy_count = 5.0
nosy_names = ['rhettinger', 'vstinner', 'lukasz.langa', 'serhiy.storchaka', 'corona10']
pr_nums = ['16602', '16611']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue38210'
versions = ['Python 3.9']

[serhiy-storchaka]

In 3.8:

>>> {1: 2, 3: 4}.keys() & {1, 2}
{1}

In master:

>>> {1: 2, 3: 4}.keys() & iter([1, 2])
set()

The behavior was changed in bpo-27575.

[serhiy-storchaka]

New changeset c38e725 by Serhiy Storchaka (Dong-hee Na) in branch 'master':
bpo-38210: Fix intersection operation with dict view and iterator. (GH-16602)
c38e725

[serhiy-storchaka]

Thank you Dong-hee for your fix!

[rhettinger]

Please add a test for this regression.

[serhiy-storchaka]

It was added. Do you mean any special?

[vstinner]

New changeset d97f1ce by Victor Stinner in branch 'master':
bpo-38210: Fix compiler warning in dictobject.c (GH-16611)
d97f1ce

[ambv]

This caused an unintentional behavior change in the following code:

>>> {1: 2}.items() & {1: {2: 3}}.items()
set()

Before this change, Python 3.6 - 3.8 behaved like this instead:

>>> {1: 2}.items() & {1: {2: 3}}.items()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'dict'

Interestingly, this doesn't seem to have a negative effect on correctness as the silently omitted unhashable (k, v) pair is only omitted if it's different between the two dictionaries:

>>> {1: {2: 4}}.items() & {1: {2: 3}}.items()
set()
>>> {2: 1, 1: {2: 4}}.items() & {2: 1, 1: {2: 3}}.items()
{(2, 1)}

If it's the same, we still get an error in Python 3.9:

>>> {1: {2: 3}}.items() & {1: {2: 3}}.items()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'dict'
>>> {2: 1, 1: {2: 3}}.items() & {2: 1, 1: {2: 3}}.items()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'dict'

[corona10]

Interestingly, this doesn't seem to have a negative effect on correctness as the silently omitted unhashable

I think so too.

The error actually raises when adding the object into the set.

cpython/Objects/dictobject.c

Line 4384 in 818628c

if (PySet_Add(result, key)) {

Since the target object to be added is dynamically generated, I think that the issue does not need to be fixed.
Otherwise, we have to check that all objects are addable to set object before executing this operation but it looks harmful to performance.