bpo-36452: dictiter: track maximum iteration count

[thp]

Using: Python 3.8 (git commit ID: d5a5a33), the following code snippet:

a = {0: 0}

for i in a:
    del a[i]
    a[i+1] = 0
    print(i)

Prints the following output:

0
1
2
3
4

The reason for this seems to be the way the internal key list is managed and the "next" value in this list is retrieved. The amount of items seems to be related to USABLE_FRACTION(PyDict_MINSIZE).

Since cases where the dictionary size changes are detected with a RuntimeError, I would expect the invariant "the number of iterations is the len() of the dict at the time the iterator is created" to be enforced. Whether to raise a StopIteration instead or raising a RuntimeError is up for debate.

This pull request tries to detect this corner case and raise a RuntimeError instead (plus a unit test).

Note also that without the patch, the __length_hint__() of the iterator actually underflows:

a = {0: 0}
it = iter(a)
print('Length hint:', it.__length_hint__())
next(it)
print('Length hint:', it.__length_hint__())
del a[0]
a[1] = 0
next(it)
print('Length hint:', it.__length_hint__())

Which - on an AMD64 machine - prints:

Length hint: 1
Length hint: 0
Length hint: 18446744073709551615

https://bugs.python.org/issue36452

[the-knights-who-say-ni]

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed the PSF contributor agreement (CLA).

Unfortunately we couldn't find an account corresponding to your GitHub username on bugs.python.org (b.p.o) to verify you have signed the CLA (this might be simply due to a missing "GitHub Name" entry in your b.p.o account settings). This is necessary for legal reasons before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

You can check yourself to see if the CLA has been received.

Thanks again for your contribution, we look forward to reviewing it!