Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increased test coverage of django.utils.datastructures to 100%. #13494

Merged
merged 4 commits into from Oct 30, 2020

Conversation

ngnpope
Copy link
Member

@ngnpope ngnpope commented Oct 5, 2020

Supersedes #12442. I split out the tests for MultiValueDict and OrderedSet into two commits and added additional missing test coverage to these. Also in relation to MultiValueDict.update(), in that PR, Mads said:

I don't know how to trigger TypeError, and my attempts only triggered an AttributeError.

I came to the conclusion that it wasn't possible to trigger TypeError as the call to .items() was always going to trigger AttributeError for every other common type. Further, I noted that the behaviour of MultiValueDict.update() was inconsistent with and more strict than dict.update(). To my understanding, the main reason for overriding the .update() method is to a) ensure that the values are always lists, and b) special case to handle provision of another MultiValueDict instance.

So I added a commit that makes the behaviour as consistent with dict.update() as possible. A few exceptions are slightly differently worded due to the way dict.update() is implemented in C versus this Python implementation for MultiValueDict.update(), but they are comparable..

I've also attached a quick script I used to compare output of dict.update() and MultiValueDict.update() as we'll as before and after output:

Script to compare behaviour.
import sys
import traceback
from django.utils.datastructures import MultiValueDict

def test(values):
    for value in values:
        print(f'>>> dict().update({value!r})')
        try:
            result = dict().update(value)
        except Exception:
            traceback.print_exception(*sys.exc_info()[:-1], None)
        else:
            print(result)
    for value in values:
        print(f'>>> MultiValueDict().update({value!r})')
        try:
            result = MultiValueDict().update(value)
        except Exception:
            traceback.print_exception(*sys.exc_info()[:-1], None)
        else:
            print(result)

non_iterables = [None, True, False, 123, 123.45]
empty_iterables = ['', b'', (), [], set(), {}]
paired_iterables = [(('a', 1),), [('a', 1)], {('a', 1)}]
non_empty_iterables = ['123', b'123', (1, 2, 3), [1, 2, 3], {1, 2, 3}, {'a': 1}]

test(non_iterables)
test(empty_iterables)
test(paired_iterables)
test(non_empty_iterables)
Comparison before change in behavior.
>>> dict().update(None)
TypeError: 'NoneType' object is not iterable
>>> dict().update(True)
TypeError: 'bool' object is not iterable
>>> dict().update(False)
TypeError: 'bool' object is not iterable
>>> dict().update(123)
TypeError: 'int' object is not iterable
>>> dict().update(123.45)
TypeError: 'float' object is not iterable
>>> MultiValueDict().update(None)
AttributeError: 'NoneType' object has no attribute 'items'
>>> MultiValueDict().update(True)
AttributeError: 'bool' object has no attribute 'items'
>>> MultiValueDict().update(False)
AttributeError: 'bool' object has no attribute 'items'
>>> MultiValueDict().update(123)
AttributeError: 'int' object has no attribute 'items'
>>> MultiValueDict().update(123.45)
AttributeError: 'float' object has no attribute 'items'
>>> dict().update('')
None
>>> dict().update(b'')
None
>>> dict().update(())
None
>>> dict().update([])
None
>>> dict().update(set())
None
>>> dict().update({})
None
>>> MultiValueDict().update('')
AttributeError: 'str' object has no attribute 'items'
>>> MultiValueDict().update(b'')
AttributeError: 'bytes' object has no attribute 'items'
>>> MultiValueDict().update(())
AttributeError: 'tuple' object has no attribute 'items'
>>> MultiValueDict().update([])
AttributeError: 'list' object has no attribute 'items'
>>> MultiValueDict().update(set())
AttributeError: 'set' object has no attribute 'items'
>>> MultiValueDict().update({})
None
>>> dict().update((('a', 1),))
None
>>> dict().update([('a', 1)])
None
>>> dict().update({('a', 1)})
None
>>> MultiValueDict().update((('a', 1),))
AttributeError: 'tuple' object has no attribute 'items'
>>> MultiValueDict().update([('a', 1)])
AttributeError: 'list' object has no attribute 'items'
>>> MultiValueDict().update({('a', 1)})
AttributeError: 'set' object has no attribute 'items'
>>> dict().update('123')
ValueError: dictionary update sequence element #0 has length 1; 2 is required
>>> dict().update(b'123')
TypeError: cannot convert dictionary update sequence element #0 to a sequence
>>> dict().update((1, 2, 3))
TypeError: cannot convert dictionary update sequence element #0 to a sequence
>>> dict().update([1, 2, 3])
TypeError: cannot convert dictionary update sequence element #0 to a sequence
>>> dict().update({1, 2, 3})
TypeError: cannot convert dictionary update sequence element #0 to a sequence
>>> dict().update({'a': 1})
None
>>> MultiValueDict().update('123')
AttributeError: 'str' object has no attribute 'items'
>>> MultiValueDict().update(b'123')
AttributeError: 'bytes' object has no attribute 'items'
>>> MultiValueDict().update((1, 2, 3))
AttributeError: 'tuple' object has no attribute 'items'
>>> MultiValueDict().update([1, 2, 3])
AttributeError: 'list' object has no attribute 'items'
>>> MultiValueDict().update({1, 2, 3})
AttributeError: 'set' object has no attribute 'items'
>>> MultiValueDict().update({'a': 1})
None
Comparison after change in behavior.
>>> dict().update(None)
TypeError: 'NoneType' object is not iterable
>>> dict().update(True)
TypeError: 'bool' object is not iterable
>>> dict().update(False)
TypeError: 'bool' object is not iterable
>>> dict().update(123)
TypeError: 'int' object is not iterable
>>> dict().update(123.45)
TypeError: 'float' object is not iterable
>>> MultiValueDict().update(None)
TypeError: 'NoneType' object is not iterable
>>> MultiValueDict().update(True)
TypeError: 'bool' object is not iterable
>>> MultiValueDict().update(False)
TypeError: 'bool' object is not iterable
>>> MultiValueDict().update(123)
TypeError: 'int' object is not iterable
>>> MultiValueDict().update(123.45)
TypeError: 'float' object is not iterable
>>> dict().update('')
None
>>> dict().update(b'')
None
>>> dict().update(())
None
>>> dict().update([])
None
>>> dict().update(set())
None
>>> dict().update({})
None
>>> MultiValueDict().update('')
None
>>> MultiValueDict().update(b'')
None
>>> MultiValueDict().update(())
None
>>> MultiValueDict().update([])
None
>>> MultiValueDict().update(set())
None
>>> MultiValueDict().update({})
None
>>> dict().update((('a', 1),))
None
>>> dict().update([('a', 1)])
None
>>> dict().update({('a', 1)})
None
>>> MultiValueDict().update((('a', 1),))
None
>>> MultiValueDict().update([('a', 1)])
None
>>> MultiValueDict().update({('a', 1)})
None
>>> dict().update('123')
ValueError: dictionary update sequence element #0 has length 1; 2 is required
>>> dict().update(b'123')
TypeError: cannot convert dictionary update sequence element #0 to a sequence
>>> dict().update((1, 2, 3))
TypeError: cannot convert dictionary update sequence element #0 to a sequence
>>> dict().update([1, 2, 3])
TypeError: cannot convert dictionary update sequence element #0 to a sequence
>>> dict().update({1, 2, 3})
TypeError: cannot convert dictionary update sequence element #0 to a sequence
>>> dict().update({'a': 1})
None
>>> MultiValueDict().update('123')
ValueError: not enough values to unpack (expected 2, got 1)
>>> MultiValueDict().update(b'123')
TypeError: cannot unpack non-iterable int object
>>> MultiValueDict().update((1, 2, 3))
TypeError: cannot unpack non-iterable int object
>>> MultiValueDict().update([1, 2, 3])
TypeError: cannot unpack non-iterable int object
>>> MultiValueDict().update({1, 2, 3})
TypeError: cannot unpack non-iterable int object
>>> MultiValueDict().update({'a': 1})
None

Finally, there is an extra commit that removes support for providing an instance of Exception to the warning argument for ImmutableList. This is untested, undocumented and unused in Django itself. It didn't seem worth adding a test and preserving this behavior.

@ngnpope ngnpope force-pushed the datastructures-coverage branch 2 times, most recently from 40c3896 to ca4024f Compare October 7, 2020 09:14
@carltongibson
Copy link
Member

Thanks for this @pope1ni. Looks good at first glance. Let me just give it a ponder. (Will respond fully for next week.)

Copy link
Member

@carltongibson carltongibson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @pope1ni. I think this is great. 👍

The 4 main commits make sense separately I think.

Copy link
Member

@felixxm felixxm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pope1ni Thanks 👍 just some nitpicks

tests/utils_tests/test_datastructures.py Outdated Show resolved Hide resolved
tests/utils_tests/test_datastructures.py Outdated Show resolved Hide resolved
tests/utils_tests/test_datastructures.py Outdated Show resolved Hide resolved
tests/utils_tests/test_datastructures.py Outdated Show resolved Hide resolved
tests/utils_tests/test_datastructures.py Show resolved Hide resolved
tests/utils_tests/test_datastructures.py Outdated Show resolved Hide resolved
Copy link
Member

@felixxm felixxm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pope1ni Thanks👍

"Removed..." in the third commit😉

@ngnpope
Copy link
Member Author

ngnpope commented Oct 29, 2020

"Removed..." in the third commit

Auuugh! 🙀 😂

atombrella and others added 4 commits October 30, 2020 10:44
Co-authored-by: Nick Pope <nick.pope@flightdataservices.com>
If the warning provided was an instance of Exception, then it would be
used as-is. In practice this is untested, unused and ImmutableList is
an undocumented internal datastructure.
Changes in behavior include:

- Accepting iteration over empty sequences, updating nothing.
- Accepting iterable of 2-tuples providing key-value pairs.
- Failing with the same or comparable exceptions for invalid input.

Notably this replaces the previous attempt to catch TypeError which was
unreachable as the call to .items() resulted in AttributeError on
non-dict objects.
@ngnpope
Copy link
Member Author

ngnpope commented Oct 30, 2020

I'm really being mocked now: Test failure in cache.tests.PyMemcacheCacheTests.test_touch. 😂

@felixxm felixxm merged commit 966b5b4 into django:master Oct 30, 2020
@ngnpope ngnpope deleted the datastructures-coverage branch October 30, 2020 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants