Skip to content

DO NOT MERGE - cut-down tests to show random FilterDict test failure#180

Closed
tomlongridge wants to merge 4 commits into
masterfrom
tom/filter_dict_failures
Closed

DO NOT MERGE - cut-down tests to show random FilterDict test failure#180
tomlongridge wants to merge 4 commits into
masterfrom
tom/filter_dict_failures

Conversation

@tomlongridge
Copy link
Copy Markdown
Contributor

@tomlongridge tomlongridge commented Jun 12, 2019

Goal

In trying to add runtime information to the device payload I've added a nested dict to the structure and this seems to be failing unit tests sporadically. I have reduced the payload in a unit test down to the following:

        {
            "level1-key1": {
                "level2-key1": FilterDict({
                    "level3-key1": {
                        'level4-key1': 'level4-value1'}
                }),
                "level2-key2": FilterDict({
                    "level3-key2": "level3-value1",
                    "level3-key3": {
                        'level4-key2': 'level4-value2'}
                }),
            }
        }

It's the addition of the last dict that I'm trying to achieve and yet it doesn't occur if I remove the other FilterDict or the simple string pair preceding it.

I've added the test_to_death_tx.sh script that can be run to continuously run the tests - they fail after <10 usually. It only seems to fail through tox on python3.5 and before - running nosetest doesn't seem to make it fail.

I've added print lines to the utils function to show what's happening and on a failed test you can see two dict's added to the ignored list with the same id despite being different objects.

It is always a clash between level4-key1 and level4-key2:

***** Added: 4552960392 -> {'level4-key2': 'level4-value2'}
!!!!! Recursive: 4552960392 -> {'level4-key1': 'level4-value1'}

I wondered whether it was to do with the order of dict iteration being undefined but it doesn't seem to fix it with a sorted list either.

@tomlongridge tomlongridge added the bug Confirmed bug label Jun 12, 2019
kattrali added a commit that referenced this pull request Jun 12, 2019
References to nested dicts/field are recycled when going out of scope,
which is immediately since the references are not retained for the
lifetime of the sanitization pass. This change keeps references to seen
nested objects in addition to ignored objects to ensure id() returns
unique references.

Related reports on the Python issue tracker which explain the issue:
* https://bugs.python.org/issue36156
* https://bugs.python.org/issue33685

>>> help(id)

Help on built-in function id in module __builtin__:

id(...)
    id(object) -> integer

    Return the identity of an object.  This is guaranteed to be unique
    among simultaneously existing objects.  (Hint: it's the object's 
    memory address.)

Closes #180
@tomlongridge
Copy link
Copy Markdown
Contributor Author

Fixed in #181

@tomlongridge tomlongridge deleted the tom/filter_dict_failures branch June 14, 2019 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Confirmed bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant