-
-
Notifications
You must be signed in to change notification settings - Fork 367
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cache_hash can give the wrong hash code for deserialized objects #482
Comments
I think it would be more elegant to clear the cache – would that be feasible? |
Good point - I will just clear the cache |
bump? :) |
Last week was busier than expected, but I hope to complete this tomorrow |
gabbard
pushed a commit
to gabbard/attrs
that referenced
this issue
Jan 22, 2019
Because the hash code cache field gets serialized and deserialized by Pickle, previously when you deserialize a cache_hash=True attrs object, the hashcode will be the hashcode the object had at serialization-time. However, if your object had fields with hash codes which were not deterministic between interpreter runs, then on a new interpreter run your deserialized object would have a hash code which differs from a newly created identical object. This commit fixes that by clearing the cache on deserialization. It needs to override the __setstate__ method to do so, so this commit also forbids using a custom __setstate__ on a cache_hash=True object. Closes python-attrs#482 .
gabbard
pushed a commit
to gabbard/attrs
that referenced
this issue
Jan 22, 2019
Because the hash code cache field gets serialized and deserialized by Pickle, previously when you deserialize a cache_hash=True attrs object, the hashcode will be the hashcode the object had at serialization-time. However, if your object had fields with hash codes which were not deterministic between interpreter runs, then on a new interpreter run your deserialized object would have a hash code which differs from a newly created identical object. This commit fixes that by clearing the cache on deserialization. It needs to override the __setstate__ method to do so, so this commit also forbids using a custom __setstate__ on a cache_hash=True object. Closes python-attrs#482 .
bors bot
referenced
this issue
in mozilla/normandy
Mar 4, 2019
1784: Scheduled weekly dependency update for week 09 r=peterbe a=pyup-bot ### Update [attrs](https://pypi.org/project/attrs) from **18.2.0** to **19.1.0**. <details> <summary>Changelog</summary> ### 19.1.0 ``` ------------------- Backward-incompatible Changes ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - Fixed a bug where deserialized objects with ``cache_hash=True`` could have incorrect hash code values. This change breaks classes with ``cache_hash=True`` when a custom ``__setstate__`` is present. An exception will be thrown when applying the ``attrs`` annotation to such a class. This limitation is tracked in issue `494 <https://github.com/python-attrs/attrs/issues/494>`_. `482 <https://github.com/python-attrs/attrs/issues/482>`_ Changes ^^^^^^^ - Add ``is_callable``, ``deep_iterable``, and ``deep_mapping`` validators. * ``is_callable``: validates that a value is callable * ``deep_iterable``: Allows recursion down into an iterable, applying another validator to every member in the iterable as well as applying an optional validator to the iterable itself. * ``deep_mapping``: Allows recursion down into the items in a mapping object, applying a key validator and a value validator to the key and value in every item. Also applies an optional validator to the mapping object itself. You can find them in the ``attr.validators`` package. `425 <https://github.com/python-attrs/attrs/issues/425>`_ - Fixed stub files to prevent errors raised by mypy's ``disallow_any_generics = True`` option. `443 <https://github.com/python-attrs/attrs/issues/443>`_ - Attributes with ``init=False`` now can follow after ``kw_only=True`` attributes. `450 <https://github.com/python-attrs/attrs/issues/450>`_ - ``attrs`` now has first class support for defining exception classes. If you define a class using ``attr.s(auto_exc=True)`` and subclass an exception, the class will behave like a well-behaved exception class including an appropriate ``__str__`` method, and all attributes additionally available in an ``args`` attribute. `500 <https://github.com/python-attrs/attrs/issues/500>`_ - Clarified documentation for hashing to warn that hashable objects should be deeply immutable (in their usage, even if this is not enforced). `503 <https://github.com/python-attrs/attrs/issues/503>`_ ---- ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/attrs - Changelog: https://pyup.io/changelogs/attrs/ - Homepage: https://www.attrs.org/ </details> ### Update [botocore](https://pypi.org/project/botocore) from **1.12.101** to **1.12.106**. <details> <summary>Changelog</summary> ### 1.12.106 ``` ======== * api-change:``ec2``: Update ec2 client to latest version * api-change:``autoscaling-plans``: Update autoscaling-plans client to latest version ``` ### 1.12.105 ``` ======== * api-change:``ssm``: Update ssm client to latest version * api-change:``apigatewayv2``: Update apigatewayv2 client to latest version * api-change:``alexaforbusiness``: Update alexaforbusiness client to latest version * api-change:``application-autoscaling``: Update application-autoscaling client to latest version ``` ### 1.12.104 ``` ======== * api-change:``waf-regional``: Update waf-regional client to latest version * api-change:``waf``: Update waf client to latest version ``` ### 1.12.103 ``` ======== * api-change:``discovery``: Update discovery client to latest version * api-change:``organizations``: Update organizations client to latest version * api-change:``resource-groups``: Update resource-groups client to latest version * api-change:``opsworkscm``: Update opsworkscm client to latest version * api-change:``pinpoint``: Update pinpoint client to latest version * api-change:``mediaconvert``: Update mediaconvert client to latest version * api-change:``cur``: Update cur client to latest version ``` ### 1.12.102 ``` ======== * api-change:``elbv2``: Update elbv2 client to latest version * api-change:``mediastore``: Update mediastore client to latest version * api-change:``ce``: Update ce client to latest version * api-change:``autoscaling``: Update autoscaling client to latest version ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/botocore - Changelog: https://pyup.io/changelogs/botocore/ - Repo: https://github.com/boto/botocore </details> ### Update [cffi](https://pypi.org/project/cffi) from **1.12.1** to **1.12.2**. *The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)* <details> <summary>Links</summary> - PyPI: https://pypi.org/project/cffi - Changelog: https://pyup.io/changelogs/cffi/ - Docs: http://cffi.readthedocs.org </details> ### Update [idna](https://pypi.org/project/idna) from **2.6** to **2.8**. <details> <summary>Changelog</summary> ### 2.8 ``` ++++++++++++++++ - Update to Unicode 11.0.0. - Provide more specific exceptions for some malformed labels. ``` ### 2.7 ``` ++++++++++++++++ - Update to Unicode 10.0.0. - No longer accepts dot-prefixed domains (e.g. ".example") as valid. This is to be more conformant with the UTS 46 spec. Users should strip dot prefixes from domains before processing. ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/idna - Changelog: https://pyup.io/changelogs/idna/ - Repo: https://github.com/kjd/idna </details> ### Update [pyflakes](https://pypi.org/project/pyflakes) from **2.1.0** to **2.1.1**. *The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)* <details> <summary>Links</summary> - PyPI: https://pypi.org/project/pyflakes - Changelog: https://pyup.io/changelogs/pyflakes/ - Repo: https://github.com/PyCQA/pyflakes </details> ### Update [google-api-core](https://pypi.org/project/google-api-core) from **1.7.0** to **1.8.0**. *The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)* <details> <summary>Links</summary> - PyPI: https://pypi.org/project/google-api-core - Repo: https://github.com/GoogleCloudPlatform/google-cloud-python </details> ### Update [google-cloud-core](https://pypi.org/project/google-cloud-core) from **0.28.1** to **0.29.1**. *The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)* <details> <summary>Links</summary> - PyPI: https://pypi.org/project/google-cloud-core - Repo: https://github.com/GoogleCloudPlatform/google-cloud-python </details> ### Update [protobuf](https://pypi.org/project/protobuf) from **3.6.1** to **3.7.0**. *The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)* <details> <summary>Links</summary> - PyPI: https://pypi.org/project/protobuf - Changelog: https://pyup.io/changelogs/protobuf/ - Repo: https://github.com/protocolbuffers/protobuf/releases - Homepage: https://developers.google.com/protocol-buffers/ </details> ### Update [boto3](https://pypi.org/project/boto3) from **1.9.101** to **1.9.106**. <details> <summary>Changelog</summary> ### 1.9.106 ``` ======= * api-change:``ec2``: [``botocore``] Update ec2 client to latest version * api-change:``autoscaling-plans``: [``botocore``] Update autoscaling-plans client to latest version ``` ### 1.9.105 ``` ======= * api-change:``ssm``: [``botocore``] Update ssm client to latest version * api-change:``apigatewayv2``: [``botocore``] Update apigatewayv2 client to latest version * api-change:``alexaforbusiness``: [``botocore``] Update alexaforbusiness client to latest version * api-change:``application-autoscaling``: [``botocore``] Update application-autoscaling client to latest version ``` ### 1.9.104 ``` ======= * api-change:``waf-regional``: [``botocore``] Update waf-regional client to latest version * api-change:``waf``: [``botocore``] Update waf client to latest version ``` ### 1.9.103 ``` ======= * api-change:``discovery``: [``botocore``] Update discovery client to latest version * api-change:``organizations``: [``botocore``] Update organizations client to latest version * api-change:``resource-groups``: [``botocore``] Update resource-groups client to latest version * api-change:``opsworkscm``: [``botocore``] Update opsworkscm client to latest version * api-change:``pinpoint``: [``botocore``] Update pinpoint client to latest version * api-change:``mediaconvert``: [``botocore``] Update mediaconvert client to latest version * api-change:``cur``: [``botocore``] Update cur client to latest version ``` ### 1.9.102 ``` ======= * api-change:``elbv2``: [``botocore``] Update elbv2 client to latest version * api-change:``mediastore``: [``botocore``] Update mediastore client to latest version * api-change:``ce``: [``botocore``] Update ce client to latest version * api-change:``autoscaling``: [``botocore``] Update autoscaling client to latest version ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/boto3 - Changelog: https://pyup.io/changelogs/boto3/ - Repo: https://github.com/boto/boto3 </details> ### Update [django-cors-headers](https://pypi.org/project/django-cors-headers) from **2.4.0** to **2.4.1**. <details> <summary>Changelog</summary> ### 2.4.1 ``` ------------------ * Fix ``DeprecationWarning`` from importing ``collections.abc.Sequence`` on Python 3.7. ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/django-cors-headers - Changelog: https://pyup.io/changelogs/django-cors-headers/ - Repo: https://github.com/ottoyiu/django-cors-headers </details> ### Update [django-mozilla-product-details](https://pypi.org/project/django-mozilla-product-details) from **0.13** to **0.13.1**. *The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)* <details> <summary>Links</summary> - PyPI: https://pypi.org/project/django-mozilla-product-details - Repo: https://github.com/mozilla/django-product-details/ </details> ### Update [djangorestframework](https://pypi.org/project/djangorestframework) from **3.9.1** to **3.9.2**. *The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)* <details> <summary>Links</summary> - PyPI: https://pypi.org/project/djangorestframework - Changelog: https://pyup.io/changelogs/djangorestframework/ - Homepage: https://www.django-rest-framework.org/ </details> ### Update [jsonschema](https://pypi.org/project/jsonschema) from **3.0.0** to **3.0.1**. *The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)* <details> <summary>Links</summary> - PyPI: https://pypi.org/project/jsonschema - Changelog: https://pyup.io/changelogs/jsonschema/ - Repo: https://github.com/Julian/jsonschema </details> ### Update [pytest-django](https://pypi.org/project/pytest-django) from **3.4.7** to **3.4.8**. <details> <summary>Changelog</summary> ### 3.4.8 ``` ------------------ Bugfixes ^^^^^^^^ * Fix DB renaming fixture for Multi-DB environment with SQLite (679) ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/pytest-django - Changelog: https://pyup.io/changelogs/pytest-django/ - Docs: https://pytest-django.readthedocs.io/ </details> ### Update [google-cloud-storage](https://pypi.org/project/google-cloud-storage) from **1.13.0** to **1.14.0**. *The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)* <details> <summary>Links</summary> - PyPI: https://pypi.org/project/google-cloud-storage - Repo: https://github.com/GoogleCloudPlatform/google-cloud-python </details> Co-authored-by: pyup-bot <github-bot@pyup.io>
I feel slightly better that I am not the only one to screw this up. :-) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I just realized a bug in the code I wrote for hash code caching (#426). Because the hash code cache field gets serialized and deserialized by Pickle, when you deserialize a
cache_hash=True
attrs
object, the hashcode will be the hashcode the object had at serialization-time. However, if your object has fields with hash codes which are not deterministic between interpreter runs, then on a new interpreter run your deserialized object will have a hash code which differs from a newly created identical object.We can fix this for
pickle
by recomputing the hash code in__setstate__
. Other serialization libraries which don't respect__setstate__
will still have a problem, but I don't think we can do anything about that. If the__setstate__
solution sounds acceptable I will implement it next week.The text was updated successfully, but these errors were encountered: