New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix pickling for instances of classes created using make_class() #282

Merged
merged 7 commits into from Nov 4, 2017

Conversation

Projects
None yet
2 participants
@trendels
Contributor

trendels commented Nov 4, 2017

Currently, trying to pickle an instance of a class created using attrs.make_class fails (tested on Python 2.7 and 3.5):

>>> import attr
>>> import pickle
>>> C = attr.make_class('C', ['x'])
>>> obj = C(1)
>>> pickle.dumps(obj)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
_pickle.PicklingError: Can't pickle <class 'attr._make.C'>: attribute lookup C on attr._make failed

It has something to do with pickle looking for the class/type in the wrong module. I was curious how collections.namedtuple solves this and found this (also present in Python 2.7):

https://github.com/python/cpython/blob/v3.5.4/Lib/collections/__init__.py#L436-L445

This PR adds the same code (with comment) verbatim to attrs, to make pickling work. As a side effect, the repr of classes created this way now also contain name of the module where make_class() was called instead of "attr._make". I'm not sure if this needs a Changelog/Documentation update?

I also noticed that Python 3.6 added an explicit module parameter to collections.namedtuple, to allow manually overriding the module name to fix issues with pickling on other Python implementations such as IronPython (https://bugs.python.org/issue17941). Maybe this would also be useful to add to attrs in the future.

@codecov

This comment has been minimized.

codecov bot commented Nov 4, 2017

Codecov Report

Merging #282 into master will not change coverage.
The diff coverage is 100%.

Impacted file tree graph

@@          Coverage Diff          @@
##           master   #282   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files           9      9           
  Lines         660    666    +6     
  Branches      138    138           
=====================================
+ Hits          660    666    +6
Impacted Files Coverage Δ
src/attr/_make.py 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1bb8fbd...93cd3f8. Read the comment docs.

@hynek

This comment has been minimized.

Member

hynek commented Nov 4, 2017

That looks like a good last-minute change for 17.3 (famous last words).

Would you mind completing coverage and add a changelog entry? There’s a bunch of cosmetic problems but I can deal with them myself once it’s merged.

Thanks!

@hynek hynek added this to the 17.3 milestone Nov 4, 2017

@trendels

This comment has been minimized.

Contributor

trendels commented Nov 4, 2017

That would be awesome.

I've added a changelog entry. I'm not sure what do do about the coverage though, other than using # pragma: no cover. The line that is missing coverage is only there to silence exceptions on more "esoteric" Python interpreters (IronPython, Jython) where sys._getframes is not available or works differently than in CPython, according to the comment from collections.namedtuple.

@hynek

This comment has been minimized.

Member

hynek commented Nov 4, 2017

Maybe try monkeypatch.delattr(sys, "_getframe)"?

If that doesn’t work because it’s too invasive, just add a no cover and call it a Saturday.

@python-attrs python-attrs deleted a comment from codecov bot Nov 4, 2017

@trendels

This comment has been minimized.

Contributor

trendels commented Nov 4, 2017

Ah yes, that should work. TIL about pytest monkeypatch :)

@hynek

hynek approved these changes Nov 4, 2017

@hynek hynek merged commit c1adf5c into python-attrs:master Nov 4, 2017

3 checks passed

codecov/patch 100% of diff hit (target 100%)
Details
codecov/project 100% (+0%) compared to 1bb8fbd
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@hynek

This comment has been minimized.

Member

hynek commented Nov 4, 2017

Wonderful, so you got something out of it too. ;) Thank you for your contribution!

@trendels trendels deleted the trendels:fix-pickle-make_class branch Nov 4, 2017

bors-fusion bot added a commit to fusionapp/fusion-index that referenced this pull request Nov 15, 2017

Merge #167
167: Scheduled weekly dependency update for week 46 r=mithrandi




## Updates
Here's a list of all the updates bundled in this pull request. I've added some links to make it easier for you to find all the information you need.
<table align="center">

<tr>
<td><b>attrs</b></td>
<td align="center">17.2.0</td>
<td align="center">&raquo;</td>
<td align="center">17.3.0</td>
<td>
     <a href="https://pypi.python.org/pypi/attrs">PyPI</a> | <a href="https://pyup.io/changelogs/attrs/">Changelog</a> | <a href="http://www.attrs.org/">Homepage</a> 

</td>

<tr>
<td><b>cryptography</b></td>
<td align="center">2.1.2</td>
<td align="center">&raquo;</td>
<td align="center">2.1.3</td>
<td>
     <a href="https://pypi.python.org/pypi/cryptography">PyPI</a> | <a href="https://pyup.io/changelogs/cryptography/">Changelog</a> | <a href="https://github.com/pyca/cryptography">Repo</a> 

</td>

<tr>
<td><b>hypothesis</b></td>
<td align="center">3.33.0</td>
<td align="center">&raquo;</td>
<td align="center">3.37.0</td>
<td>
     <a href="https://pypi.python.org/pypi/hypothesis">PyPI</a> | <a href="https://pyup.io/changelogs/hypothesis/">Changelog</a> | <a href="https://github.com/HypothesisWorks/hypothesis/issues">Repo</a> 

</td>

</tr>
</table>



## Changelogs


### attrs 17.2.0 -> 17.3.0

>### 17.3.0

>-------------------

>Backward-incompatible Changes
>^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

>- Attributes are not defined on the class body anymore.

>  This means that if you define a class ``C`` with an attribute ``x``, the class will *not* have an attribute ``x`` for introspection anymore.
>  Instead of ``C.x``, use ``attr.fields(C).x`` or look at ``C.__attrs_attrs__``.
>  The old behavior has been deprecated since version 16.1.
>  (`253 &lt;https://github.com/python-attrs/attrs/issues/253&gt;`_)


>Changes
>^^^^^^^

>- ``super()`` and ``__class__`` now work on Python 3 when ``slots=True``.
>  (`102 &lt;https://github.com/python-attrs/attrs/issues/102&gt;`_, `226 &lt;https://github.com/python-attrs/attrs/issues/226&gt;`_, `269 &lt;https://github.com/python-attrs/attrs/issues/269&gt;`_, `270 &lt;https://github.com/python-attrs/attrs/issues/270&gt;`_, `272 &lt;https://github.com/python-attrs/attrs/issues/272&gt;`_)
>- Added ``type`` argument to ``attr.ib()`` and corresponding ``type`` attribute to ``attr.Attribute``.

>  This change paves the way for automatic type checking and serialization (though as of this release ``attrs`` does not make use of it).
>  In Python 3.6 or higher, the value of ``attr.Attribute.type`` can alternately be set using variable type annotations
>  (see `PEP 526 &lt;https://www.python.org/dev/peps/pep-0526/&gt;`_). (`151 &lt;https://github.com/python-attrs/attrs/issues/151&gt;`_, `214 &lt;https://github.com/python-attrs/attrs/issues/214&gt;`_, `215 &lt;https://github.com/python-attrs/attrs/issues/215&gt;`_, `239 &lt;https://github.com/python-attrs/attrs/issues/239&gt;`_)
>- The combination of ``str=True`` and ``slots=True`` now works on Python 2.
>  (`198 &lt;https://github.com/python-attrs/attrs/issues/198&gt;`_)
>- ``attr.Factory`` is hashable again. (`204
>  &lt;https://github.com/python-attrs/attrs/issues/204&gt;`_)
>- Subclasses now can overwrite attribute definitions of their superclass.

>  That means that you can -- for example -- change the default value for an attribute by redefining it.
>  (`221 &lt;https://github.com/python-attrs/attrs/issues/221&gt;`_, `229 &lt;https://github.com/python-attrs/attrs/issues/229&gt;`_)
>- Added new option ``auto_attribs`` to ``attr.s`` that allows to collect annotated fields without setting them to ``attr.ib()``.

>  Setting a field to an ``attr.ib()`` is still possible to supply options like validators.
>  Setting it to any other value is treated like it was passed as ``attr.ib(default=value)`` -- passing an instance of ``attr.Factory`` also works as expected.
>  (`262 &lt;https://github.com/python-attrs/attrs/issues/262&gt;`_, `277 &lt;https://github.com/python-attrs/attrs/issues/277&gt;`_)
>- Instances of classes created using ``attr.make_class()`` can now be pickled.
>  (`282 &lt;https://github.com/python-attrs/attrs/issues/282&gt;`_)


>----








### hypothesis 3.33.0 -> 3.37.0

>### 3.37.0

>-------------------

>This is a deprecation release for some health check related features.

>The following are now deprecated:

>* Passing :attr:`~hypothesis.HealthCheck.exception_in_generation` to
>  :attr:`~hypothesis.settings.suppress_health_check`. This no longer does
>  anything even when passed -  All errors that occur during data generation
>  will now be immediately reraised rather than going through the health check
>  mechanism.
>* Passing :attr:`~hypothesis.HealthCheck.random_module` to
>  :attr:`~hypothesis.settings.suppress_health_check`. This hasn&#39;t done anything
>  for a long time, but was never explicitly deprecated. Hypothesis always seeds
>  the random module when running given tests, so this is no longer an error
>  and suppressing it doesn&#39;t do anything.
>* Passing non-:class:`~hypothesis.HealthCheck` values in
>  :attr:`~hypothesis.settings.suppress_health_check`. This was previously
>  allowed but never did anything useful.

>In addition, passing a non-iterable value as :attr:`~hypothesis.settings.suppress_health_check`
>will now raise an error immediately (it would never have worked correctly, but
>it would previously have failed later). Some validation error messages have
>also been updated.

>This work was funded by `Smarkets &lt;https://smarkets.com/&gt;`_.

>-------------------


>### 3.36.1

>-------------------

>This is a yak shaving release, mostly concerned with our own tests.

>While :func:`~python:inspect.getfullargspec` was documented as deprecated
>in Python 3.5, it never actually emitted a warning.  Our code to silence
>this (nonexistent) warning has therefore been removed.

>We now run our tests with ``DeprecationWarning`` as an error, and made some
>minor changes to our own tests as a result.  This required similar upstream
>updates to :pypi:`coverage` and :pypi:`execnet` (a test-time dependency via
>:pypi:`pytest-xdist`).

>There is no user-visible change in Hypothesis itself, but we encourage you
>to consider enabling deprecations as errors in your own tests.

>-------------------


>### 3.36.0

>-------------------

>This release adds a setting to the public API, and does some internal cleanup:

>- The :attr:`~hypothesis.settings.derandomize` setting is now documented (:issue:`890`)
>- Removed - and disallowed - all &#39;bare excepts&#39; in Hypothesis (:issue:`953`)
>- Documented the :attr:`~hypothesis.settings.strict` setting as deprecated, and
>  updated the build so our docs always match deprecations in the code.

>-------------------


>### 3.35.0

>-------------------

>This minor release supports constraining :func:`~hypothesis.strategies.uuids`
>to generate :class:`~python:uuid.UUID`s of a particular version.
>(:issue:`721`)

>Thanks to Dion Misic for this feature.

>-------------------


>### 3.34.1

>-------------------

>This patch updates the documentation to suggest
>:func:`builds(callable) &lt;hypothesis.strategies.builds&gt;` instead of
>:func:`just(callable()) &lt;hypothesis.strategies.just&gt;`.

>-------------------


>### 3.34.0

>-------------------

>Hypothesis now emits deprecation warnings if you apply
>:func:`given &lt;hypothesis.given&gt;` more than once to a target.

>Applying :func:`given &lt;hypothesis.given&gt;` repeatedly wraps the target multiple
>times. Each wrapper will search the space of of possible parameters separately.
>This is equivalent but will be much more inefficient than doing it with a
>single call to :func:`given &lt;hypothesis.given&gt;`.

>For example, instead of
>``given(booleans()) given(integers())``, you could write
>``given(booleans(), integers())``

>-------------------


>### 3.33.1

>-------------------

>This is a bugfix release:

>- :func:`~hypothesis.strategies.builds` would try to infer a strategy for
>  required positional arguments of the target from type hints, even if they
>  had been given to :func:`~hypothesis.strategies.builds` as positional
>  arguments (:issue:`946`).  Now it only infers missing required arguments.
>- An internal introspection function wrongly reported ``self`` as a required
>  argument for bound methods, which might also have affected
>  :func:`~hypothesis.strategies.builds`.  Now it knows better.

>-------------------









That's it for now!

Happy merging! 🤖

bors-fusion bot added a commit to fusionapp/entropy that referenced this pull request Nov 15, 2017

Merge #161
161: Scheduled weekly dependency update for week 46 r=mithrandi




## Updates
Here's a list of all the updates bundled in this pull request. I've added some links to make it easier for you to find all the information you need.
<table align="center">

<tr>
<td><b>attrs</b></td>
<td align="center">17.2.0</td>
<td align="center">&raquo;</td>
<td align="center">17.3.0</td>
<td>
     <a href="https://pypi.python.org/pypi/attrs">PyPI</a> | <a href="https://pyup.io/changelogs/attrs/">Changelog</a> | <a href="http://www.attrs.org/">Homepage</a> 

</td>

<tr>
<td><b>cryptography</b></td>
<td align="center">2.1.2</td>
<td align="center">&raquo;</td>
<td align="center">2.1.3</td>
<td>
     <a href="https://pypi.python.org/pypi/cryptography">PyPI</a> | <a href="https://pyup.io/changelogs/cryptography/">Changelog</a> | <a href="https://github.com/pyca/cryptography">Repo</a> 

</td>

<tr>
<td><b>lxml</b></td>
<td align="center">4.1.0</td>
<td align="center">&raquo;</td>
<td align="center">4.1.1</td>
<td>
     <a href="https://pypi.python.org/pypi/lxml">PyPI</a> | <a href="https://pyup.io/changelogs/lxml/">Changelog</a> | <a href="http://lxml.de/">Homepage</a> | <a href="https://bugs.launchpad.net/lxml">Bugtracker</a> 

</td>

<tr>
<td><b>pytz</b></td>
<td align="center">2017.2</td>
<td align="center">&raquo;</td>
<td align="center">2017.3</td>
<td>
     <a href="https://pypi.python.org/pypi/pytz">PyPI</a> | <a href="http://pythonhosted.org/pytz">Homepage</a> | <a href="http://pythonhosted.org/pytz/">Docs</a> 

</td>

</tr>
</table>



## Changelogs


### attrs 17.2.0 -> 17.3.0

>### 17.3.0

>-------------------

>Backward-incompatible Changes
>^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

>- Attributes are not defined on the class body anymore.

>  This means that if you define a class ``C`` with an attribute ``x``, the class will *not* have an attribute ``x`` for introspection anymore.
>  Instead of ``C.x``, use ``attr.fields(C).x`` or look at ``C.__attrs_attrs__``.
>  The old behavior has been deprecated since version 16.1.
>  (`253 &lt;https://github.com/python-attrs/attrs/issues/253&gt;`_)


>Changes
>^^^^^^^

>- ``super()`` and ``__class__`` now work on Python 3 when ``slots=True``.
>  (`102 &lt;https://github.com/python-attrs/attrs/issues/102&gt;`_, `226 &lt;https://github.com/python-attrs/attrs/issues/226&gt;`_, `269 &lt;https://github.com/python-attrs/attrs/issues/269&gt;`_, `270 &lt;https://github.com/python-attrs/attrs/issues/270&gt;`_, `272 &lt;https://github.com/python-attrs/attrs/issues/272&gt;`_)
>- Added ``type`` argument to ``attr.ib()`` and corresponding ``type`` attribute to ``attr.Attribute``.

>  This change paves the way for automatic type checking and serialization (though as of this release ``attrs`` does not make use of it).
>  In Python 3.6 or higher, the value of ``attr.Attribute.type`` can alternately be set using variable type annotations
>  (see `PEP 526 &lt;https://www.python.org/dev/peps/pep-0526/&gt;`_). (`151 &lt;https://github.com/python-attrs/attrs/issues/151&gt;`_, `214 &lt;https://github.com/python-attrs/attrs/issues/214&gt;`_, `215 &lt;https://github.com/python-attrs/attrs/issues/215&gt;`_, `239 &lt;https://github.com/python-attrs/attrs/issues/239&gt;`_)
>- The combination of ``str=True`` and ``slots=True`` now works on Python 2.
>  (`198 &lt;https://github.com/python-attrs/attrs/issues/198&gt;`_)
>- ``attr.Factory`` is hashable again. (`204
>  &lt;https://github.com/python-attrs/attrs/issues/204&gt;`_)
>- Subclasses now can overwrite attribute definitions of their superclass.

>  That means that you can -- for example -- change the default value for an attribute by redefining it.
>  (`221 &lt;https://github.com/python-attrs/attrs/issues/221&gt;`_, `229 &lt;https://github.com/python-attrs/attrs/issues/229&gt;`_)
>- Added new option ``auto_attribs`` to ``attr.s`` that allows to collect annotated fields without setting them to ``attr.ib()``.

>  Setting a field to an ``attr.ib()`` is still possible to supply options like validators.
>  Setting it to any other value is treated like it was passed as ``attr.ib(default=value)`` -- passing an instance of ``attr.Factory`` also works as expected.
>  (`262 &lt;https://github.com/python-attrs/attrs/issues/262&gt;`_, `277 &lt;https://github.com/python-attrs/attrs/issues/277&gt;`_)
>- Instances of classes created using ``attr.make_class()`` can now be pickled.
>  (`282 &lt;https://github.com/python-attrs/attrs/issues/282&gt;`_)


>----








### lxml 4.1.0 -> 4.1.1

>### 4.1.1

>==================

>* Rebuild with Cython 0.27.3 to improve support for Py3.7.











That's it for now!

Happy merging! 🤖

bors-fusion bot added a commit to fusionapp/documint that referenced this pull request Nov 17, 2017

Merge #118
118: Update attrs to 17.3.0 r=mithrandi


There's a new version of [attrs](https://pypi.python.org/pypi/attrs) available.
You are currently using **17.2.0**. I have updated it to **17.3.0**



These links might come in handy:  <a href="https://pypi.python.org/pypi/attrs">PyPI</a> | <a href="https://pyup.io/changelogs/attrs/">Changelog</a> | <a href="http://www.attrs.org/">Homepage</a> 



### Changelog
> 
>### 17.3.0

>-------------------

>Backward-incompatible Changes
>^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

>- Attributes are not defined on the class body anymore.

>  This means that if you define a class ``C`` with an attribute ``x``, the class will *not* have an attribute ``x`` for introspection anymore.
>  Instead of ``C.x``, use ``attr.fields(C).x`` or look at ``C.__attrs_attrs__``.
>  The old behavior has been deprecated since version 16.1.
>  (`253 &lt;https://github.com/python-attrs/attrs/issues/253&gt;`_)


>Changes
>^^^^^^^

>- ``super()`` and ``__class__`` now work on Python 3 when ``slots=True``.
>  (`102 &lt;https://github.com/python-attrs/attrs/issues/102&gt;`_, `226 &lt;https://github.com/python-attrs/attrs/issues/226&gt;`_, `269 &lt;https://github.com/python-attrs/attrs/issues/269&gt;`_, `270 &lt;https://github.com/python-attrs/attrs/issues/270&gt;`_, `272 &lt;https://github.com/python-attrs/attrs/issues/272&gt;`_)
>- Added ``type`` argument to ``attr.ib()`` and corresponding ``type`` attribute to ``attr.Attribute``.

>  This change paves the way for automatic type checking and serialization (though as of this release ``attrs`` does not make use of it).
>  In Python 3.6 or higher, the value of ``attr.Attribute.type`` can alternately be set using variable type annotations
>  (see `PEP 526 &lt;https://www.python.org/dev/peps/pep-0526/&gt;`_). (`151 &lt;https://github.com/python-attrs/attrs/issues/151&gt;`_, `214 &lt;https://github.com/python-attrs/attrs/issues/214&gt;`_, `215 &lt;https://github.com/python-attrs/attrs/issues/215&gt;`_, `239 &lt;https://github.com/python-attrs/attrs/issues/239&gt;`_)
>- The combination of ``str=True`` and ``slots=True`` now works on Python 2.
>  (`198 &lt;https://github.com/python-attrs/attrs/issues/198&gt;`_)
>- ``attr.Factory`` is hashable again. (`204
>  &lt;https://github.com/python-attrs/attrs/issues/204&gt;`_)
>- Subclasses now can overwrite attribute definitions of their superclass.

>  That means that you can -- for example -- change the default value for an attribute by redefining it.
>  (`221 &lt;https://github.com/python-attrs/attrs/issues/221&gt;`_, `229 &lt;https://github.com/python-attrs/attrs/issues/229&gt;`_)
>- Added new option ``auto_attribs`` to ``attr.s`` that allows to collect annotated fields without setting them to ``attr.ib()``.

>  Setting a field to an ``attr.ib()`` is still possible to supply options like validators.
>  Setting it to any other value is treated like it was passed as ``attr.ib(default=value)`` -- passing an instance of ``attr.Factory`` also works as expected.
>  (`262 &lt;https://github.com/python-attrs/attrs/issues/262&gt;`_, `277 &lt;https://github.com/python-attrs/attrs/issues/277&gt;`_)
>- Instances of classes created using ``attr.make_class()`` can now be pickled.
>  (`282 &lt;https://github.com/python-attrs/attrs/issues/282&gt;`_)


>----








*Got merge conflicts? Close this PR and delete the branch. I'll create a new PR for you.*

Happy merging! 🤖
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment