Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unpickling can corrupt an internal cache of bytes object (py3) #370

Closed
rlamy opened this issue Jul 30, 2012 · 3 comments
Closed

Unpickling can corrupt an internal cache of bytes object (py3) #370

rlamy opened this issue Jul 30, 2012 · 3 comments

Comments

@rlamy
Copy link
Contributor

rlamy commented Jul 30, 2012

The Python 3 interpreter has an internal cache of single-byte bytes objects, from which objects are retrieved in a few operations, notably computing length-1 slices of bytes objects. Pickling and unpickling certain arrays can give direct write-access into that cache. A simple way to reproduce the problem is:

>>> a = numpy.array([1], 'b')
>>> b = pickle.loads(pickle.dumps(a))
>>> b[0] = 77
>>> b'\x01  '[0:1]
b'M'

This applies not only to master, but also to released versions, e.g. to Ubuntu 12.04's python3-numpy, which has numpy 1.6.1 and Python 3.2.3.

A similar problem has apparently been encountered before, as the test suite has test_pickle_string_overwrite() in numpy/core/tests/test_regression.py, which triggers the cache corruption but doesn't detect it.

@pv
Copy link
Member

pv commented Jul 30, 2012

Relevant source code location: numpy/core/src/multiarray/methods.c:1588
The comment /* Bytes are never interned */ there is invalid.
The fix is one-liner, just take the branch unconditionally.

@rlamy
Copy link
Contributor Author

rlamy commented Jul 31, 2012

Thanks, I did exactly that.

@njsmith
Copy link
Member

njsmith commented Aug 31, 2012

This got fixed a while ago :-).

@njsmith njsmith closed this as completed Aug 31, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants