Skip to content

Commit

Permalink
BUG: Use Python pickle protocol version 4 for np.save (#26388)
Browse files Browse the repository at this point in the history
* BUG: Use default Python pickle protocol version rather than outdated protocol 3

* Update default pickle of np.save to 4

* Adds document for default pickle protocol 4
  • Loading branch information
vxst committed May 16, 2024
1 parent cf19544 commit 98e8913
Show file tree
Hide file tree
Showing 3 changed files with 5 additions and 2 deletions.
3 changes: 3 additions & 0 deletions doc/release/upcoming_changes/26388.performance.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
* `numpy.save` now uses pickle protocol version 4 for saving arrays with
object dtype, which allows for pickle objects larger than 4GB and improves
saving speed by about 5% for large arrays.
2 changes: 1 addition & 1 deletion numpy/lib/_npyio_impl.py
Original file line number Diff line number Diff line change
Expand Up @@ -520,7 +520,7 @@ def save(file, arr, allow_pickle=True, fix_imports=True):
arbitrary code) and portability (pickled objects may not be loadable
on different Python installations, for example if the stored objects
require libraries that are not available, and not all pickled data is
compatible between Python 2 and Python 3).
compatible between different versions of Python).
Default: True
fix_imports : bool, optional
Only useful in forcing objects in object arrays on Python 3 to be
Expand Down
2 changes: 1 addition & 1 deletion numpy/lib/format.py
Original file line number Diff line number Diff line change
Expand Up @@ -741,7 +741,7 @@ def write_array(fp, array, version=None, allow_pickle=True, pickle_kwargs=None):
"when allow_pickle=False")
if pickle_kwargs is None:
pickle_kwargs = {}
pickle.dump(array, fp, protocol=3, **pickle_kwargs)
pickle.dump(array, fp, protocol=4, **pickle_kwargs)
elif array.flags.f_contiguous and not array.flags.c_contiguous:
if isfileobj(fp):
array.T.tofile(fp)
Expand Down

0 comments on commit 98e8913

Please sign in to comment.