-
Notifications
You must be signed in to change notification settings - Fork 429
Fixing byte-order consistency/mismatch for cross-endian platform #1181
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
pradghos
commented
May 5, 2021
- Addressing dtype.byteorder is not consistent during cross platform joblib.load() #1123
Circle CI failed at ./continuous_integration/build_doc.sh
Before this commit Now Any pointers would really help . Thank you ! |
@ogrisel It would be great if you can take a look. Thanks in advance ! |
Thanks a lot for this PR! What would be great is to have a test for this. Is there a way to store a pickle string obtained from a big endian machine and test that it can be loaded fine in the CI (little endian machine)? I think we can ignore the CI failures as they do seem spurious. |
Alternatively could you just please add a couple of unit tests for |
Codecov Report
@@ Coverage Diff @@
## master #1181 +/- ##
==========================================
+ Coverage 94.51% 94.54% +0.02%
==========================================
Files 47 47
Lines 6997 7034 +37
==========================================
+ Hits 6613 6650 +37
Misses 384 384
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some nitpick. I plan to push a new commit to better test the byteswap operation itself after that.
Merged!, thanks @pradghos! |
Release 1.1.0 Fix byte order inconsistency issue during deserialization using joblib.load in cross-endian environment: the numpy arrays are now always loaded to use the system byte order, independently of the byte order of the system that serialized the pickle. joblib/joblib#1181 Fix joblib.Memory bug with the ignore parameter when the cached function is a decorated function. joblib/joblib#1165 Fix joblib.Memory to properly handle caching for functions defined interactively in a IPython session or in Jupyter notebook cell. joblib/joblib#1214 Update vendored loky (from version 2.9 to 3.0) and cloudpickle (from version 1.6 to 2.0) joblib/joblib#1218
…lib#1181) Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Release 1.1.0 Fix byte order inconsistency issue during deserialization using joblib.load in cross-endian environment: the numpy arrays are now always loaded to use the system byte order, independently of the byte order of the system that serialized the pickle. joblib/joblib#1181 Fix joblib.Memory bug with the ignore parameter when the cached function is a decorated function. joblib/joblib#1165 Fix joblib.Memory to properly handle caching for functions defined interactively in a IPython session or in Jupyter notebook cell. joblib/joblib#1214 Update vendored loky (from version 2.9 to 3.0) and cloudpickle (from version 1.6 to 2.0) joblib/joblib#1218