Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC emphasize security sensitivity of joblib.load #879

Merged
merged 3 commits into from May 29, 2019

Conversation

@ogrisel
Copy link
Contributor

ogrisel commented May 29, 2019

Update the documentation to make it explicit that joblib.load should never be used to load files from an untrusted source.

Also add compat for numpy 1.16.3 and later.

@joblib joblib deleted a comment from codecov bot May 29, 2019
@joblib joblib deleted a comment from codecov bot May 29, 2019
@codecov

This comment has been minimized.

Copy link

codecov bot commented May 29, 2019

Codecov Report

Merging #879 into master will decrease coverage by 11.16%.
The diff coverage is 14.28%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master     #879       +/-   ##
===========================================
- Coverage   95.28%   84.12%   -11.17%     
===========================================
  Files          45       45               
  Lines        6425     6412       -13     
===========================================
- Hits         6122     5394      -728     
- Misses        303     1018      +715
Impacted Files Coverage Δ
joblib/numpy_pickle.py 90.64% <ø> (-7.89%) ⬇️
joblib/numpy_pickle_compat.py 42.85% <14.28%> (-48.15%) ⬇️
joblib/test/test_dask.py 4.47% <0%> (-94.03%) ⬇️
joblib/_dask.py 25.98% <0%> (-69.5%) ⬇️
joblib/backports.py 35.41% <0%> (-58.34%) ⬇️
joblib/_compat.py 72.72% <0%> (-27.28%) ⬇️
joblib/test/test_module.py 77.77% <0%> (-22.23%) ⬇️
joblib/test/common.py 67.79% <0%> (-20.34%) ⬇️
joblib/test/test_numpy_pickle.py 85.55% <0%> (-12.81%) ⬇️
... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aa26263...0b31520. Read the comment docs.

@ogrisel ogrisel merged commit 0f1f647 into joblib:master May 29, 2019
2 of 3 checks passed
2 of 3 checks passed
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
@ogrisel ogrisel deleted the ogrisel:joblib-load-security-warning branch May 29, 2019
@ogrisel ogrisel mentioned this pull request May 29, 2019
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Oct 31, 2019
Release 0.14.0
Improved the load balancing between workers to avoid stranglers caused by an excessively large batch size when the task duration is varying significantly (because of the combined use of joblib.Parallel and joblib.Memory with a partially warmed cache for instance). joblib/joblib#899
Add official support for Python 3.8: fixed protocol number in Hasher and updated tests.
Fix a deadlock when using the dask backend (when scattering large numpy arrays). joblib/joblib#914
Warn users that they should never use joblib.load with files from untrusted sources. Fix security related API change introduced in numpy 1.6.3 that would prevent using joblib with recent numpy versions. joblib/joblib#879
Upgrade to cloudpickle 1.1.1 that add supports for the upcoming Python 3.8 release among other things. joblib/joblib#878
Fix semaphore availability checker to avoid spawning resource trackers on module import. joblib/joblib#893
Fix the oversubscription protection to only protect against nested Parallel calls. This allows joblib to be run in background threads. joblib/joblib#934
Fix ValueError (negative dimensions) when pickling large numpy arrays on Windows. joblib/joblib#920
Upgrade to loky 2.6.0 that add supports for the setting environment variables in child before loading any module. joblib/joblib#940
Fix the oversubscription protection for native libraries using threadpools (OpenBLAS, MKL, Blis and OpenMP runtimes). The maximal number of threads is can now be set in children using the inner_max_num_threads in parallel_backend. It defaults to cpu_count() // n_jobs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

1 participant
You can’t perform that action at this time.