Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
[MRG] Fix auto memmap gc failure #294
The automatic memmap feature of joblib can be mislead by the reuse of
This PR fixes the issues by systematically hashing the arrays to find a robust unique identifier for the filenames of the temporary memmap'ed arrays.
I ran the benchmark script varying the parameters and could not find a configuration where hashing could cause a very large performance degradation w.r.t. the current master.
This fixes scikit-learn/scikit-learn#6063.
BTW this AppVeyor failure seems genuine (from there):
* tag '0.9.4': (46 commits) Release 0.9.4 DOC add missing changelog entry for joblib#296 DOC add entry to changelog for joblib#294 ENH spare one file descriptor / syscall in automemmap FIX auto-memmap gc bug by always hashing arrays TST non-regression test for auto-memmap / gc bug Add link to github issues for 0.9.4 changelog entries Fix my_exceptions._mk_exception when input exception is not inheritable add entry in changelog fixing hashing with mixed dtype + test Use _compat.PY3_OR_LATER where possible COSMIT fix some PEP8 horizontal misalignments Move definition of PY3_OR_LATER to _compat.py Do not use inspect.getargspec FIX joblib#295: deadlock between async dispatch and exception handling Add section in CHANGES.rst TRAVIS use numpy 1.10 FIX style and pyflakes in test_pool.py Fix Parallel hanging with exhausted iterator remove useless section about versions of python prior to 2.6 ...