RF+ENH+BUG: Use npz for cache; store means #177
Conversation
scottclowe
added
bug
enhancement
vX.Y.0 New Minor Release
Codecov Report
@@ Coverage Diff @@
## master #177 +/- ##
==========================================
+ Coverage 92.65% 92.93% +0.27%
==========================================
Files 8 8
Lines 831 863 +32
Branches 165 178 +13
==========================================
+ Hits 770 802 +32
Misses 31 31
Partials 30 30
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
scottclowe
force-pushed
the
bug_mean-not-saved
branch
from
583b4f8
to
fd8074f
Compare
scottclowe
force-pushed
the
bug_mean-not-saved
branch
from
fd8074f
to
0b7dc5c
Compare
The `dirs_exist_ok` argument was added to shutil.copytree in python 3.6. We just delete the target directory instead, so it is compatible with python 3.5.
…riments in same test
There was a problem hiding this comment.
Overall I agree with these changes.
The only thing I would suggest is to add a short section to the jupyter notebook about interacting with the cache files without using FISSA. This was not there before because we didn't really consider this usecase, but makes a lot of sense with these changes. Could also be done as a separate PR.
Change from saving a list of a few numpy.ndarrays with numpy.save, we save several arrays associated with keys using
numpy.savez_compressed.
At the same time, we refactor some code to remove a superfluous nesting depth for the main preparation and separation code blocks.
We now save the means into the .npz file, so this is now restored. This resolves the issue (described in PR #169) that running an experiment that loads from cache does not set
experiment.means.