hdf5 reader for ensemble prediction #681
Conversation
MerktSimon
changed the title
pypesto/store/read_from_hdf5.py
Outdated
| base = Path(base_path) | ||
|
|
||
| # open file | ||
| f = h5py.File(input_file, 'r') |
There was a problem hiding this comment.
Rather make it into With h5py.File(input_file, 'r') as f: that way we ensure it is closed after reading.
pypesto/store/read_from_hdf5.py
Outdated
| prediction_id = str(f[os.path.join(base, PREDICTION_ID)][()], 'utf-8') | ||
|
|
||
| pred_res_list = [] | ||
| for i_pred in range(5): #len(f.keys())-1): |
There was a problem hiding this comment.
need to replace range(5). Probably easier to just loop over the keys of f and skip prediction_id?
|
Ok, I put the reader function into util, as there were import problem otherwise. Will write correct reader and writer function/classes at some later timepoints. |
Codecov Report
@@ Coverage Diff @@
## develop #681 +/- ##
============================================
+ Coverage 46.50% 90.37% +43.87%
============================================
Files 96 96
Lines 6400 6463 +63
============================================
+ Hits 2976 5841 +2865
+ Misses 3424 622 -2802
Continue to review full report at Codecov.
|
| prediction_id=AMICI_X, | ||
| engine=engine) | ||
|
|
||
| fn = 'test_file.hdf5' |
There was a problem hiding this comment.
maybe better with https://docs.python.org/3/library/tempfile.html
What do you think?
There was a problem hiding this comment.
do not remember why I have started avoiding it tbh :D
Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com>
Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com>
Writing already exists:
pyPESTO/pypesto/ensemble/utils.py
Line 149 in 2998e15
Useful e.g. for large sample based predictions, which take several hours and a lot of memory, if one wants to come back to earlier predictions.
Progress