-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
hdf5 reader for ensemble prediction #681
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will work on it, thanks for setting up something already 👍
pypesto/store/read_from_hdf5.py
Outdated
base = Path(base_path) | ||
|
||
# open file | ||
f = h5py.File(input_file, 'r') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rather make it into With h5py.File(input_file, 'r') as f:
that way we ensure it is closed after reading.
pypesto/store/read_from_hdf5.py
Outdated
prediction_id = str(f[os.path.join(base, PREDICTION_ID)][()], 'utf-8') | ||
|
||
pred_res_list = [] | ||
for i_pred in range(5): #len(f.keys())-1): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
need to replace range(5). Probably easier to just loop over the keys of f
and skip prediction_id
?
Ok, I put the reader function into util, as there were import problem otherwise. Will write correct reader and writer function/classes at some later timepoints. |
Codecov Report
@@ Coverage Diff @@
## develop #681 +/- ##
============================================
+ Coverage 46.50% 90.37% +43.87%
============================================
Files 96 96
Lines 6400 6463 +63
============================================
+ Hits 2976 5841 +2865
+ Misses 3424 622 -2802
Continue to review full report at Codecov.
|
prediction_id=AMICI_X, | ||
engine=engine) | ||
|
||
fn = 'test_file.hdf5' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe better with https://docs.python.org/3/library/tempfile.html
What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do not remember why I have started avoiding it tbh :D
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com>
Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com>
Writing already exists:
pyPESTO/pypesto/ensemble/utils.py
Line 149 in 2998e15
Useful e.g. for large sample based predictions, which take several hours and a lot of memory, if one wants to come back to earlier predictions.
Progress