Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reading Ensemble features from optimization result #640

Merged
merged 38 commits into from
May 12, 2021

Conversation

PaulJonasJost
Copy link
Collaborator

New features:

  • load Ensemble from an Optimization with or without usage of the history.
  • load Ensemble from an Hdf5 File

@codecov-commenter
Copy link

codecov-commenter commented Apr 22, 2021

Codecov Report

Merging #640 (e9f58ed) into develop (c7c94e1) will decrease coverage by 2.46%.
The diff coverage is 78.26%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #640      +/-   ##
===========================================
- Coverage    89.82%   87.35%   -2.47%     
===========================================
  Files           94       94              
  Lines         6013     6081      +68     
===========================================
- Hits          5401     5312      -89     
- Misses         612      769     +157     
Impacted Files Coverage Δ
pypesto/ensemble/utils.py 21.21% <22.22%> (-41.95%) ⬇️
pypesto/ensemble/ensemble.py 76.85% <85.96%> (-4.33%) ⬇️
pypesto/ensemble/constants.py 91.83% <100.00%> (+0.53%) ⬆️
pypesto/predict/result.py 33.33% <0.00%> (-61.12%) ⬇️
pypesto/petab/pysb_importer.py 73.33% <0.00%> (-20.00%) ⬇️
pypesto/petab/importer.py 70.25% <0.00%> (-15.19%) ⬇️
pypesto/predict/amici_predictor.py 82.71% <0.00%> (-14.82%) ⬇️
pypesto/objective/amici_util.py 75.43% <0.00%> (-8.78%) ⬇️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c7c94e1...e9f58ed. Read the comment docs.

@PaulJonasJost PaulJonasJost marked this pull request as ready for review April 22, 2021 11:58
Copy link
Member

@dilpath dilpath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think some tests might help clarify usage (and of course ensure behavior) a bit.

pypesto/ensemble/ensemble.py Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/utils.py Outdated Show resolved Hide resolved
pypesto/ensemble/utils.py Outdated Show resolved Hide resolved
pypesto/ensemble/utils.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
PaulJonasJost and others added 10 commits April 22, 2021 15:12
Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com>
Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com>
Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com>
Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com>
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/utils.py Show resolved Hide resolved
pypesto/ensemble/ensemble.py Show resolved Hide resolved
pypesto/ensemble/ensemble.py Show resolved Hide resolved
'Either the cutoff value was too small or the '
'result.optimize_result object might be empty.')
elif len(x_vectors) < max_size:
logger.info(f'The ensemble contains {len(x_vectors)} parameter '
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would make this a warning rather than an info, but that's a bit of a matter of taste...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would leave that to @dilpath in a democratic vote then 👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

both fine 🎸

pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
'Either the cutoff value was too small or the '
'result.optimize_result object might be empty.')
elif len(x_vectors) < max_size:
logger.info(f'The ensemble contains {len(x_vectors)} parameter '
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

both fine 🎸

pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
n_starts += 1

if n_starts*max_per_start > max_size:
logger.info(f'The number of starts that can contribute an '
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure how this looks on screen, but consider also using line breaks after e.g. 80 chars

pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/utils.py Outdated Show resolved Hide resolved
pypesto/ensemble/utils.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
Co-authored-by: Yannik Schälte <31767307+yannikschaelte@users.noreply.github.com>
@jvanhoefer
Copy link
Member

maybe before merging a more explanatory PR title would be nice (for future references, e.g. when preparing a new release)

Copy link
Member

@dilpath dilpath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
Comment on lines 842 to 843
indices = np.round(np.linspace(0, len(candidates)-1, n_per_start))
return candidates[indices.astype(int)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe convert to set? Otherwise is it possible that e.g. len(candidates)-1==1 and n_per_start==6 so indices==[0, 0, 0, 1, 1, 1] and so the returned vector indices are duplicates?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no n_per_start will give you a number that will definitely be lower than len(candidates) as per code so this should not be a problem. I do like lists more but that is a personal preference :D

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, if it's not intended for a user to call this method directly, then the name can be prefixed with a _ e.g. _indices_per_start

test/base/test_ensemble.py Outdated Show resolved Hide resolved
PaulJonasJost and others added 2 commits May 10, 2021 10:51
Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com>
@PaulJonasJost PaulJonasJost changed the title Extend Ensemle features Add reading Ensemble features from optimization result May 10, 2021
Copy link
Contributor

@paulstapor paulstapor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Only really minor stuff left

@@ -594,3 +749,97 @@ def check_identifiability(self) -> pd.DataFrame:
parameter_identifiability['parameterId']

return parameter_identifiability


def entries_per_start(fval_traces: List['np.ndarray'],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't "Sequence" do the trick here? Why do we sometimes give the typehint for Sequence and sometimes for List? There are no list-specific features used here... Actually, we only need "Iterable", right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would. note subtle differences, e.g. numpy.ndarray is an Iterable but no Sequence

pypesto/ensemble/ensemble.py Outdated Show resolved Hide resolved
Comment on lines +797 to +799
n_per_start = [min(n, max_per_start-decr)
for n in n_per_start]
decr += 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With a little hack on this, we could do something which is fully satisfying... However, I think that's fine from my side for the current version

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what would you have in mind? :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something that exactly matches max_size whenever possible?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no I meant what would your little hack be 🙂

@PaulJonasJost PaulJonasJost merged commit 69e834d into develop May 12, 2021
@PaulJonasJost PaulJonasJost deleted the feature_ensembles branch May 12, 2021 15:54
@yannikschaelte yannikschaelte mentioned this pull request May 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants