Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parameterset catalogue #118

Merged
merged 51 commits into from Jul 2, 2021
Merged

Add parameterset catalogue #118

merged 51 commits into from Jul 2, 2021

Conversation

Peter9192
Copy link
Collaborator

@Peter9192 Peter9192 commented Jun 17, 2021

Here's an initial idea on how we could support a parameterset catalogue.

Adds

  • ewatercycle.parameter_sets.available_parameter_sets() + ewatercycle.parameter_sets.get_parameter_set() as shown in api notebook
  • ewatercycle.parameter_sets.download_example_parameter_sets() to download example parameter sets.
  • System setup notebook see Setup guide for system admins #120

Changes:

  • Moved MaskMap from parameter set to Lisflood.setup() see maskmap from lisflood parameterset to setup #121
  • In notebooks replaced SubversionCopier for parameter sets with get_parameter_set
  • Applied PyCharm formatting or black to touched files
  • Renamed config file in user dir from ~/.config/.ewatercycle/ewatercycle.yaml to ~/.config/ewatercycle/ewatercycle.yaml
  • AbstractModel now uses generics so model subclasses can have their own forcing class

Removed:

  • Models no longer have their own parameter set class, they all use same one

Not part of this PR:

@Peter9192
Copy link
Collaborator Author

Nice progress! @sverhoeven

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@sverhoeven
Copy link
Member

@Peter9192 I think the PR is ready for review. What do you think?

Copy link
Collaborator Author

@Peter9192 Peter9192 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a first pass, haven't tested or looked at the notebook yet. I have a few comments so far.

tests/models/test_lisflood.py Outdated Show resolved Hide resolved
ewatercycle/models/lisflood.py Show resolved Hide resolved
tests/models/test_lisflood.py Outdated Show resolved Hide resolved
ewatercycle/models/abstract.py Outdated Show resolved Hide resolved
ewatercycle/models/marrmot.py Outdated Show resolved Hide resolved
tests/models/test_abstract.py Outdated Show resolved Hide resolved
ewatercycle/models/abstract.py Show resolved Hide resolved
tests/parameter_sets/test_example.py Outdated Show resolved Hide resolved
@sverhoeven sverhoeven marked this pull request as ready for review June 30, 2021 12:56
Copy link
Collaborator Author

@Peter9192 Peter9192 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested with the notebooks. A few more comments about the functionality.

ewatercycle/parameter_sets/__init__.py Outdated Show resolved Hide resolved
return parametersets


def available_parameter_sets(target_model: str = None) -> Iterable[str]:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ewatercycle.parameter_sets.available_parameter_sets(target_model="lisflood") currently returns a generator, which is not ideal for the public API (and not consistent with the example notebook). It would be nice if we can incorporate this example in one of our notebooks, and ideally it should return a list of parameterset names or a dict with them names as keys.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed it to tuple of strings as that more closely matches to api notebook.

ewatercycle/parameter_sets/__init__.py Show resolved Hide resolved
ewatercycle/models/abstract.py Outdated Show resolved Hide resolved
ewatercycle/parameter_sets/default.py Show resolved Hide resolved
@sverhoeven
Copy link
Member

sverhoeven commented Jul 1, 2021

@Peter9192 could you have another look?

Copy link
Collaborator Author

@Peter9192 Peter9192 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there. I made a suggestion for that warning message. Also two minor points about the notebooks:

  • download_example_parameter_sets fails when one of the parameterset dirs already exists. Might be nice to have a 'skip_existing' and 'force_overwrite' option. Now, the CFG is also not updated, and my CFG.reload results in an old config file.
  • can we change id to doi in the example API notebook as well, and maybe execute all the commands in the cell that's about the parameter sets, instead of having fake output in comments?

parameters = [
('IrrigationEfficiency', self._get_textvar_value('IrrigationEfficiency')),
('IrrigationEfficiency', self._get_textvar_value('IrrigationEfficiency'), 0.75),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So here we pretend there's a default, even if it's not set in the model config? And then the model itself will fail?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it will fail when area.map or area.nc file does not exist

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted having a default in the model, will raise KeyError when MaskMap or IrrigationEfficiency can not be found in config.
An bmi model will return error when file from config does not exist

ewatercycle/models/abstract.py Outdated Show resolved Hide resolved
@sonarcloud
Copy link

sonarcloud bot commented Jul 2, 2021

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 5 Code Smells

94.8% 94.8% Coverage
0.0% 0.0% Duplication

@sverhoeven sverhoeven merged commit 911f449 into master Jul 2, 2021
@sverhoeven sverhoeven deleted the catalogue branch July 2, 2021 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants