Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indent block to pick up and raise exception if cmorizer data not found (TierX dir is not there) #1877

Merged
merged 3 commits into from
Mar 2, 2021

Conversation

valeriupredoi
Copy link
Contributor

@valeriupredoi valeriupredoi commented Oct 21, 2020

Before you start, please read our contribution guidelines.

Please discuss your idea with the development team before getting started, to avoid disappointment later. The way to do this is to open a new issue on GitHub. If you are planning to modify an existing functionality, please discuss it with the original author(s) by tagging them in the issue.


Tasks

  • Create an issue to discuss what you are going to do, if you haven't done so already (and add the link at the bottom)
  • Give this pull request a descriptive title that can be used as a one line summary in a changelog
  • Make sure your code is composed of functions of no more than 50 lines and uses meaningful names for variables
  • Circle/CI tests pass. Status can be seen below your pull request. If the tests are failing, click the link to find out why.
  • Preferably Codacy code quality checks pass, however a few remaining hard to solve Codacy issues are still acceptable. Status can be seen below your pull request. If there is an error, click the link to find out why. If you suspect Codacy may be wrong, please ask by commenting.
  • Please use yamllint to check that your YAML files do not contain mistakes
  • (Only if really necessary) Add any additional dependencies needed for the diagnostic script to setup.py, esmvaltool/install/R/r_requirements.txt or esmvaltool/install/Julia/Project.toml (depending on the language of your script) and also to package/meta.yaml for conda dependencies (includes Python and others, but not R/Julia)
  • If new dependencies are introduced, check that the license is compatible with Apache2.0

New recipe/diagnostic

  • Add documentation for the recipe to the doc/sphinx/source/recipes folder and add a new entry to index.rst
  • Add provenance information

Modified recipe/diagnostic

  • Update documentation for the recipe to the doc/sphinx/source/recipes folder
  • Update provenance information if needed
  • Assign the author(s) of the affected recipe(s) as reviewer(s)

New data reformatting script

  • Test the CMORized data using recipes/example/recipe_check_obs.yml, to make sure the CMOR checks pass without errors
  • Add the new dataset to the table in the documentation
  • Tag @mattiarighi in this pull request, so that the new dataset can be added to the OBS data pool at DKRZ and synchronized with CEDA-Jasmin

Modified data reformatting script

  • Test the CMORized data using recipes/example/recipe_check_obs.yml, to make sure the CMOR checks pass without errors
  • Tag @mattiarighi in this pull request, so that the updated dataset can be added to the OBS data pool at DKRZ and synchronized with CEDA-Jasmin

If you need help with any of the tasks above, please do not hesitate to ask by commenting in the issue or pull request.


Closes #issue_number

Copy link
Member

@bouweandela bouweandela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look good, with these changes the program will issue a warning for every Tier in which the dataset is not found, while it should only issue a warning if the dataset is not found in any Tier (the current behaviour).

@valeriupredoi
Copy link
Contributor Author

but at least it wouldn't throw a very opaque variable tier referenced before assignment if there is no Tier in the raw obs dir - and I think cautioning there is no data for TierY when there is data for TierX is useful since the user might have forgotten to add data for TierY there

@bouweandela
Copy link
Member

Maybe you could just change the message to something like 'Could not find the data in any tier'?

@valeriupredoi
Copy link
Contributor Author

the message doen't get stdout-ed man, it crashes with UnboundLocalError - I need to rewrite it to first check for anything in the raw obs, then see if there's any Tier 2-4 data

@valeriupredoi
Copy link
Contributor Author

valeriupredoi commented Feb 18, 2021

OK @bouweandela @remi-kazeroni - I have now refactored so it outputs a warning when it's actually needed, pls review 👍

Copy link
Contributor

@remi-kazeroni remi-kazeroni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition @valeriupredoi! I ran a few tests and it runs as expected: the warning is only issued if there is no TierX directory in the user's RAWOBS directory. Helps to understand the local variable tier referenced before assignment error in that case 👍

@valeriupredoi
Copy link
Contributor Author

excellent, cheers very much @remi-kazeroni 🍺 @bouweandela if you could stale your review and merge while at it, that'd be fab 👍

@valeriupredoi
Copy link
Contributor Author

cheers for the review gents, all merged and snug now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants