Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow multiple references for a cmorizer script #1953

Merged
merged 4 commits into from Dec 14, 2020

Conversation

SarahAlidoost
Copy link
Contributor

@SarahAlidoost SarahAlidoost commented Dec 10, 2020

Please discuss your idea with the development team before getting started, to avoid disappointment later. The way to do this is to open a new issue on GitHub.
If you are planning to modify existing functionality, please discuss it with the original author(s) by tagging them in the issue.

Before you start, please read our contribution guidelines.
To understand how we review and merge pull requests, have a look at our review guidelines.

Closes #1951


Checklist for technical review

  • Create an issue to discuss what you are going to do, if you haven't done so already (and add the link at the bottom)
  • The pull request has a descriptive title that can be used as a one line summary in the changelog
  • The code is composed of functions of no more than 50 lines and uses meaningful names for variables and follows the style guide
  • Documentation is available
  • Please use yamllint to check that your YAML files do not contain mistakes
  • (Only if really necessary) Add any additional dependencies needed for the diagnostic script to setup.py, esmvaltool/install/R/r_requirements.txt or esmvaltool/install/Julia/Project.toml (depending on the language of your script) and also to package/meta.yaml for conda dependencies (includes Python and others, but not R/Julia). Also check that the license of the dependency you want to add and any of its dependencies are compatible with Apache2.0.

New or updated recipe/diagnostic:

  • Documentation for the recipe/diagnostic is available in the doc/sphinx/source/recipes folder and an entry has been added to index.rst
  • Provenance information has been added and no warnings related to provenance are generated when running the recipe

New or updated data reformatting script:

  • Add a new dataset to the table in the documentation
  • Add a test for the CMORized data to recipes/example/recipe_check_obs.yml and run the recipe, to make sure the CMOR checks pass without errors
  • There are clear instructions on how to obtain the data
  • Tag @remi-kazeroni in this pull request, so that the new dataset can be added to the OBS data pool at DKRZ and synchronized with CEDA-Jasmin

Automated checks pass, status can be seen below the pull request:

  • Circle/CI tests pass. If the tests are failing, click the Details link to find out why.
  • Preferably Codacy code quality checks pass, however a few remaining hard to solve Codacy issues are still acceptable. If there is an error, click the link to find out why. If you suspect Codacy may be wrong, please ask by commenting.
  • The documentation is building successfully on readthedocs and looks well formatted, click the Details link to see it.

Checklist for scientific review

New or updated recipe/diagnostic:

  • The documentation for the new/updated recipes/diagnostics clearly describes what the recipe does and how to use it
  • The recipe runs successfully on your own machine/cluster or with the @esmvalbot without any modifications to the recipe and with all data specified in the recipe
  • The figure(s) and data look as expected from the literature and/or the paper that is reproduced by the recipe
  • The code contains comments with references to formulas, figures, tables, etc. that are used from papers/online resources

New or updated data reformatting script:

  • The numbers and units of the data look physically meaningful

If you need help with any of the items on the checklists above, please do not hesitate to ask by commenting in the issue or pull request.

@SarahAlidoost SarahAlidoost marked this pull request as ready for review December 10, 2020 15:34
@SarahAlidoost
Copy link
Contributor Author

@axel-lauer, @zklaus and @stefsmeets I am wondering if one of you can review this PR. Thanks in advance.

Copy link
Contributor

@stefsmeets stefsmeets left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Just a small suggestion to re-factor the failure condition.

Comment on lines +196 to +210
if bibtex_file.is_file():
reference_entry = bibtex_file.read_text()
if re.search("doi", reference_entry):
reference_doi.append(
f'doi:{re.search(pattern, reference_entry).group(1)}'
)
else:
reference_doi.append('doi not found')
logger.warning(
'The reference file %s does not have a doi.', bibtex_file
)
else:
reference_doi.append('doi not found')
logger.warning(
'The reference file %s does not exist.', bibtex_file
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just wondering if we can do this to get rid of the double failure condition:

Suggested change
if bibtex_file.is_file():
reference_entry = bibtex_file.read_text()
if re.search("doi", reference_entry):
reference_doi.append(
f'doi:{re.search(pattern, reference_entry).group(1)}'
)
else:
reference_doi.append('doi not found')
logger.warning(
'The reference file %s does not have a doi.', bibtex_file
)
else:
reference_doi.append('doi not found')
logger.warning(
'The reference file %s does not exist.', bibtex_file
success = False
if bibtex_file.is_file():
reference_entry = bibtex_file.read_text()
if re.search("doi", reference_entry):
reference_doi.append(
f'doi:{re.search(pattern, reference_entry).group(1)}'
)
success = True
if not success:
reference_doi.append('doi not found')
logger.warning(
'The reference file %s does not exist.', bibtex_file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stefsmeets thank you for the suggestion. Please note the failure and logger messages are not the same. The first failure is when there is not any doi in the bibtex file whereas the second indicates there is no a bibtex file for the tag. I think that clear logger messages are useful, what do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I completely looked over that. I agree with you, I just think this function got a bit too complicated for what it does.

esmvaltool/cmorizers/obs/utilities.py Outdated Show resolved Hide resolved
@axel-lauer
Copy link
Contributor

I just tested this new feature and it works nicely. @SarahAlidoost thanks for your quick help! Would you like to consider the changes proposed by @stefsmeets ? This could be merged then and resolves the problem we ran into in PR #1895.

Co-authored-by: Stef Smeets <stefsmeets@users.noreply.github.com>
@axel-lauer
Copy link
Contributor

Thanks @SarahAlidoost and @stefsmeets. This looks great and if there are no objections I would like to get this merged very soon.

@axel-lauer axel-lauer merged commit e53b5a0 into master Dec 14, 2020
@axel-lauer axel-lauer deleted the support_multiple_references branch December 14, 2020 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow multiple references for a cmorizer script
4 participants