Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade the RTW to work with ESMValTool v2.8.0 #3169

Merged
merged 7 commits into from Jun 26, 2023
Merged

Upgrade the RTW to work with ESMValTool v2.8.0 #3169

merged 7 commits into from Jun 26, 2023

Conversation

ehogan
Copy link
Contributor

@ehogan ehogan commented May 4, 2023 •
edited

Description

Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

To help with the number of pull requests:

@ehogan ehogan self-assigned this May 4, 2023
@ehogan ehogan marked this pull request as ready for review May 4, 2023 20:00
@ehogan ehogan requested a review from mo-tgeddes May 4, 2023 20:00
@ehogan
Copy link
Contributor Author

ehogan commented May 4, 2023

@mo-tgeddes I have just opened #3170 to discuss the pre-commit changes in the PR, but this PR should resolve the issues you were seeing. I have added some useful print statements and updated the KGOs at the MO.

Just in case you need a reminder on how to run this:

export CYLC_VERSION=8
cd esmvaltool/utils/recipe_test_workflow/recipe_test_workflow/
cylc vip -O metoffice

😊

@ehogan ehogan linked an issue May 5, 2023 that may be closed by this pull request
Upgrade the RTW to work with ESMValTool v2.8.0 #3168
Closed
This was referenced May 5, 2023
Closed
Open
@ehogan ehogan added the Recipe Test Workflow (RTW) Items relevant to the Recipe Test Workflow label May 5, 2023
mo-tgeddes
mo-tgeddes reviewed Jun 2, 2023
View reviewed changes
Copy link
Contributor

@mo-tgeddes mo-tgeddes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, when running this in the job.err file I get this:

path_to_run_dir/lib/ESMValCore/esmvalcore/config/_config_validators.py:340: ESMValCoreDeprecationWarning: The configuration option or command line argument `offline` has been deprecated in ESMValCore version 2.8.0 and is scheduled for removal in version 2.10.0. Please use the options `search_esgf=never` (for `offline=True`) or `search_esgf=when_missing` (for `offline=False`) instead. These are exact replacements.
  warnings.warn(deprecation_msg, ESMValCoreDeprecationWarning)

Since this PR is updating the RTW to work with the new environment should this be addressed here or do you think we should open a new PR and just make sure it is addressed for 2.10.0?

@ehogan
Copy link
Contributor Author

ehogan commented Jun 6, 2023

Hi, when running this in the job.err file I get this:

path_to_run_dir/lib/ESMValCore/esmvalcore/config/_config_validators.py:340: ESMValCoreDeprecationWarning: The configuration option or command line argument `offline` has been deprecated in ESMValCore version 2.8.0 and is scheduled for removal in version 2.10.0. Please use the options `search_esgf=never` (for `offline=True`) or `search_esgf=when_missing` (for `offline=False`) instead. These are exact replacements.
  warnings.warn(deprecation_msg, ESMValCoreDeprecationWarning)

Since this PR is updating the RTW to work with the new environment should this be addressed here or do you think we should open a new PR and just make sure it is addressed for 2.10.0?

I don't see this message (all my job.err files are empty). After some investigation, I believe this warning is raised because offline exists in your personal ESMValTool user configuration file and not in anything to do with the RTW. It might be possible to make changes to ESMValCore to prevent this from happening, but we can open a new issue to address that :)

mo-tgeddes
mo-tgeddes approved these changes Jun 14, 2023
View reviewed changes
Copy link
Contributor

@mo-tgeddes mo-tgeddes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested the pre-commits by running them on configure.py. Everything runs as expected and changes look good. 😄 The new print statements are very helpful!

@ehogan ehogan mentioned this pull request Jun 26, 2023
Open
@ehogan
Copy link
Contributor Author

ehogan commented Jun 26, 2023

Hi, when running this in the job.err file I get this:

path_to_run_dir/lib/ESMValCore/esmvalcore/config/_config_validators.py:340: ESMValCoreDeprecationWarning: The configuration option or command line argument `offline` has been deprecated in ESMValCore version 2.8.0 and is scheduled for removal in version 2.10.0. Please use the options `search_esgf=never` (for `offline=True`) or `search_esgf=when_missing` (for `offline=False`) instead. These are exact replacements.
  warnings.warn(deprecation_msg, ESMValCoreDeprecationWarning)

Since this PR is updating the RTW to work with the new environment should this be addressed here or do you think we should open a new PR and just make sure it is addressed for 2.10.0?

I don't see this message (all my job.err files are empty). After some investigation, I believe this warning is raised because offline exists in your personal ESMValTool user configuration file and not in anything to do with the RTW. It might be possible to make changes to ESMValCore to prevent this from happening, but we can open a new issue to address that :)

New issue opened: ESMValGroup/ESMValCore#2113 :)

@ehogan ehogan merged commit b307854 into recipe_test_workflow_prototype Jun 26, 2023
7 of 8 checks passed
@ehogan ehogan deleted the 3168_update_RTW_to_v2.8.0 branch June 26, 2023 12:55
ehogan added a commit that referenced this pull request Jun 26, 2023
@ehogan
Merge branch 'recipe_test_workflow_prototype' into separate_install_c…
d8efedf 
…old_and_clone_task_rtw

* recipe_test_workflow_prototype: (199 commits)
  #3169: Upgrade the RTW to work with ESMValTool v2.8.0
  [Condalock] Update Linux condalock file (#3237)
  Modified links to the tutorial (#3236)
  Add ESMValCore release `v2.8.1` into the documentation (#3235)
  Generate climatology on the fly for AutoAssess soil moisture (#3197)
  New recipe and diagnostic for Arctic-midlatitude research (#3021)
  Fixed pandas diagnostics for pandas>=2.0.0 (#3209)
  Update obs4MIPs dataset to the current naming scheme in recipe_smpi.yml (#2991)
  Add variable long names to provenance record in monitoring diagnostics (#3222)
  Extension of NASA MERRA2 CMORizer (cl, cli, clivi, clw, clwvi) (#3167)
  Remove "fx_variable" from recipe_wenzel14jgr.yml (#3212)
  [Condalock] Update Linux condalock file (#3217)
  Add Seaborn diagnostic (#3155)
  Remove fx_variables from ipccwg1ar5ch9 recipes (#3215)
  Remove "fx_variable" from recipe_tebaldi21esd.yml (#3211)
  Update recipe_impact.yml to work with newer versions of `pandas` (#3220)
  Use ESMValCore v2.9.0 release candidates (#3219)
  [Github Actions ] Check if python minor version changed after Julia install in development installation test (#3213)
  New plot_type 1d_profile in monitor  (#3178)
  Add support for using a dask distributed scheduler (#3151)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mo-tgeddes mo-tgeddes mo-tgeddes approved these changes

Assignees

@ehogan ehogan

Labels
Recipe Test Workflow (RTW) Items relevant to the Recipe Test Workflow
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Upgrade the RTW to work with ESMValTool v2.8.0
2 participants
@ehogan @mo-tgeddes