ESMValTool v2.11.0 release

[mo-gill]

Manager	Buddy	Feature Freeze	Feature branch	Milestone
Met Office (@ehogan @mo-gill @chrisbillowsMO)	@bouweandela	2024-05-06

Minor Release	Status	Date	RTD	Release	conda-forge	PyPI	Docker
v2.11.0		2024-07-04	Docs	v2.11.0	esmvaltool	esmvaltool	docker

Related ESMValCore release: ESMValGroup/ESMValCore#2389

[mo-gill]

We are starting the ESMValTool release process today! 🚀

1. ESMValTool feature freeze - Steps 3.1 and 3.2

• A release branch is created and branch protection rules are set up so only the release manager (i.e. the person in charge of the release branch) can push commits to that branch.
• The creation of the release branch is announced to the ESMValTool development team along with the procedures to use the branch for testing and making last-minute changes (see next step).

[mo-gill]

We're waiting until PR #2785 is merged before creating a release branch.

[mo-gill]

2. Some additional testing of ESMValTool - Steps 4.1-4.5

• Run all the recipes to check that they still work and generate the overview HTML pages.
• Upload the results to the webpage at https://esmvaltool.dkrz.de/shared/esmvaltool/.
• Compare the results to those obtained with the previous release.
• Create a GitHub discussion to communicate about the results.
• If there are differences with the previous release, ask recipe maintainers or authors to review the plots and NetCDF files of their diagnostics, for example by mentioning them in the discussion.

[mo-gill]

This link says to open an issue rather than a GitHub discussion, so we've done that https://docs.esmvaltool.org/en/latest/community/release_strategy/detailed_release_procedure.html#open-an-issue-on-github

[mo-gill]

We opened an issue to detail the testing and comparison here: #3616

[mo-gill]

We uploaded the results to the webpage at https://esmvaltool.dkrz.de/shared/esmvaltool/v2.11.0rc1/

[mo-gill]

We've just asked recipe maintainers to review the plots: #3616 (comment)

[mo-gill]

3. Update the list of broken recipes - Step 4.7

• Update the list of broken recipes with new recipes that could not be run successfully during the testing. Open a separate GitHub issue for each failing recipe and assign the next milestone. Open an overview issue, see Issue #3484 for an example, and review past overview issues. Take action to ensure that the broken recipe policy is followed.

[mo-gill]

We'll give recipe maintainers the next week to check their recipes. On Wednesday next week we'll update the list of broken recipes and complete the release.

[ehogan]

I will follow the How to make an ESMValTool release instructions to get to the point where the release branch is created, then revert #3643, as discussed in ESMValGroup/ESMValCore#2445 (comment).

4. Check that all tests and builds work - Step 1

• Check that the nightly test run on CircleCI was successful.
• Check that the GitHub Actions test runs were successful.
• Check that the documentation builds successfully on readthedocs.
• Check that the Docker images are building successfully.

[ehogan]

5. Increase the version number and add release notes - Steps 2 & 3

The version number is automatically generated from the information provided by git using setuptools-scm, but a static version number is stored in CITATION.cff.

• Make sure to update the version number and release date in CITATION.cff. (See https://semver.org/ for more information on choosing a version number.)
• Make sure that the ESMValCore version that is being used is set to the latest version. See the dependencies section in order to find more details on how update the ESMValCore version.
• Use the script draft_release_notes.py to create a draft of the release notes.
• Open a discussion to allow members of the development team to nominate pull requests as highlights.
• Add the most voted pull requests as highlights at the beginning of changelog.
• After the highlights section, list any backward incompatible changes that the release may include. (The backward compatibility policy lists the information that should be provided by the developer of any backward incompatible change.)
• Make sure to also list any deprecations that the release may include, as well as a brief description on how to upgrade a deprecated feature.
• Review the results, and if anything needs changing, change it on GitHub and re-run the script until the changelog looks acceptable.
• Copy the result to the file doc/sphinx/source/changelog.rst.
• If possible, try to set the script dates to the date of the release you are managing.
• Make a pull request.
• Get it merged into main.

[ehogan]

See #3688 😊

[ehogan]

6. Create a release branch - Step 4

• Create a branch off the main branch and push it to GitHub.
• Ask someone with administrative permissions to set up branch protection rules for it so only you and the person helping you with the release can push to it.
• Announce the name of the branch in an issue and ask the members of the ESMValTool development team to run their favourite recipe using this branch.

[ehogan]

• Created and pushed branch, see v2.11.x release branch
• Reverted Updated esmf-related pins #3643, see see "Jun 26, 2024" commit on the v2.11.x release branch 😊

[ehogan]

#3689 cherry-picked, see "Jun 27, 2024" commit on the v2.11.x release branch 😊

[ehogan]

Following the conversation at #3700 (comment), I am re-creating the v2.11.x release branch as follows:

Get current version of v2.11.x release branch as backup

• git clone git@github.com:ESMValGroup/ESMValTool.git ESMValTool-v2.11.x
• cd ESMValTool-v2.11.x
• git checkout v2.11.x

Remove v2.11.x release branch

• cd ../ESMValTool
• git checkout main
• git pull
• git branch -d v2.11.x
• Remove branch from GitHub: https://github.com/ESMValGroup/ESMValTool/branches/yours
• git push

Create new release branch from the current version of main

• git tag v2.11.0rc1
• git push origin tag v2.11.0rc1
• git checkout -b v2.11.x

Update new release branch

• git cherry-pick c7abf572721e1e73abf6aa6c5b45f8f92d92fbe3
• git cherry-pick c0ab625aadac8a4330efa6b0158a6aa2e61bc052

Compare new release branch with old release branch

• cd ../
• diff -r ESMValTool ESMValTool-v2.11.x

  Files ESMValTool/esmvaltool/diag_scripts/emergent_constraints/__init__.py and
  ESMValTool-v2.11.x/esmvaltool/diag_scripts/emergent_constraints/__init__.py differ
  

This is ok, because I hadn't cherry picked #3704 into the old release branch, but it is included in the new release branch, see the "Jul 2, 2024" commit.

[ehogan]

The following PRs have been cherry-picked, see "Jul 03, 2024" commit on the v2.11.x release branch 😊

• Update AERONET data version #3692
• Update the list of datasets used in recipe_easy_ipcc.yml #3710
• Update the list of broken recipes for v2.11.0 #3706

[ehogan]

The following PRs have been cherry-picked, see "Jul 04, 2024" commit on the v2.11.x release branch 😊

• Pin matplotlib <3.9.0 on ESMValTool release branch #3712
• Update the version number and release notes for v2.11.0 #3695

[ehogan]

@ESMValGroup/esmvaltool-developmentteam The release is now available! 🥳 A big thanks to everyone who contributed! 🌟 Please let us know if you experience any issues with the new release.

[bouweandela]

Thanks for all your work on this @ehogan @mo-gill @chrisbillowsMO! ✨

[hb326]

Awesome work, @ehogan @mo-gill @chrisbillowsMO!! 🥇