Streamline the release process

[tsalo]

I think we could use integrations and Actions to simplify the release process, with most of the work happening on GitHub instead of on the release maker's local machine.

We've discussed this elsewhere (perhaps in a dev call?), but I can't find any notes on it. Elizabeth pointed out that the main blocker was that the documentation needs to be built locally, so I know the workflow I'm most familiar with won't work on its own, but I'm sure there are some elements that can be adopted. In particular, I think versioneer and a PyPi deployment Action would be most useful.

Benefits to the change

It would reduce the work necessary by maintainers to make a new release.

The current release process

1. Prepare a new release on a new branch.
2. Update whats_new.rst by cleaning up existing notes and adding a highlights section.
3. Manually update the version string in nilearn/version.py.
4. Add new release post to News section of documentation.
5. Submit a PR from the release branch.
6. Merge release branch PR.
7. Tag the merge commit locally.
8. Build the tagged version and push to PyPi, also locally.
9. Build documentation and push it to the GitHub Pages repo.
10. Update nilearn/version.py and whats_new.rst.

A pie-in-the-sky release process

0. Use integrations to manage version strings and release notes.
   • Use release-drafter to automatically compile the release notes, along with a GitHub Action to update a CHANGES/whatsnew file.
     • This would probably use PR names in the files, so it's probably not sufficiently detailed for nilearn's purposes, but I could be wrong.
   • Use versioneer to automatically manage the version string.
1. Manually clean up whats_new.rst and update the documentation News section in a single PR.
   • Automatically grabbing the three most recent posts should be doable, right?
2. Update the pre-drafted release notes on GitHub and include the appropriate tag and name for the new release.
3. Make the release on GitHub.
   • A GitHub Action will build and deploy to PyPi.
4. Build documentation and push to GitHub Pages repo.
   • I would love to automate this, but my understanding is that it's just not possible with current resources. Is that correct?

[bthirion]

"Automatically grabbing the three most recent posts should be doable, right?" -> note that you don't want the most recent, but the most important.

[tsalo]

Could that rule be formalized as major and/or minor releases, but not patches, or does it need to be more of judgment call?

[NicolasGensollen]

Could that rule be formalized as major and/or minor releases, but not patches, or does it need to be more of judgment call?

I must admit that for the highlights of the latest releases I picked some of the whatsnew entries (not necessarily three...) in a purely arbitrary way.

It would reduce the work necessary by maintainers to make a new release.

Most of the steps in the workflow you describe are easy and fast (with the exception of the documentation build which is pretty slow...), and you end up doing these only once every 3 months (our release cycle atm). So I never felt like it was a huge burden but I agree that you can mess things up and that automation could be helpful as long as the maintenance cost of the automated pipeline is not greater than the one of making releases "manually".
@tsalo Do you have examples of packages using such workflows that we could take a look at?

[tsalo]

Most packages I regularly contribute to use most of the same tools. In both NiMARE and tedana, the release process is just (1) update the release notes draft generated by release drafter and (2) publish the release.

The two big differences between those packages' release processes and the one I propose above are (1) the documentation building issue and (2) we don't have whats_new files, although we're looking into ways to automate that in tedana (see ME-ICA/tedana#676).

[jeromedockes]

We've discussed this elsewhere (perhaps in a dev call?), but I can't find any notes on it

maybe you are referring to the discussion that started here ?

I continue thinking that as pushing to pypi is only 2 commands once every 3 months, I would rather do it manually and avoid having the pypi password on github

[arokem]

With apologies for commenting from the peanut gallery: If the concern is about access to resources on pypi, you can limit that quite substantially by using an API token: https://pypi.org/help/#apitoken. But I would understand if that's not sufficient security in your opinion. If the API token is compromised, someone could potentially hack the nilearn pypi page and distro.

[jeromedockes]

You are right, if we decide to push to pypi from the ci we will use an api token.

but that still requires making the token available to the github action, and as that token allows deciding what runs when someone does pip install nilearn, it would raise a little bit the level of attention we have to pay to security notifications from ci platforms, and how careful we have to be not to mess up the ci config. I completely agree it is not a very big deal, but as the benefit would be saving 10 seconds every 3 months I lean a bit more towards keeping the manual deployment.

but if most people prefer automating it then it's ok!

[jeromedockes]

note that's just one of the points in the issue description, I don't have a strong opinion re. the other ones. If there was a way to avoid having to build the new doc locally that would certainly be useful because it takes a long time

[arokem]

We use this config to build documentation for pyAFQ on Github actions: https://github.com/yeatmanlab/pyAFQ/blob/master/.github/workflows/docbuild.yml. It's nice because it also uploads a package with the built docs on every PR push, so we can download and inspect these as part of the review, without having to fetch that branch.

[ymzayek]

With PR #4160 and #4182 the stable docs will be built in a remote github runner and then pushed to the github pages repo as soon as a release is published. The steps before that remain mostly the same but they are not too taxing IMO