Skip to content

Release Process

Jump to bottom Edit New page
Brett Graham edited this page Mar 14, 2023 · 54 revisions

The following steps assume that the asdf-format/asdf remote is named upstream.

Determine Release Number and Order

As described by the ASDF Release Cycle and Major Dependency Support Policy. ASDF follows a major, minor, bugfix versioning scheme a.b.c where a is the major version, b is the minor version, and c is the bugfix version. Moreover, ASDF maintains LTS and a rolling release. Make sure the release number you have selected follows this pattern, and does not already exist.

In general, it is expected that any bugfixes backported to LTS versions will appear in an LTS version of ASDF before appearing in the rolling release. In practice this means that when making a release as part of the rolling release, one will first make the LTS bugfix releases first, assuming there are bugfixes that have been backported to the LTS versions of ASDF which have not been previously released.

In the case where multiple releases (LTS and rolling) are required, it is best practice to prepare and make the releases (entirely) in the ascending order of release number. This is because GitHub only allows one milestone to be applied to each PR/issue, which has forced us to pick milestones to be the lowest release version the change will appear in. By doing this once can ensure all the changes slated for a given release have been made prior to making the release.

Determine the type of Release

In order for our backport system and related CI to function properly. We follow a strict release policy of which branches include which releases.

  • All major/minor releases must occur on their own release branches branched directly off the master branch. These be named a.b.x, where a is the major version and b is the minor the minor version.
  • All bugfix releases will occur directly on their release branch. That is the a.b.c will be made on the existing a.b.x branch.

In some cases the release branch may have already been created, even when no official release has been made on that version. This usually occurs when development is occurring on two distinct versions (usually ahead of a major release). As such we will divide the release process into two categories:

  1. Releases requiring a release branch to be created.
  2. Releases on existing branches.

Check Pull Requests

Before starting the main release process, we need to make sure that all PRs (and backports) destined for the given release have been merged.

Note that he ASDF CI ensures that all PRs have the appropriate milestone added to them automatically which is based upon the backport-/development label applied. Thus we can get a list of all the PRs (and issues) slated for a given release by using this milestone, which can be found by selecting the correct milestone at: https://github.com/asdf-format/asdf/milestones.

All issues and PRs for this milestone in this list should be resolved prior to continuing the release process. This can be done in two ways:

  1. This can be done by either merging PRs or closing issues which have already been resolved.
  2. Re-milestoning PRs and issues. Note that PR milestones are set by their backport-/development label, not by setting the milestone manually.
    • If you determine that a PR should not be part of the release you are making then reassess the backport-/development label and adjust the label accordingly.
    • If these labels should not be adjusted, then remove the milestone and mark the PR as a draft. Any such PRs will be taken out of draft after the post-release steps.

Releases from existing branches

If the branch you are releasing from already exists, then you must check that no manual backports are needed (because if a manual backport is needed, then no PR will be created so that can appear in the milestone list).

The meeseeksmachine robot will mark any PR that it failed to backport with the label Still Needs Manual Backport. Go to this label and then select the backport- label relevant for your PR. You now need to resolve all of these PRs.

To do this, for each PR:

  1. Check PR has already been backported, either manually or via a manual trigger of meeseeksmachine. In this case, the PR's comments should include a link to the PR in question. In this case simply remove the Still Needs Manual Backport label from the PR.

  2. If the PR has not been backported try manualling retriggering meeseeksmachine to automatically backport the PR by commenting: @meeseeksdev backport to a.b.x.

    • If meeseeksmachine is successful, then you will see the a backport PR opened. Remove the Still Needs Manual Backport Label and then merge the backport PR when its CI passes.
    • Otherwise, meeseeksmachine will make a comment replying that something went wrong and include directions on how to generate a manual backport.

  3. Now you will have to manually create a backport PR. To do this, follow the procedure meeseeksbot details. Note that it will require you to manually resolve some sort of merge conflict successfully. Once the manual backport PR is open (against the a.b.x branch):

    1. Remove the Still Needs Manual Backport label from the original branch.
    2. Add a Manual Backport and the correct backport- label to the new backport branch.

  4. If you determine the PR should not be backported, adjust the backport- label (and milestone for merged PRs), then remove the Still Needs Manual Backport label.

Once all the Still Needs Manual Backport PRs have been addressed, double check the milestone to make sure they have all been merged correctly.

Prepare master for release

The steps should be undertaken on the master branch:

  1. Update the project changelog (CHANGES.rst in the repo root) and change "unreleased" next to your intended release to the current date in YYYY-MM-DD format.

  2. Open a PR and containing those changes.

    1. When opening this PR make sure that the correct backport-/development label is added. This will ensure the meeseeksmachine bot will backport the changes to the correct branch if necessary.

    2. Once the CI has passed and all branch protections have been resolved, merge the PR.

  3. If a backport- label was applied, then ensure that the meeseeksmachine successfully opened a PR against the correct release branch and ensure that this PR automatically merges.

Create a release branch

If a release branch already exists for the version of ASDF being released (i.e. you are releasing a.b.c and a.b.x already exits), then skip this section and move to the next section. Otherwise, you will need to create a release branch.

  1. Fetch and checkout the upstream master:
$ git fetch upstream
$ git checkout upstream/master
  1. Inspect the log to ensure that no commits have snuck in since your changelog updates:
$ git log
  1. Create a new release branch. The name of the release branch should share the major and minor version of your release version, but the patch version should be x. For example, when releasing 1.8.0, name the branch 1.8.x (note the trailing .x).
$ git checkout -b a.b.x
  1. Push the branch to the upstream remote:
$ git push -u upstream HEAD

Create the release tag

At this point, the release branch for your release should exist and have all the desired changes in it assuming you followed earlier. Now we can create the release tag:

  1. Checkout the release branch:
$ git checkout a.b.x
  1. Create an annotated tag with a name that matches your intended release:
$ git tag -a a.b.c -m "Tagging a.b.c release on a.b.x release branch"
  1. Push the new tag to the upstream remote:
$ git push upstream a.b.c

Review CI results on the new tag

The creation of the tag should have triggered the CI, Downstream, and s390x workflows. Find the results here, and ensure all the relevant CI is passing before proceeding. In particular, pay close attention to any unexplained warnings or failures in the Downstream workflow, as this will indicate if the release will break something in one of our main downstream packages.

Run Regression Tests

Once the release branch is situated, it's a good idea to confirm that our release candidate doesn't break other packages' regression tests. In particular, it is important to check the regression tests for:

Tag master

If we're not already working on a later release in master, then we'll need to tag the HEAD of master with a development tag. This allows the setuptools-scm to identify code installed from master as the latest version. For example if version 1.1.1 was released, add a 1.1.2.dev tag.

$ git fetch upstream
$ git checkout upstream/master
$ git tag -a a.b.d.dev -m "Tagging a.b.d.dev on master"
$ git push upstream a.b.d.dev

Publish Release

We publish the asdf package on GitHub, PyPI and conda-forge.

Publish to GitHub releases

  1. Visit the ASDF repository's releases page. You should see the new tag for your release version listed at the top, without a link or description.

  2. Click Draft a new release.

  3. Select the existing tag, and title the release the same as the tag (i.e., a.b.c).

  4. Copy the contents of your newest CHANGES.rst entries to the description field. Change any formatting from reStructuredText to markdown.

  5. Press the Publish release button.

Publish to PyPI

This is performed automatically by the Publish to PyPI action, which will be triggered by any new GitHub release. Simply, watch for a successful result here. Then confirm that it appears on PyPi

Publish to conda-forge

  1. Wait or the regro-cf-autotick-bot user to open a PR against the asdf-feedstock repository. This is triggered by the appearance of the new package version on PyPi and can take some time (i.e., check back tomorrow).

  2. Make any necessary updates to the bot's branch and merge it. This will be required if the release included changes to ASDF's dependencies.

Confirm successful readthedocs build

New tags on the repository should provoke a readthedocs build that automatically activates and builds the new version, and sets those new docs as the default. Confirm that the asdf readthedocs home shows your new What's New in ASDF a.b.c? section.

Update the Labels, Milestones, and CI

Once, the package is published it is best to go ahead and update the "flow" for the ASDF package CI. This will insure that the backport and milestone system operate correctly.

Labels

ASDF uses labels to help control where and/how code gets moved to different release branches. Thus it is important to update these.

New Label

If you created a new release branch, got to labels and create a new label:

  • Name:
backport-a.b.x
  • Description:
on-merge: backport to a.b.x
  • Color:
#E56A98

for your new release branch. It is very important that the description and name follow the exact formulation given so that the CI and bot correctly interpret them.

Update Existing Labels

If a new major or minor release occurred of the rolling-release, then per ASDF Release Cycle and Major Dependency Support Policy we no longer need to maintain the older versions of the rolling-release. Thus we can "decommission" the backport- label(s) for those branches, to do this edit the label's name so that it begins with :zzz: (so that its name starts with :zzz:). This will prevent any of the CI or bots from triggering on this label.

Milestones

Milestones help us track that changes are making it to the correct releases, so it is important to make sure our milestones remain up to date.

New Milestone(s)

Create the following milestones if applicable:

  • One for the next bugfix release(s) such as 2.15.1.
  • One for the next development release such as 3.1.0.

These should cover all the new development directions.

Close Milestone(s)

The main thing is to close the milestone which was used for the last release (you should have opened new milestone(s) to replace this in the last step). In addition, close any milestones for versions we will not do any further support for (i.e. if 3.2.0 is released then the 3.1.2 can be closed).

Update the CI

Finally, the Label Checker CI job needs to be updated to reflect the backport- label and milestone changes.

This should be so that:

  1. All active backport- labels should appear in: any_of command.
  2. set_milesone sets the milestone to the smallest value possible for each backport- label.

Reopen Draft PRs

Any PRs set as draft in Check Pull Requests should now be re-opened, adjusting labels and milestones as needed.

Notify the mailing lists

Send a triumphant email to asdf-developers@googlegroups.com and asdf-users@googlegroups.com to announce the new release.

Maintain the stable branch (after no issues have been found for some time)

The stable branch points to the latest official release of asdf. If the current release has become the latest, then the next step is to rewrite the stable branch to point our new tag.

$ git checkout stable
$ git reset --hard a.b.c
$ git push upstream stable --force
Add a custom footer

Clone this wiki locally