Skip to content

Update release manager guide for providers #50822

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kaxil merged 1 commit into from
May 20, 2025
+16 −67
Merged

Update release manager guide for providers #50822

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 16 additions & 67 deletions dev/README_RELEASE_PROVIDERS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@
- [Commit the source packages to Apache SVN repo](#commit-the-source-packages-to-apache-svn-repo)
- [Publish the Regular distributions to PyPI (release candidates)](#publish-the-regular-distributions-to-pypi-release-candidates)
- [Add tags in git](#add-tags-in-git)
- [Publish release candidate documentation](#publish-release-candidate-documentation)
- [Prepare issue in GitHub to keep status of testing](#prepare-issue-in-github-to-keep-status-of-testing)
- [Prepare voting email for Providers release candidate](#prepare-voting-email-for-providers-release-candidate)
- [Verify the release candidate by PMC members](#verify-the-release-candidate-by-pmc-members)
Expand All @@ -47,7 +46,7 @@
- [Publish release to SVN](#publish-release-to-svn)
- [Publish the packages to PyPI](#publish-the-packages-to-pypi)
- [Add tags in git](#add-tags-in-git-1)
- [Publish final documentation](#publish-final-documentation)
- [Publish documentation](#publish-documentation)
- [Update providers metadata](#update-providers-metadata)
- [Notify developers of release](#notify-developers-of-release)
- [Send announcements about security issues fixed in the release](#send-announcements-about-security-issues-fixed-in-the-release)
Expand Down Expand Up @@ -486,63 +485,6 @@ If you want to disable this behaviour, set the env **CLEAN_LOCAL_TAGS** to false
breeze release-management tag-providers
```

## Publish release candidate documentation

Documentation is an essential part of the product and should be made available to users.
In our cases, documentation for the released versions is published in the staging S3 bucket, and the site is
kept in a separate repository - [`apache/airflow-site`](https://github.com/apache/airflow-site),
but the documentation source code and build tools are available in the `apache/airflow` repository, so
you need to run several workflows to publish the documentation. More details about it can be found in
[Docs README](../docs/README.md) showing the architecture and workflows including manual workflows for
emergency cases.

There are two steps to publish the documentation:

1. Publish the documentation to the staging S3 bucket.

The release manager publishes the documentation using GitHub Actions workflow
[Publish Docs to S3](https://github.com/apache/airflow/actions/workflows/publish-docs-to-s3.yml).

You should specify the final tag to use to build the docs and list of providers to publish
(separated by spaces) or ``all-providers`` in case you want to publish all providers
(optionally you can exclude some of those providers). You should use `staging` bucket to publish the release
candidate documentation.

After that step, the provider documentation should be available under the http://airflow.staged.apache.org URL
(also present in the PyPI packages) but stable links and drop-down boxes should not be yet updated.

2. Invalidate Fastly cache, update version drop-down and stable links with the new versions of the documentation.

Before doing it - review the state of removed, suspended, new packages in
[the docs index](https://github.com/apache/airflow-site/blob/master/landing-pages/site/content/en/docs/_index.md):
Make sure to use `staging` branch to run the workflow.

There are few special considerations when the list of provider is updated.

- If you publish a new package, you must add it to the list of packages in the index.
- If there are changes to suspension or removal status of a package, you must move it appropriate section.

- In case you need to make any changes - create the commit and push changes and merge it to `staging` branch.
in [airflow-site](https://github.com/apache/airflow-site) repository.

```shell script
cd "${AIRFLOW_SITE_DIRECTORY}"
branch="add-documentation-$(date "+%Y-%m-%d%n")"
git checkout -b "${branch}"
git add .
git commit -m "Add documentation for packages - $(date "+%Y-%m-%d%n")"
git push --set-upstream origin "${branch}"
```

Merging the PR with the index changes to `staging` will trigger site publishing.

If you do not need to merge a PR, you should manually run the
[Build docs](https://github.com/apache/airflow-site/actions/workflows/build.yml)
workflow in `airflow-site` repository to refresh indexes and drop-downs.

After that build from PR or workflow completes, the new version should be available in the drop-down
list and stable links should be updated, also Fastly cache will be invalidated.

## Prepare issue in GitHub to keep status of testing

Create a GitHub issue with the content generated via manual execution of the command below. You will use
Expand Down Expand Up @@ -607,12 +549,20 @@ Make sure the packages are in https://dist.apache.org/repos/dist/dev/airflow/pro
Send out a vote to the dev@airflow.apache.org mailing list. Here you can prepare text of the
email.

subject:
```shell script
export VOTE_DURATION_IN_HOURS=72
export IS_SHORTEN_VOTE=$([ $VOTE_DURATION_IN_HOURS -ge 72 ] && echo "false" || echo "true")
export SHORTEN_VOTE_TEXT="This is a shortened ($VOTE_DURATION_IN_HOURS hours vote) as agreed by policy set it https://lists.apache.org/thread/cv194w1fqqykrhswhmm54zy9gnnv6kgm"
export VOTE_END_TIME=$(LANG=en_US.UTF-8 TZ=UTC date -v+"${VOTE_DURATION_IN_HOURS}"H "+%B %d, %Y %H:%M %p")
export RELEASE_MANAGER_NAME="Elad Kalif"
export GITHUB_ISSUE_LINK="LINK_TO_GITHUB_ISSUE"
```

subject:

```shell script
cat <<EOF
[VOTE] Airflow Providers prepared on $(date "+%B %d, %Y")
[VOTE] Airflow Providers prepared on $(LANG=en_US.UTF-8 TZ=UTC date "+%B %d, %Y")
EOF
```

Expand All @@ -621,8 +571,8 @@ cat <<EOF
Hey all,

I have just cut the new wave Airflow Providers packages. This email is calling a vote on the release,
which will last for 72 hours - which means that it will end on $(TZ=UTC date -v+3d "+%B %d, %Y %H:%M %p" ) UTC and until 3 binding +1 votes have been received.

which will last for $VOTE_DURATION_IN_HOURS hours - which means that it will end on $VOTE_END_TIME UTC and until 3 binding +1 votes have been received.
$([ "$IS_SHORTEN_VOTE" = "true" ] && echo "${SHORTEN_VOTE_TEXT}" || echo "")

Consider this my (binding) +1.

Expand Down Expand Up @@ -661,7 +611,7 @@ This will allow us to rename the artifact without modifying
the artifact checksums when we actually release.

The status of testing the providers by the community is kept here:
<TODO COPY LINK TO THE ISSUE CREATED>
$GITHUB_ISSUE_LINK

The issue is also the easiest way to see important PRs included in the RC candidates.
Detailed changelog for the providers will be published in the documentation after the
Expand All @@ -672,7 +622,7 @@ You can find the RC packages in PyPI following these links:
<PASTE TWINE UPLOAD LINKS HERE. SORT THEM BEFORE!>

Cheers,
<TODO: Your Name>
$RELEASE_MANAGER_NAME

EOF
```
Expand Down Expand Up @@ -1194,7 +1144,7 @@ If you want to disable this behaviour, set the env **CLEAN_LOCAL_TAGS** to false
breeze release-management tag-providers
```

## Publish final documentation
## Publish documentation

Documentation is an essential part of the product and should be made available to users.
In our cases, documentation for the released versions is published in the `live` S3 bucket, and the site is
Expand Down Expand Up @@ -1234,7 +1184,6 @@ There are few special considerations when the list of provider is updated.

- If you publish a new package, you must add it to the list of packages in the index.
- If there are changes to suspension or removal status of a package, you must move it appropriate section.

- In case you need to make any changes - create the commit and push changes and merge it to `main` branch.
in [airflow-site](https://github.com/apache/airflow-site) repository.

Expand Down