A set of helper scripts and GitHub Actions to aid in automated releases of Python and npm packages.
-
Enforces best practices:
- Automated changelog for every release
- Published to test server and verified with install and import of dist asset(s)
- Commit message with hashes of dist file(s)
- Annotated git tag in standard format
- GitHub release with changelog entry
- Verified url links in markdown and reStructuredText files
- Verified integrity of Python manifest
- Revert to Dev version after release (optional)
- Forward port changelog entries into default branch
- Dry run publish on CI
-
Prerequisites (see checklist below for details):
- Markdown changelog
- Bump version configuration (if using Python), for example tbump
- Access token with access to target GitHub repo to run GitHub Actions
- Access token for the test PyPI registry stored as
TEST_PYPI_TOKEN
- If needed, access token for npm.
- When ready to make a release, a fork of
jupyter-releaser
and go to the Actions panel - Select the
Draft Changelog
workflow - Run the workflow with the version spec (usually the new version number)
- When the run completes, review the changelog PR that was opened, making any desired edits
- Merge the PR
- Return to the Actions panel in the
jupyter-releaser
fork - Select the
Draft Release
workflow
- Run the workflow with the same version spec as before, and an optional post version spec if you want to go back to a dev version in the target branch.
- When the workflow completes, go to the releases page in the target repository and verify that the new draft release is there with the correct changelog and dist files.
- Copy the url of the draft release.
- Run the
Publish Release
workflow from thejupyter-releaser
fork.
- If the release was on a backport branch, a forward port PR will have been opened against the default branch with the new changelog entry. Review and merge this PR.
To install the latest release locally, make sure you have pip installed and run:
pip install git+https://github.com/jupyter-server/jupyter-releaser
jupyter-releaser --help
jupyter-releaser build-python --help
jupyter-releaser check-links --help
All of the commands support CLI and Environment Variable Overrides.
The environment variables are defined by the envvar
parameters in the
command options in cli.py
. The environment variables unique to
jupyter-releaser
are prefixed with RH_
. A list of all env variables can be seen
by running jupyter-releaser list-envvars
.
The default values can also be overriden using a config file.
Options can be overridden using the options
section.
You can also define hooks to run before and after
commands in a hooks
section. Hooks can be a shell command to run or
a list of shell commands, and are specified to run before-
or after-
a command.
Note: the only invalid hook name is before-prep-git
, since a checkout of the target repository is not yet available at that point.
This is where jupyter-releaser
looks for configuration (first one found is used):
.jupyter-releaser.toml
pyproject.toml (in the tools.jupyter-releaser section )
package.json (in the jupyter-releaser property)
Example .jupyter-releaser.toml
:
[options]
dist_dir = mydist
[hooks]
before-tag-version = "npm run pre:tag:script"
Example pyproject.toml
section:
[tools.jupyter-releaser.options]
dist_dir = mydist
[tools.jupyter-releaser.hooks]
after-build-python = ["python scripts/cleanup.py", "python scripts/send_email.py"]
Example package.json
:
{
"name": "my-package",
"jupyter-releaser": {
"options": {
"dist_dir": "mydist"
},
"hooks": {
"before-publish-dist": "npm run pre:publish:dist"
}
}
}
Prep jupyter_releaser
:
- Clone this repository onto your GitHub user account.
- GitHub Access token with access to target GitHub repo to run GitHub Actions, saved as
GITHUB_TOKEN
in the repository secrets. - Access tokens for the test PyPI registry stored as
TEST_PYPI_TOKEN
- If needed, access token for npm.
Prep target repository:
- Switch to Markdown Changelog
- We recommend MyST, especially if some of your docs are in reStructuredText.
- Can use
pandoc -s changelog.rst -o changelog.md
and some hand edits as needed. - Note that directives can still be used
- Add HTML start and end comment markers to Changelog file - see example in CHANGELOG.md (view in raw mode)
- Add tbump support if using Python - see example metadata in pyproject.toml
- We recommend putting
setuptools
metadata insetup.cfg
and usingversion attr: <package_name>.__version__
, see examplesetup.cfg
- See documentation on
setup.cfg
metadata - If previously providing
version_info
, useget_version_info
fromjupyter_packaging
, sincetbump
requires the intact version string, e.g.
- We recommend putting
from jupyter_packaging import get_version_info
# Version string must appear intact for tbump versioning
__version__ = '1.4.0.dev0'
version_info = get_version_info(__version__)
- Add a GitHub Actions CI step to run the
check_release
action. For example:
- name: Check Release
if: ${{ matrix.python-version == '3.9' }}
uses: jupyter-server/jupyter_releaser/.github/actions/check-release@v1
env:
TWINE_PASSWORD: ${{ secrets.TEST_PYPI_TOKEN }}
with:
token: ${{ secrets.GITHUB_TOKEN }}
- Update or add
RELEASE.md
that describes the onboarding and release process, e.g.
# Release Workflow
- [ ] Set up a fork of `jupyter-releaser` if you have not yet done so.
- [ ] Run through the release process, targeting this repo and the appropriate branch
- Optionally add configuration to the target repository if non-standard options or hooks are needed.
- Optionally add workflow for
cancel
to cancel previous workflow runs when a new one is started - see cancel.yml - If desired, add
check_release
job, changelog, andtbump
support to other active release branches - Try out the
Draft Changelog
andDraft Release
process against a fork of the target repo first so you don't accidentally push tags and GitHub releases to the source repository.
- Create backport branches the usual way, e.g.
git checkout -b 3.0.x v3.0.1; git push origin 3.0.x
- When running the
Publish Release
Workflow, an automatic PR is generated for the default branch in the target repo, positioned in the appropriate place in the changelog.
- Manual Github workflow
- Inputs are the target repo, branch, and the version spec
- Bumps the version
- By default, uses tbump or bump2version to bump the version based on presence of config files
- We recommend
tbump
instead ofbump2version
for most cases because it does not handle patch releases well when using prereleases.
- We recommend
- By default, uses tbump or bump2version to bump the version based on presence of config files
- Prepares the environment
- Sets up git config and branch
- Generates a changelog (using github-activity) using the PRs since the last tag on this branch.
- Gets the current version and then does a git checkout to clear state
- Adds a new version entry using a HTML comment markers in the changelog file
- Optionally resolves meeseeks backport PRs to their original PR
- Creates a PR with the changelog changes
- Can be re-run using the same version spec. It will add new entries but preserve existing ones (in case they have been hand modified).
- Note: Pre-release changelog sections are not automatically combined, but you may wish to do so manually.
- Manual Github workflow
- Inputs are target repository, branch, version spec and optional post version spec
- Bumps version using the same method as the changelog action
- Prepares the environment using the same method as the changelog action
- Checks the changelog entry
- Looks for the current entry using the HTML comment markers
- Gets the expected changelog values using
github-activity
- Ensures that all PRs are the same between the two
- For Python packages:
- Builds the wheel and source distributions if applicable
- Makes sure Python dists can be installed and imported in a virtual environment
- For npm package(s) (including workspace support):
- Builds tarball(s) using
npm pack
- Make sure tarball(s) can be installed and imported in a new npm package
- Builds tarball(s) using
- Checks the package manifest using
check-manifest
- Checks the links in Markdown and reStructuredText files
- Adds a commit that includes the hashes of the dist files
- Creates an annotated version tag in standard format
- If given, bumps the version using the post version spec
- Pushes the commits and tag to the target
branch
- Publishes a draft GitHub release for the tag with the changelog entry as the text
- Manual Github workflow
- Input is the url of the draft release
- Downloads the dist assets from the release
- Verifies shas and integrity of release assets
- Publishes assets to appropriate registries
- If the tag is on a backport branch, makes a forwardport PR for the changelog entry
- Runs on CI in the target repository to verify compatibility and release-ability.
- Runs the
Draft Changelog
,Draft Release
, andPublish Release
actions in dry run mode - Publishes to the Test PyPI server
- Deletes the Release
- Does not make PRs or push git changes
Create a new manual PR to fix the PR and re-orient the changelog entry markers.
The release will fail to push commits because it will not be up to date. Delete the pushed tags and re-start with "Draft Changelog" to pick up the new PR.