Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 New release maker #1

Merged
merged 4 commits into from
Jun 2, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
59 changes: 59 additions & 0 deletions .github/workflows/gh_release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# This workflow will load Python, run a script to generate assets, and then
# bundle a github release

name: Release generator
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fancy ⚡
It may be better to publish this as an action that way we can make updates to it without having to have everyone copy all the new files. See GH Docs on that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea. I'll look into it.

Copy link
Contributor Author

@fiendish fiendish May 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upon reading up about actions/workflows/deployment, I don't think this will be feasible. I called this an action, but really it's a workflow, and a workflow doesn't only encode what happens but also when it happens, which a published action doesn't do, but which is very important for this to work right.

One way to provide centralized reusability could be to use git submodules instead of copying the file around.


on:
pull_request:
types:
- closed

jobs:
create_release:
if: github.base_ref == 'master' && github.event.pull_request.merged && contains( github.event.pull_request.labels.*.name, 'release')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This requires a new release label? Is that preferable to a regex?

Copy link
Contributor Author

@fiendish fiendish May 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is probably preferable? Keep in mind that the label is added automatically by the user tool, so it's not like there's extra user effort involved.

We could use both to be double extra safe?

Copy link
Contributor Author

@fiendish fiendish May 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm open to being convinced against using a label though. I don't have a strong feeling about best practice either way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Label tag sound good because we currently label all PRs except for releases. We just need to make sure we have a standard release label across all repos.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the the label idea! Especially because the user doesn't have to remember to add it since the tool does it

runs-on: ubuntu-latest
steps:

- uses: actions/checkout@v2

- name: Set up Python 3.8
uses: actions/setup-python@v1
with:
python-version: 3.8

- name: Create tag from title and run asset script
id: find_tag_and_prepare_assets
run: |
TAG=$(echo ${{ github.event.pull_request.title }} | sed -E "s/^.*Release (.+\..+\..+)$/\1/g")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR titles should all be simpler than the full range of valid semver. But we can try it!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The official regex is pretty complex. As written, it breaks sed on macos. :\

Copy link
Contributor Author

@fiendish fiendish May 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, given that the CLI will only generate PRs with the the format 🔖 Release #.#.#, and that all this has to do is pick up the #.#.# part to turn it into a tag, which do you prefer:

# https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string
SEMVER="(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
TAG=$(echo ${{ github.event.pull_request.title }} | perl -pe "s/^.*Release ${SEMVER}\$/\1.\2.\3/g")

or

TAG=$(echo ${{ github.event.pull_request.title }} | sed -E "s/^.*Release (.+\..+\..+)$/\1/g")

or something else?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only other thing I can think of that doesn't require a specific PR title format would be to have the CLI create a second label #.#.# which this workflow step could use and then delete after its finished?

This might also come in handy for the user provided script in step 8 of What the CLI Does. I'm thinking for most of the repos with Python packages, the user provided script will be used to update the package's __init__.py with the current version. For the kf-model-fhir repo, the user script will also need the version to in order to tag the FHIR model's resources with it.

Copy link
Contributor Author

@fiendish fiendish May 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the user script will also need the version

Done in f4b2119

create a second label #.#.# which this workflow step could use and then delete after its finished

I like that. I'll see if it works and post that change if it does.
The downside is that it will pollute the label list if you use the CLI without using this GH action workflow. Thoughts on that?

Copy link
Collaborator

@znatty22 znatty22 May 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in f4b2119

I think we're talking about two different user scripts. You're referring to the prepare assets script that gets run after the release and I'm referring to the pre-release script that gets executed in step 8 of release build right before the release branch is pushed up to Github.

Although, now that you mention the environment variable.. Maybe the CLI could write the version to a environment variable (e.g. GH_RELEASE_VERSION, something more specific than TAG) and then the pre-release script can read the version from there if it needs it.

If we can do this, then I'd say let's keep this workflow step (get tag) simple and stick with your first regex to get the version from the PR title?

Copy link
Contributor Author

@fiendish fiendish May 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both scripts are now passed the version as input. If they want to set environment variables, they can.

echo "::set-output name=tag::$TAG"

SCRIPT=.github/prepare_assets.sh
if [ -f $SCRIPT ]; then
chmod u+x $SCRIPT
$SCRIPT $TAG
fi

- name: Create Release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ steps.find_tag_and_prepare_assets.outputs.tag }}
release_name: ${{ github.event.pull_request.title }}
body: ${{ github.event.pull_request.body }}
draft: false
prerelease: false

- name: Upload Assets
run: |
upload_url=${{ steps.create_release.outputs.upload_url }}
if [ -f .github/release_assets.txt ]; then
while IFS="" read -r FILE || [ -n "$FILE" ]
do
curl -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
-H "Content-Type: $(file -b --mime-type $FILE)" \
--data-binary "@$FILE" \
"${upload_url%\{*}?name=$(basename $FILE)"
done < .github/release_assets.txt
fi
35 changes: 35 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Dotfiles/dotdirs
.*

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# Environments
env/
venv/
ENV/
env.bak/
venv.bak/

# Distribution / packaging
build/
develop-eggs/
dist/
downloads/
eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
*.egg
MANIFEST

# Sphinx documentation
docs/_build/
67 changes: 65 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,65 @@
# d3b-release-maker
Tool to generate GitHub-based software releases
# D3b Release Maker

Tool to automate GitHub-based software releases for projects with the following
characteristics:

- Feature branching with merge-commits into master
- Gitmoji-style PR messages
- Semantic versioning tags for releases (<https://semver.org>)

## Part 1: CLI that generates release notes and creates a PR on GitHub for review

Install with pip:
`pip install git+https://github.com/d3b-center/d3b-release-maker.git`

Create a temporary auth token on GitHub and add it to your environment as
`GH_TOKEN`.

Then run: `release --help`

## Part 2: GitHub Actions Workflow that automates tagging releases and asset uploading

Copy `gh_releases.yml` from this repository's `.github/workflows/` directory
into `.github/workflows/` in your project repository. (Read more about GitHub
Actions and Workflows at <https://help.github.com/en/actions>)

If you want to attach binary assets to your GitHub releases, add a
`.github/prepare_assets.sh` script that receives one input argument with the
new release version and generates your assets and then creates
`.github/release_assets.txt` containing a list of the files you want to upload,
one per line, in the order that you want them to appear.

## What the parts do

### What the CLI does

When you run the `release build` command:

1. The CLI looks at the most recent git tag that looks like a Semantic Version.
2. Then it looks for all PRs that were merged into master after that tag which
do not themselves look like a release merge.
3. Emojis at the start of the PR titles are grouped and counted according to
the gitmoji emoji guide (<https://gitmoji.carloscuesta.me>).
4. Markdown is generated with links to PRs, merge commits, and committers and is
shown to the user.
5. The user is prompted to provide a type of version update (major, minor,
patch) based on the shown list of changes, and a new version number is
generated.
6. A fresh copy of your repository is cloned to a temporary location.
7. The new changes list is added to the CHANGELOG.md file (one is created if
one doesn't already exist).
8. An optional user-provided script is then run if you need to e.g. add the new
version number to files in your repository.
9. All newly modified files are commited with a special release commit and
pushed to a special release branch, and a Pull Request into master is opened
on GitHub for review.

### What the GitHub Actions Workflow does

When a special release branch is merged into master:

1. Your repository is tagged with the new semantic version, and a GitHub
release is created with the contents of the just-merged Pull Request body.
2. If a `.github/prepare_assets.sh` script exists, it is run.
3. If a `.github/release_assets.txt` file exists, any files listed in it are
then uploaded to the GitHub release.
Empty file added d3b_release_maker/__init__.py
Empty file.
60 changes: 60 additions & 0 deletions d3b_release_maker/cli.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
#!/usr/bin/env python
import click

from d3b_release_maker.release_maker import (
make_release,
new_notes,
)


@click.group(context_settings={"help_option_names": ["-h", "--help"]})
def cli():
"""
Container for the cli
"""
pass


def options(function):
function = click.option(
"--blurb_file",
prompt="Optional markdown file containing a custom message to prepend to the notes for this release",
default="",
help="Optional markdown file containing a custom message to prepend to the notes for this release",
)(function)
function = click.option(
"--repo",
prompt="The github repository (e.g. my-organization/my-project-name)",
help="The github organization/repository to make a release for",
)(function)
return function


@click.command(
name="preview", short_help="Preview the changes for a new release"
)
@options
def preview_changelog_cmd(repo, blurb_file):
new_notes(repo, blurb_file)


@click.command(name="build", short_help="Generate a new release on GitHub")
@options
@click.option(
"--project_title",
prompt="The title of the project",
default="",
help="This will be put before the release number in the generated notes",
)
@click.option(
"--pre_release_script",
prompt="Shell script to run before pushing the release to GitHub",
default="",
help="Shell script to run before pushing the release to GitHub",
)
def make_release_cmd(repo, project_title, blurb_file, pre_release_script):
make_release(repo, project_title, blurb_file, pre_release_script)


cli.add_command(preview_changelog_cmd)
cli.add_command(make_release_cmd)
49 changes: 49 additions & 0 deletions d3b_release_maker/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
"""
Default config values for release maker
"""

GITHUB_API = "https://api.github.com"
GITHUB_RAW = "https://raw.githubusercontent.com"

GH_TOKEN_VAR = "GH_TOKEN"
RELEASE_EMOJIS = "🏷🔖"
EMOJI_CATEGORIES = {
"Additions": {"✨", "🎉", "📈", "➕", "🌐", "🔀", "🔊"},
"Documentation": {"💡", "📝"},
"Removals": {"🔥", "➖", "⏪", "🔇", "🗑"},
"Fixes": {
"🐛",
"🚑",
"🔒",
"🍎",
"🐧",
"🏁",
"🤖",
"🍏",
"🚨",
"✏️",
"👽",
"👌",
"♿️",
"💬",
"🚸",
"🥅",
},
"Ops": {
"🚀",
"💚",
"⬇️",
"⬆️",
"📌",
"👷",
"🐳",
"📦",
"👥",
"🙈",
"📸",
"☸️",
"🌱",
"🚩",
},
}
OTHER_CATEGORY = "Other Changes"