Adds a new release GitHub Actions workflow #394
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dkliban
force-pushed
the
7868
branch
7 times, most recently
from
7c3fc88
to
d64f8de
Compare
|
Attached issue: https://pulp.plan.io/issues/7868 |
|
I think some of the scripts can go into |
daviddavis
reviewed
Jun 1, 2021
Comment on lines
1
to
80
| import argparse | ||
| import asyncio | ||
| import os | ||
| import textwrap | ||
|
|
||
| from pypi_tools import get_package_from_pypi | ||
|
|
||
| from git import Repo | ||
|
|
||
| helper = textwrap.dedent( | ||
| """\ | ||
| Create a new tag and build a Python package. | ||
|
|
||
| Example: | ||
|
|
||
| $ python .ci/scripts/create_tag_and_build_package.py 3.2.5 a96ba79cfbaae2f344c86a3641a | ||
|
|
||
| If the specified tag does not exist, a new tag is created in the repository. | ||
|
|
||
| If the tag already exists, this script checks if the package with such a version string | ||
| already exists on PyPI. If a package exists, the package is simply downloaded from PyPI | ||
| and saved as {{ plugin_dash }}-.tar.gz and {{ plugin_name }}-[tag]-py3-none-any.whl in the | ||
| 'dist' directory. | ||
|
|
||
| If the package does not exist on PyPI, two new packages are built with the names | ||
| {{ plugin_dash }}-[tag].tar.gz and {{ plugin_name }}-[tag]-py3-none-any.whl. They are | ||
| stored in the 'dist' directory. | ||
| """ | ||
| ) | ||
| parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter, description=helper) | ||
| parser.add_argument( | ||
| "desired_tag", | ||
| type=str, | ||
| help="The tag that should be created.", | ||
| ) | ||
| parser.add_argument( | ||
| "commit_sha", | ||
| type=str, | ||
| help="The commit that should be used for the tag.", | ||
| ) | ||
| args = parser.parse_args() | ||
| desired_tag = args.desired_tag | ||
| commit_sha = args.commit_sha | ||
|
|
||
| release_path = os.path.dirname(os.path.abspath(__file__)) | ||
| plugin_path = release_path.split("/.github")[0] | ||
|
|
||
| repo = Repo(plugin_path) | ||
|
|
||
| # Remove auth header config | ||
| with repo.config_writer() as conf: | ||
| conf.remove_section('http "https://github.com/"') | ||
| conf.release() | ||
|
|
||
| # Determine if a tag exists and if it matches the specified commit sha | ||
| tag = None | ||
| for existing_tag in repo.tags: | ||
| if existing_tag.name == desired_tag: | ||
| if existing_tag.commit.hexsha == commit_sha: | ||
| tag = existing_tag | ||
| else: | ||
| raise RuntimeError( | ||
| "The '{desired_tag}' tag already exists, but the commit sha does not match " | ||
| "'{commit_sha}'." | ||
| ) | ||
|
|
||
| # Create a tag if one does not exist | ||
| if not tag: | ||
| tag = repo.create_tag(desired_tag, ref=commit_sha) | ||
|
|
||
| # Checkout the desired tag and reset the tree | ||
| repo.head.reference = tag.commit | ||
| repo.head.reset(index=True, working_tree=True) | ||
|
|
||
| # Check if Package is available on PyPI | ||
| loop = asyncio.get_event_loop() | ||
| package_found = asyncio.run(get_package_from_pypi("{{ plugin_dash }}=={tag.name}", plugin_path)) | ||
|
|
||
| if not package_found: | ||
| os.system("python3 setup.py sdist bdist_wheel --python-tag py3") |
daviddavis
reviewed
Jun 1, 2021
| @@ -0,0 +1,41 @@ | |||
| import os | |||
There was a problem hiding this comment.
Where is this file used?
There was a problem hiding this comment.
I was going to re-use this for the client package also, but then I decided not to do it so that I could keep the code similar for python client and ruby client. I could move this code back into create_tag_and_build_package.py.j2 if you think that will be better.
There was a problem hiding this comment.
I am guessing create_tag_and_build_package.py.j2 went away?
daviddavis
reviewed
Jun 1, 2021
Comment on lines
1
to
11
| #!/bin/sh | ||
| set -e | ||
|
|
||
| curl -s -X POST https://api.github.com/repos/$GITHUB_REPOSITORY/git/refs \ | ||
| -H "Authorization: token $GITHUB_TOKEN" \ | ||
| -d @- << EOF | ||
| { | ||
| "ref": "refs/tags/$1", | ||
| "sha": "$2" | ||
| } | ||
| EOF |
fao89
reviewed
Jun 1, 2021
dkliban
force-pushed
the
7868
branch
2 times, most recently
from
27524da
to
f52a1ce
Compare
The new release workflow has 3 stages:
Build artifacts - build the plugin Python packages or downloads them from PyPI.
Test - build a container from the package in the previous stage. Build the Python
and Ruby clients. Build docs. Run unit and function tests using filesystem
and S3 storage.
Publish - publish all packages to PyPI if they are not present there. Publish
Ruby client to RubyGems.org if it's not there. Publish docs. Push
tag to GitHub.
fixes: #7868
https://pulp.plan.io/issues/7868
dkliban
force-pushed
the
7868
branch
2 times, most recently
from
96387cf
to
f747dc1
Compare
daviddavis
reviewed
Jun 11, 2021
| ".ci/scripts/release_requirements.txt", | ||
| ".ci/scripts/release.py", |
There was a problem hiding this comment.
👍 for remembering to deprecate files.
dkliban
force-pushed
the
7868
branch
2 times, most recently
from
26d7097
to
10ea1c8
Compare
daviddavis
approved these changes
Jun 14, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The new release workflow has 3 serial jobs:
Build artifacts:
Create change log using towncrier. Bump version to GA (from dev). Commits these changes. Bump version to the next Z.dev release and create a commit. If the changes already exist in the repository, the previous steps are skipped. Build the plugin Python package or download it from PyPI. The entire repository (including the dist directory with built package) is archived and uploaded to the next job.
Test:
Build a container from the package in the previous job. Build the Python client or install it from PyPI if it has already been published. Build the Ruby client. Build docs. Run unit and function tests using filesystem and S3 storage. Upload the Python and Ruby clients and the docs to be used by the next job.
Publish:
Publish plugin and client packages to PyPI if they are not present there. Publish Ruby client to RubyGems.org if it's not there. Publish docs. Push the commits created in the first job to GitHub. Push the tag to GitHub. Create a release on GitHub from the tag.
fixes: #7868
https://pulp.plan.io/issues/7868