Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build source and documentation in GitHub Actions #71

Merged
merged 6 commits into from Mar 19, 2024

Conversation

sethmlarson
Copy link
Collaborator

@sethmlarson sethmlarson commented Nov 14, 2023

Part of #66

This PR accomplishes two things that are related:

  • Moves the building of source and docs artifacts and testing of built artifacts to a GitHub Actions workflow.
  • Moves the signing of artifacts using GPG/Sigstore out of release.py (due to now being used for automation) to a task within run_release.py.

I've tested the GitHub Action on my own fork of release-tools, however I haven't tested the run_release.py script since it requires a lot of credentials and setup (would appreciate if it's possible to dry-run this sort of thing?)

I'm going to leave some comments of my own below based on this PR, happy to revise the implementation or the direction if this doesn't match expectations.

name: "Build Python source and docs artifacts"

jobs:
source-and-docs:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would like to add a GitHub Environment to allow staging and reviewing an execution w/ approvals. I can create a separate GitHub issue for this if it's desirable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have this for Windows installers so yeah, that would be nice!

- name: "Setup Python"
uses: actions/setup-python@65d7f2d534ac1bc67fcd62888c5f4f3d2cb2b236 # v4.7.1
with:
python-version: 3.11
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should update this to 3.x? Wish there was a way to track this with Dependabot.

uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
# Conditionally run this step if there is a 'docs/' directory.
# Docs aren't built for alpha or beta releases.
if: ${{ hashFiles('cpython/${{ inputs.cpython_release }}/docs') != '' }}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This conditional doesn't match the above, but I figure if docs got built then we'd want to upload them as artifacts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this run:
https://github.com/python/release-tools/actions/runs/8350750234/job/22857827913

the docs upload was skipped for whatever reason. They were built though. The source artifacts uploaded just fine. I don't immediately see why that would be so I just built the docs locally after for this release. I compared the logs from my local run with the docs build on GHA, they look the same. They both have the line:

2024-03-19T22:28:44.9625823Z chdir'ing to Python-3.10.14/Doc
2024-03-19T22:28:44.9626238Z Removing doc build artifacts

This comes from here:

release-tools/release.py

Lines 342 to 350 in c1fceb7

with pushd(os.path.join(archivename, 'Doc')):
print('Removing doc build artifacts')
shutil.rmtree('venv', ignore_errors=True)
shutil.rmtree('build', ignore_errors=True)
shutil.rmtree('dist', ignore_errors=True)
shutil.rmtree('tools/docutils', ignore_errors=True)
shutil.rmtree('tools/jinja2', ignore_errors=True)
shutil.rmtree('tools/pygments', ignore_errors=True)
shutil.rmtree('tools/sphinx', ignore_errors=True)

which you can see is immediately preceded by a shutil.copytree, which in my case at least copied everything where expected. So maybe hashFiles does something weird. IDK.

cwd=str(the_dir / "installation"),
text=True,
)
def wait_for_source_and_docs_artifacts(db: DbfilenameShelf) -> None:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This task I'd like feedback if this matches expectations of release managers, I saw there was another section that has "waiting" similar to this.


# We ask for human verification at this point since this commit SHA is 'locked in'
print()
print(f"Go to https://github.com/{origin_remote_github_owner}/cpython/commit/{commit_sha}")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are extra steps for ensuring that the commit SHA isn't on a fork and that there haven't been any unexpected upstream changes to the fork since the local commit.

run_release.py Show resolved Hide resolved
run_release.py Outdated Show resolved Hide resolved
run_release.py Outdated Show resolved Hide resolved
Comment on lines +496 to +497
while not all(path.exists() for path in wait_for_paths):
time.sleep(1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should exit if after a certain amount of seconds some files are still missing (e.g. because they haven't been created due to some error)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This follows the same model as the "waiting for binary installers" routine which waited forever. I'm not sure how release managers would like to use this tool so would need feedback if there's a preference.

Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Copy link

cpython-cla-bot bot commented Dec 4, 2023

All commit authors signed the Contributor License Agreement.
CLA signed

Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
@ambv
Copy link
Contributor

ambv commented Dec 6, 2023

I'll do a dry run of 3.13 alpha 3 with this after 3.12.1 goes out.

@ambv ambv merged commit c1fceb7 into python:master Mar 19, 2024
2 checks passed
@sethmlarson sethmlarson deleted the build-source-and-docs branch March 27, 2024 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants