Convert CI/CD to GitHub actions. #391

martin-chatterjee · 2024-02-01T20:31:42Z

Objective

Convert CI/CD setup from Travis-CI to GitHub Actions.

Motivation

@mottosso commented here, in PR #376:

Calling in all assists!
(...) We're currently on Travis-CI, because in the past it used to be free.. So part of the challenge would be converting it into GitHub Actions. (...)."

My long term goal is to have PySide6 support in Qt.py.

I see this PR as the first step before we can:

modernise/extend the test suite.
add test coverage to any future PySide6 support.
run the test suite against more modern VFX reference platform versions.
(Potentially using the official ASWF docker containers?)

Details

The configurations for GitHub Actions live in .github/workflows/.

I have separated the previous Travis-CI configuration into two separate workflows.

1. Run the test suite

→ .github/workflows/run-tests.yml

Runs the test suite matrix on ubuntu-latest GitHub runners, using the same docker images.
Runs on every push to any branch.

2. Release to PyPi

→ .github/workflows/release-to-pypi.yml

Releases to PyPi, using PyPi's own official GitHub PyPi Publish actions.
Runs on every push of a tag, as long as repository owner is mottosso.
ℹ️ Expects the GitHub repo to be set up as a "Trusted Publisher" against the PyPi Qt.py project.
ℹ️ PR is currently set up to publish to test.pypi.org, for testing purposes.

Test instructions

Testing this PR requires a bit more effort, because it needs to be pushed to mottosso/Qt.py in order to be able to test GitHub Actions.

Preparation

1. Check out `mottosso/Qt.py`:

mkdir -p ~/review-PR-391/
cd ~/review-PR-391/

git clone git@github.com:mottosso/Qt.py.git
cd Qt.py/

2. Add `martin-chatterjee/Qt.py` as a remote named `"review"`, and fetch:

git remote add review git@github.com:martin-chatterjee/Qt.py.git
git fetch review

Test "Run test suite".

1. Check out PR branch, push to `origin`:

git checkout -b pr-391-review review/convert-to-github-actions
git push origin pr-391-review

→ Visit https://github.com/mottosso/Qt.py/actions, to see the resulting workflow run.

Test "Release to PyPi".

ℹ️ This PR is set up to release to `test.pypi.org`.

→ This must be changed after all tests, and before a potential merge into main.

1. Make sure that `mottosso/Qt.py` is registered as a "Trusted Publisher".

In a nutshell, a specific GitHub repo can be registered as a "Trusted Publisher" for a specific PyPi project.
(→ More details here)

❗️ Please make sure to register mottosso/Qt.py for PyPi project Qt.py on both pypi.org and test.pypi.org.

2. Change project version to a unique version.

ℹ️ test.pypi.org and pypi.org only accept uploads for unique versions that have never been uploaded before.

→ Change version in Qt.py to something like:

__version__ = "1.3.9.dev1"

3. Commit and push.

git add .
git commit -m "Qt.py: Update version to '1.3.9.dev1'."  
git push origin pr-391-review

4. Create a test release tag, and push.

git tag 1.3.9.dev1-test-release
git push origin 1.3.9.dev1-test-release

→ Visit https://github.com/mottosso/Qt.py/actions, to see the resulting workflow run.

→ Visit https://test.pypi.org/project/Qt.py/, to see the released version.

'pgrep Xvfb' identifies the process by name, which is a more robust test. The output of 'ps aux | grep Xvfb' inside the Docker container running on Apple silicon is slightly different and incompatible with the previous while condition: $ ps aux | grep Xvfb root 3025 0.0 0.4 1451928 32588 pts/0 Sl 14:25 0:00 /rosetta/rosetta /usr/bin/Xvfb Xvfb :99 -screen 0 1024x768x16 root 3142 0.0 0.0 1206448 7808 pts/0 S+ 14:32 0:00 /rosetta/rosetta /usr/bin/grep grep --color=auto Xvfb

Prevent potential infinite loop if Xvfb is not able to start up, for whatever reason. Exit with exit code 124 ("Command timed out") after 60 seconds.

One-to-one conversion of the previous Travis CI based testing workflow. ❗️ Currently runs on every push ❗️

A separate Github Action will take care of the package deployment to PyPi. Therefore, the pip installation within the Docker container can be removed.

'Run tests' should not run on pushed tags, otherwise we it would get run twice unneccesarily during release tag push.

- Uses PyPA's release Github action. - The Github repo needs to be set up as a 'Trusted Publisher' on pypi.org. - ❗️ For testing purposes this workflow currently releases to test.pypi.org ❗️ → This needs to be removed before a potential merge into main.

Modern twine/build expects long_description to be set. This way 'README.md' will be displayed on https://pypi.org/project/Qt.py/.

This way it will also render correctly on https://pypi.org/project/Qt.py/.

❗️ Currently references 'martin-chatterjee/Qt.py', for testing purposes. ❗️

❗️ Currently states 'version 1.3.9'. This is an assumption, and needs to be changed before merge. ❗️

Test-releases to 'test.pypi.org' as project 'martin-chatterjee-Qt.py'.

CLAassistant · 2024-02-01T20:31:50Z

All committers have signed the CLA.

mottosso · 2024-02-02T07:38:39Z

Hi @martin-chatterjee, fantastic work, thanks for doing this!

My long term goal is to have PySide6 support in Qt.py.

Yes, agreed.

(Potentially using the official ASWF docker containers?)

This wasn't an option in the past (2020 and earlier), but nowadays it sounds likely and sensible.

I'm going to try and trigger a few builds for this so we can make sure things work, but I think we need to actually merge and tag for this to happen so I've changed this base to temp, let's continue from there!

mottosso · 2024-02-02T07:55:07Z

Looks like a success!

Great work, I know from experience this must have taken you a lot of sweat and tears; dealing with cloud-based CI is forever and always a huge pain.

Merging this into master now and distributing the new 1.3.9 version.

Merge #391

mottosso · 2024-02-02T08:00:46Z

Also gotta say, this is one of the best PR's I've ever gotten. An example in how to make a good first impression, how to get your PR merged quick and how to pay attention to details, with even the README updated to reflect the change. Well done.

martin-chatterjee · 2024-02-02T08:54:23Z

Thanks for the kind words @mottosso, really appreciate it. 🙂🙏

Stoked to see this one being reviewed, approved and merged so fast – keep up the good work! 🙌

martin-chatterjee added 20 commits January 4, 2024 14:54

run_docker.sh: Add execution permissions.

02e0e77

entrypoint.sh: Exit with code 124 if Xvfb is not running after 60 secs.

95f7d6b

Prevent potential infinite loop if Xvfb is not able to start up, for whatever reason. Exit with exit code 124 ("Command timed out") after 60 seconds.

Github Actions: Add 'run-tests' workflow.

782c2db

One-to-one conversion of the previous Travis CI based testing workflow. ❗️ Currently runs on every push ❗️

entrypoint.sh: Remove obsolete pip installation.

a2cadee

A separate Github Action will take care of the package deployment to PyPi. Therefore, the pip installation within the Docker container can be removed.

run-tests.yaml: Only run on pushed commits, not on pushed tags.

1c1a99f

'Run tests' should not run on pushed tags, otherwise we it would get run twice unneccesarily during release tag push.

Rename Github workflow config file suffix to '.yml'.

ba39275

setup.py: Pass content of 'README.md' in as long_description.

d90800a

Modern twine/build expects long_description to be set. This way 'README.md' will be displayed on https://pypi.org/project/Qt.py/.

README.md: Load logo with absolute URL.

f7e2064

This way it will also render correctly on https://pypi.org/project/Qt.py/.

README.md: Replace Travis CI badge with Github Actions badge.

1618938

❗️ Currently references 'martin-chatterjee/Qt.py', for testing purposes. ❗️

README.md: Add CI changes to News section.

636dc16

❗️ Currently states 'version 1.3.9'. This is an assumption, and needs to be changed before merge. ❗️

Update documentation.

f117855

setup.py: Add support for Python 3.7 – 3.10.

cd4cb74

Remove obsolete Travis CI config file.

c1f5885

Release test: Change project name and version.

7734178

Test-releases to 'test.pypi.org' as project 'martin-chatterjee-Qt.py'.

Revert project name and version after release test.

e9a79fb

Release to PyPi: Un-comment 'repository_owner' constraint.

8415b61

README.md: Change absolute badge URLs to 'mottosso/Qt.py'.

009591d

Qt.py: Update version to '1.3.9'.

65d87b3

martin-chatterjee mentioned this pull request Feb 1, 2024

added PySide6 shiboken6 support #376

Merged

mottosso changed the base branch from master to temp February 2, 2024 07:38

mottosso merged commit 2ca517b into mottosso:temp Feb 2, 2024
1 check passed

mottosso mentioned this pull request Feb 2, 2024

Merge #391 #392

Merged

mottosso added a commit that referenced this pull request Feb 2, 2024

Merge pull request #392 from mottosso/temp

9748c22

Merge #391

martin-chatterjee deleted the convert-to-github-actions branch March 4, 2024 09:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Convert CI/CD to GitHub actions. #391

Convert CI/CD to GitHub actions. #391

martin-chatterjee commented Feb 1, 2024 •

edited

Loading

CLAassistant commented Feb 1, 2024 •

edited

Loading

mottosso commented Feb 2, 2024

mottosso commented Feb 2, 2024

mottosso commented Feb 2, 2024

martin-chatterjee commented Feb 2, 2024

Convert CI/CD to GitHub actions. #391

Convert CI/CD to GitHub actions. #391

Conversation

martin-chatterjee commented Feb 1, 2024 • edited Loading

Objective

Motivation

Details

1. Run the test suite

2. Release to PyPi

Test instructions

Preparation

1. Check out mottosso/Qt.py:

2. Add martin-chatterjee/Qt.py as a remote named "review", and fetch:

Test "Run test suite".

1. Check out PR branch, push to origin:

→ Visit https://github.com/mottosso/Qt.py/actions, to see the resulting workflow run.

Test "Release to PyPi".

ℹ️ This PR is set up to release to test.pypi.org.

1. Make sure that mottosso/Qt.py is registered as a "Trusted Publisher".

2. Change project version to a unique version.

3. Commit and push.

4. Create a test release tag, and push.

→ Visit https://github.com/mottosso/Qt.py/actions, to see the resulting workflow run.

→ Visit https://test.pypi.org/project/Qt.py/, to see the released version.

CLAassistant commented Feb 1, 2024 • edited Loading

mottosso commented Feb 2, 2024

mottosso commented Feb 2, 2024

mottosso commented Feb 2, 2024

martin-chatterjee commented Feb 2, 2024

martin-chatterjee commented Feb 1, 2024 •

edited

Loading

1. Check out `mottosso/Qt.py`:

2. Add `martin-chatterjee/Qt.py` as a remote named `"review"`, and fetch:

1. Check out PR branch, push to `origin`:

ℹ️ This PR is set up to release to `test.pypi.org`.

1. Make sure that `mottosso/Qt.py` is registered as a "Trusted Publisher".

CLAassistant commented Feb 1, 2024 •

edited

Loading