PR: Add workflow triggers for release candidates and fix tag normalization #20396

mrclary · 2023-01-18T19:39:04Z

Description of Changes

Added workflow trigger for pull requests labeled build-installer-artifacts
Added workflow trigger for manually running installers
Fix issue where installer version normalized tag version

Workflow Triggers

Pull requests will continue to trigger all three installer workflows with limited artifacts (Full versions only) and without code-signing or uploading artifacts.

Labeling a pull request with build-installer-artifacts or pushing a tag matching the pattern v*pre* will also trigger all three installer workflows, but they will run as if it were a release (Lite and Full versions, code-signing, etc.). However, workflow artifacts will not be uploaded to a release. These two trigger methods will not have any affect on a branch in which the latest commit message contains [ci skip] (or equivalent variations).

All three installer workflows can be triggered manually from the GitHub web interface or CLI and will execute workflows in the same manner as the aforementioned labeling and tagging mechanisms. This method is not affected by the presence of [ci skip] (or variants). However, this workflow trigger mechanism will not be available until this update is propagated to the master branch, i.e. after merging to 5.x and subsequent merge to master.

All installer workflows will continue to behave as before upon creating a release.

Normalized version

setuptools_scm by default normalizes our vX.X.XpreX release candidate tags to vX.X.XrcX, resulting in an installer named EXPERIMENTAL-Spyder-5.4.2rc1-macOS-x86_64 instead of EXPERIMENTAL-Spyder-5.4.2pre1-macOS-x86_64.

To work around this, we tell setuptools_scm not to normalize the version. This results in a version vX.X.XpreX, so we further remove the leading v, finally arriving at the desired X.X.XpreX.

Note that the versions for external_deps built locally for the installer must remain normalized; otherwise conda will fail to solve the environment.

mrclary · 2023-01-18T20:05:12Z

@ccordoba12 I've also checked a release candidate tag on my fork on this branch as well and all looks good.

ccordoba12 · 2023-01-18T20:29:00Z

Thanks for your work on this @mrclary! We decided with @dalthviz to leave this testing for 5.4.3 to discuss a bit more about some details that are not clear enough for us. Since according to your testing the installers are built just fine for 5.4.2, he'll proceed with the release now.

mrclary · 2023-01-29T04:53:26Z

@ccordoba12, this should be ready for review now.

ccordoba12

Thanks @mrclary for your work on this! I left some questions for you to better understand your work here.

ccordoba12 · 2023-01-30T16:15:48Z

.github/workflows/installer-win.yml

    strategy:
      matrix:
        build_type: ['Lite', 'Full']
    env:
-      UNWANTED_PACKAGES: ${{github.event_name == 'pull_request' && 'pip spyder-kernels python-slugify Rtree QDarkStyle PyNaCl psutil' || 'pip python-slugify Rtree PyNaCl psutil' }}
+      UNWANTED_PACKAGES: ${{github.event_name == 'pull_request' && github.event.action != 'labeled' && 'pip spyder-kernels python-slugify Rtree QDarkStyle PyNaCl psutil' || 'pip python-slugify Rtree PyNaCl psutil' }}


This addition

github.event.action != 'labeled'

means that the PRs can't have any label after this change for it to work?

PRs can have any label, but we want the workflow to behave differently for different pull request actions.

Previously, this workflow triggered on the default pull_request types: opened, reopened, or sychronize. This trigger caused the workflow to build developer versions (not "release" versions). In the case of this particular line, this meant that spyder-kernels was included in the UNWANTED_PACKAGES variable. For "release" versions (i.e. non-PR triggers) spyder-kernels would not be included in this variable.

With this PR, we want the behavior of the workflow to remain the same regarding release-like and non-release-like versions, but the trigger now includes the labeled pull_request type. This changes the logic from github.event_name == 'pull_request' to github.event_name == 'pull_request' && github.event.action != 'labeled'.

But what about PRs that get labeled an arbitrary label? Well, we don't have to worry about that logic on this line because the build job will not run in this case due to line 47. github.event.action != 'labeled' on line 47 covers all the previous trigger cases; github.event.label.name == 'build-installer-artifacts' on line 47 then permits only the additional labeled trigger when the label is correct, i.e. arbitrary labels given to a PR will still trigger this workflow but the jobs will be skipped.

Unfortunately, unlike the push.tag trigger, there is no way to filter the workflow trigger on a label pattern, so it must be done at the job level.

@ccordoba12, did this sufficiently answer your question?

Yeah, it does. Sorry for the delay

.github/workflows/installers-conda.yml

RELEASE.md

ccordoba12 · 2023-01-30T16:32:12Z

However, this workflow trigger mechanism will not be available until this update is propagated to the master branch, i.e. after merging to 5.x and subsequent merge to master.

You mean after this PR is merged to 5.x and master, right?

Also, it seems you did a lot of testing here, so could you squash some of your commits if that was the case?

ccordoba12 · 2023-01-30T16:33:06Z

@dalthviz, I asked for your review so that it's clear for you what you need to do in our next release and you can ask for clarifications/improvements to this workflow too.

dalthviz

Just to be sure I'm understanding correctly, the way the triggering of the workflows will work when doing a release is:

flowchart TD
    A[Merge update dependencies PR] --> B[Commit directly to Spyder upstream 5.x to set the preX -pre0- version or update the X -0, 1, etc-]
    B--> C[Trigger the installers workflows manually from the Actions page]
    C--> D[Test manually the installation of the different builded installers]
    D--> E{Errors where found?}
    E-->|Yes| F[Submit and merge PR with fixes]
    F--> B
    E---->|No| G[Continue with the steps to do the release]

Right? If I'm understanding this correctly then sounds good to me 👍

Maybe having some basic set of things/tasks to test manually with the generated installers could be nice to have but I think we can open an issue to add them later to the RELEASE.md

mrclary · 2023-01-31T23:22:47Z

However, this workflow trigger mechanism will not be available until this update is propagated to the master branch, i.e. after merging to 5.x and subsequent merge to master.

You mean after this PR is merged to 5.x and master, right?

Correct.

Also, it seems you did a lot of testing here, so could you squash some of your commits if that was the case?

No problem.

dalthviz · 2023-01-31T23:23:57Z

Also, thank you for working on this @mrclary I think this will greatly improve the release process reliability in terms of generated assets/installers 👍

mrclary · 2023-01-31T23:36:48Z

Just to be sure I'm understanding correctly, the way the triggering of the workflows will work when doing a release is:

Right? If I'm understanding this correctly then sounds good to me 👍

That is correct.

Maybe having some basic set of things/tasks to test manually with the generated installers could be nice to have but I think we can open an issue to add them later to the RELEASE.md

This is a good idea. Presently, the original macOS installer only tests that Spyder.app launches without error. I think we could do the same for the original Windows installer as well as the new conda-based installers.

However, when we move to 6.0(alpha?), we may be able to adapt our existing unit tests to run on a release candidate installation on CI. We can discuss the cost/benefits when we get there...

…move leading "v". All other built repo versions should remain normalized, otherwise conda can't resolve environment for installer

Add helper environment variables for reference Update workflow logic

…Windows installers

…ner and combines two housekeeping jobs that are very fast.

build-installers may have run in the event that build-matrix was skipped (arbitrary PR label), resulting in a workflow error.

ccordoba12

Thanks @mrclary!

mrclary self-assigned this Jan 18, 2023

mrclary added this to the v5.4.2 milestone Jan 18, 2023

mrclary mentioned this pull request Jan 18, 2023

PR: Fix patch for conda-based installers #20376

Merged

mrclary requested a review from ccordoba12 January 18, 2023 20:03

ccordoba12 modified the milestones: v5.4.2, v5.4.3 Jan 18, 2023

mrclary force-pushed the fix-cbi-tag branch 4 times, most recently from 56c6164 to 725d6be Compare January 28, 2023 23:25

mrclary changed the title ~~PR: Do not normalize Spyder release candidate version in order to preserve "pre"~~ PR: Add workflow triggers for release candidates and fix tag normalization Jan 29, 2023

ccordoba12 reviewed Jan 30, 2023

View reviewed changes

ccordoba12 requested a review from dalthviz January 30, 2023 16:32

mrclary force-pushed the fix-cbi-tag branch from 90a8ae9 to d745c6a Compare January 30, 2023 18:09

mrclary marked this pull request as draft January 30, 2023 18:35

mrclary force-pushed the fix-cbi-tag branch from 0a86922 to d745c6a Compare January 30, 2023 18:56

mrclary marked this pull request as ready for review January 30, 2023 19:01

dalthviz reviewed Jan 31, 2023

View reviewed changes

mrclary force-pushed the fix-cbi-tag branch from a029805 to 83452c4 Compare January 31, 2023 23:52

dalthviz approved these changes Feb 1, 2023

View reviewed changes

mrclary added 2 commits February 1, 2023 09:36

Do not normalize Spyder version in order to preserve "pre", but do re…

b76b3bc

…move leading "v". All other built repo versions should remain normalized, otherwise conda can't resolve environment for installer

Add trigger on pull_request label and manual

cb5f78d

Add helper environment variables for reference Update workflow logic

mrclary added 6 commits February 1, 2023 09:36

Add labeled and workflow_dispatch to workflow triggers for macOS and …

6117416

…Windows installers

Update release instructions

87584d2

No need to checkout code

b0d85f1

Remove label as step in build-matrix. Eliminates need for another run…

7cf9e82

…ner and combines two housekeeping jobs that are very fast.

Require build-matrix to succeed.

cb986b2

build-installers may have run in the event that build-matrix was skipped (arbitrary PR label), resulting in a workflow error.

Add link to instructions for manually running a workflow

a6fe925

mrclary force-pushed the fix-cbi-tag branch from 83452c4 to a6fe925 Compare February 1, 2023 17:37

ccordoba12 approved these changes Feb 1, 2023

View reviewed changes

ccordoba12 merged commit bcb5886 into spyder-ide:5.x Feb 1, 2023

ccordoba12 added a commit that referenced this pull request Feb 1, 2023

Merge from 5.x: PR #20396

3e24ec4

mrclary deleted the fix-cbi-tag branch February 2, 2023 01:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PR: Add workflow triggers for release candidates and fix tag normalization #20396

PR: Add workflow triggers for release candidates and fix tag normalization #20396

mrclary commented Jan 18, 2023 •

edited

Loading

mrclary commented Jan 18, 2023 •

edited

Loading

ccordoba12 commented Jan 18, 2023 •

edited

Loading

mrclary commented Jan 29, 2023

ccordoba12 left a comment

ccordoba12 Jan 30, 2023 •

edited

Loading

mrclary Jan 30, 2023

mrclary Feb 1, 2023

ccordoba12 Feb 1, 2023

ccordoba12 commented Jan 30, 2023

ccordoba12 commented Jan 30, 2023 •

edited

Loading

dalthviz left a comment

mrclary commented Jan 31, 2023

dalthviz commented Jan 31, 2023

mrclary commented Jan 31, 2023

ccordoba12 left a comment

PR: Add workflow triggers for release candidates and fix tag normalization #20396

PR: Add workflow triggers for release candidates and fix tag normalization #20396

Conversation

mrclary commented Jan 18, 2023 • edited Loading

Description of Changes

Workflow Triggers

Normalized version

mrclary commented Jan 18, 2023 • edited Loading

ccordoba12 commented Jan 18, 2023 • edited Loading

mrclary commented Jan 29, 2023

ccordoba12 left a comment

Choose a reason for hiding this comment

ccordoba12 Jan 30, 2023 • edited Loading

Choose a reason for hiding this comment

mrclary Jan 30, 2023

Choose a reason for hiding this comment

mrclary Feb 1, 2023

Choose a reason for hiding this comment

ccordoba12 Feb 1, 2023

Choose a reason for hiding this comment

ccordoba12 commented Jan 30, 2023

ccordoba12 commented Jan 30, 2023 • edited Loading

dalthviz left a comment

Choose a reason for hiding this comment

mrclary commented Jan 31, 2023

dalthviz commented Jan 31, 2023

mrclary commented Jan 31, 2023

ccordoba12 left a comment

Choose a reason for hiding this comment

mrclary commented Jan 18, 2023 •

edited

Loading

mrclary commented Jan 18, 2023 •

edited

Loading

ccordoba12 commented Jan 18, 2023 •

edited

Loading

ccordoba12 Jan 30, 2023 •

edited

Loading

ccordoba12 commented Jan 30, 2023 •

edited

Loading