Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve build and deployment concurrency handling #284

Merged
merged 11 commits into from
Oct 27, 2022

Conversation

Stephan202
Copy link
Member

@Stephan202 Stephan202 commented Oct 8, 2022

Suggested commit message:

Improve build and deployment concurrency handling (#284)

Builds for the same branch are now serialized. Among other things this prevents
concurrent deployments.

While there, reduce the permission assigned to each of the two jobs.

@Stephan202 Stephan202 temporarily deployed to github-pages October 8, 2022 09:54 Inactive
@Stephan202 Stephan202 temporarily deployed to github-pages October 8, 2022 09:56 Inactive
@Stephan202 Stephan202 temporarily deployed to github-pages October 8, 2022 09:58 Inactive
@Stephan202 Stephan202 added this to the 0.4.0 milestone Oct 8, 2022
@Stephan202
Copy link
Member Author

The commit history shows my tests to reduce the scope of the permissions blocks. The initial attempts failed because I hadn't configured this branch as being allowed to deploy GitHub Pages.

@Stephan202 Stephan202 mentioned this pull request Oct 8, 2022
1 task
@Stephan202 Stephan202 marked this pull request as ready for review October 8, 2022 14:19
Copy link
Member

@japborst japborst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, an oversight. Thanks for looking into this.
Left some comments.

Comment on lines 33 to 39
concurrency:
group: pages
cancel-in-progress: true
Copy link
Member

@japborst japborst Oct 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you also consider removing this setting completely? What's the downside of deploying twice?

If I understand correctly, what's happening is:

  • PR-1 merged => master is built & deployed
  • PR-2 is merged => master is built & deployed

Where PR-2 is somehow faster and thus first PR-2 is deployed, after which PR-1 is deployed and thus leading to an outdated website. Additionally, the current setup is cancelling the build across multiple branches/PRs, which then requires one to manually re-run to fix the build.

Now, we're still grouping globally (group: pages), and this works because the if is on a single branch, but could still lead to issues across multiple branches I think.

Looking at the docs, an alternative would be moving this up again, but grouping differently. Arguably, if we merge the two PRs (from the above example), there's no need in building twice. Additionally, if you push 2 commits on a branch, we should be able to cancel them as - again - there's no use in building twice.

Suggested change
concurrency:
group: pages
cancel-in-progress: true
concurrency:
group: ${{ github.ref }}
cancel-in-progress: true

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, if you push 2 commits on a branch, we should be able to cancel them as - again - there's no use in building twice.

This part I agree with (and also crossed my mind).

Arguably, if we merge the two PRs (from the above example), there's no need in building twice.

Strictly speaking true, but IMHO it's a nice feature that each merged commit has a "complete" build status, and that if a merge breaks the build, that the preceding commit still has a "build passed" check mark. (That said, generally @rickie and I don't merge a second PR until the first passed, so in practice there's limited difference.) (And perhaps we should require that PRs are up-to-date with the base branch before merging, but that's yet another discussion.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with this change, as this is undesirable:

Additionally, the current setup is cancelling the build across multiple branches/PRs, which then requires one to manually re-run to fix the build.

Based on the linked docs, I'd say that this would be even safer (and be prepared on possible future usages of concurrency as well): group: ${{ github.workflow }}-${{ github.ref }}.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say that dropping cancel-in-progress would be fine as well.

Did you also consider removing this setting completely?

However, I'd say that it is nice if you do multiple commits that pending builds will get cancelled:

When a concurrent job or workflow is queued, if another job or workflow using the same concurrency group in the repository is in progress, the queued job or workflow will be pending. Any previously pending job or workflow in the concurrency group will be canceled.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this was an old comment that I hadn't flushed yet, hence the funny ordering. Anyway, the question is whether we're okay with the PR as-is, or want to add cancel-in-progress: true again.

Copy link
Member

@rickie rickie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested some possible tweaks / asked some questions :).

@japborst japborst added the chore A task not related to code (build, formatting, process, ...) label Oct 9, 2022
@Stephan202 Stephan202 modified the milestones: 0.4.0, 0.5.0 Oct 13, 2022
@Stephan202 Stephan202 force-pushed the sschroevers/fix-gha-website-deployment-concurrency branch from d557ce8 to 5484f24 Compare October 22, 2022 15:09
@Stephan202
Copy link
Member Author

Rebased and resolved conflicts.

Copy link
Member Author

@Stephan202 Stephan202 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the PR; sorry for the delay. @japborst @rickie WDYT?

Comment on lines 33 to 39
concurrency:
group: pages
cancel-in-progress: true
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, if you push 2 commits on a branch, we should be able to cancel them as - again - there's no use in building twice.

This part I agree with (and also crossed my mind).

Arguably, if we merge the two PRs (from the above example), there's no need in building twice.

Strictly speaking true, but IMHO it's a nice feature that each merged commit has a "complete" build status, and that if a merge breaks the build, that the preceding commit still has a "build passed" check mark. (That said, generally @rickie and I don't merge a second PR until the first passed, so in practice there's limited difference.) (And perhaps we should require that PRs are up-to-date with the base branch before merging, but that's yet another discussion.)

concurrency:
group: pages
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.ref }}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively we could do this, but as discussed it'd be nice not to cancel non-PR builds. I tried to specify cancel-in-progress: ${{ github.head_ref != null }}, but that doesn't seem to work.

Copy link
Member

@rickie rickie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice discussions! This looks good to me!

it's a nice feature that each merged commit has a "complete" build status

Completely agree.

@rickie
Copy link
Member

rickie commented Oct 26, 2022

Rebased. Anything else from your side @japborst 😄?

@rickie rickie force-pushed the sschroevers/fix-gha-website-deployment-concurrency branch from af8fa31 to c2d4fcf Compare October 26, 2022 15:12
Copy link
Member

@japborst japborst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went through the comments. Makes sense 👍 Thanks!

@rickie rickie changed the title Improve website deployment concurrency handling Improve build and deployment concurrency handling Oct 27, 2022
@rickie rickie merged commit 04d886c into master Oct 27, 2022
@rickie rickie deleted the sschroevers/fix-gha-website-deployment-concurrency branch October 27, 2022 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore A task not related to code (build, formatting, process, ...)
Development

Successfully merging this pull request may close these issues.

None yet

3 participants