Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CICD] Trigger workflow on label and dispatch #207

Merged
merged 39 commits into from
Jul 12, 2021
Merged

[CICD] Trigger workflow on label and dispatch #207

merged 39 commits into from
Jul 12, 2021

Conversation

sea212
Copy link
Member

@sea212 sea212 commented Jun 28, 2021 •
edited

closes #206

Changes

  • Only one benchmark workflow can run at a time (other workflows are queued)
  • A benchmark workflow can be triggered by dispatch now
  • A benchmark workflow can be triggered by adding the "s:benchmark-required" label to a pull request
  • When the benchmark workflow is triggered by label, the bot will:
  • Improvements in error handling
  • "Update weights" commit author changed from "zeitgeist-benchmark-bot" to the creator of the PR and committing user changed from "GitHub Actions" to "Zeitgeist Benchmark Bot"

@sea212 sea212 self-assigned this Jun 28, 2021
@sea212 sea212 added the s:benchmark-required The bot should benchmark the codebase label Jun 28, 2021
@github-actions github-actions bot added s:benchmark-done The benchmark was completed successfully (log available as comment in PR) and removed s:benchmark-required The bot should benchmark the codebase labels Jun 28, 2021
@sea212 sea212 added s:in-progress The pull requests is currently being worked on s:benchmark-required The bot should benchmark the codebase and removed s:benchmark-done The benchmark was completed successfully (log available as comment in PR) labels Jun 28, 2021
@github-actions github-actions bot added s:benchmark-done The benchmark was completed successfully (log available as comment in PR) s:benchmark-aborted The automated benchmark failed (see bot comment in PR) and removed s:benchmark-required The bot should benchmark the codebase labels Jun 28, 2021
@sea212 sea212 added s:benchmark-required The bot should benchmark the codebase and removed s:benchmark-aborted The automated benchmark failed (see bot comment in PR) s:benchmark-done The benchmark was completed successfully (log available as comment in PR) s:benchmark-required The bot should benchmark the codebase labels Jun 28, 2021
@github-actions github-actions bot added s:benchmark-aborted The automated benchmark failed (see bot comment in PR) and removed s:benchmark-required The bot should benchmark the codebase labels Jun 28, 2021
@sea212 sea212 added s:benchmark-required The bot should benchmark the codebase and removed s:benchmark-aborted The automated benchmark failed (see bot comment in PR) labels Jun 28, 2021
@github-actions github-actions bot added s:benchmark-done The benchmark was completed successfully (log available as comment in PR) and removed s:benchmark-required The bot should benchmark the codebase labels Jun 29, 2021
@github-actions
Copy link

Benchmarks completed successfully.

ResultsTBD

@sea212 sea212 added s:benchmark-required The bot should benchmark the codebase and removed s:benchmark-done The benchmark was completed successfully (log available as comment in PR) labels Jun 29, 2021
@github-actions github-actions bot added s:benchmark-aborted The automated benchmark failed (see bot comment in PR) and removed s:benchmark-required The bot should benchmark the codebase labels Jun 29, 2021
@sea212 sea212 added s:benchmark-required The bot should benchmark the codebase and removed s:benchmark-aborted The automated benchmark failed (see bot comment in PR) labels Jun 29, 2021
@github-actions github-actions bot added s:benchmark-aborted The automated benchmark failed (see bot comment in PR) and removed s:benchmark-required The bot should benchmark the codebase labels Jun 29, 2021
@sea212 sea212 marked this pull request as ready for review June 30, 2021 15:50
@sea212 sea212 requested a review from lsaether June 30, 2021 15:51
@sea212 sea212 added s:review-needed The pull request requires reviews and removed s:in-progress The pull requests is currently being worked on labels Jun 30, 2021
@sea212 sea212 requested a review from c410-f3r June 30, 2021 15:51
c410-f3r
c410-f3r approved these changes Jun 30, 2021
View reviewed changes
Copy link
Contributor

@c410-f3r c410-f3r left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some minor concerns.

  • Needs re-base
  • Substrate is using labeler to deal with labels (https://github.com/paritytech/substrate/blob/master/.github/workflows/auto-label-issues.yml) and this PR is using add-remove-label with labeler being the one with most recent updates. Just a heads-up, I don't know if they have similar or complete different functionalities.
  • Not totally sure but it is probably true that currently only team members can modify or add labels, i.e., external users won't trigger benchmarks

@sea212 sea212 force-pushed the sea212-benchmark-when-label-set branch from 607b08a to 1ab6bd7 Compare June 30, 2021 18:29
@sea212
Copy link
Member Author

sea212 commented Jun 30, 2021 •
edited

@c410-f3r

  • rebase: Thanks for mentioning this, somehow a bad rebase happened (3a40ccd). Should be fine now.
  • labeler: I didn't see that. I'll keep the other labeler, since it works (and I don't want to test those steps again 😄)
  • permission: This is really important, only members should be able to set (at least those) labels. @lsaether Can you check the permissions pls?

@lsaether lsaether merged commit 42ce273 into main Jul 12, 2021
@lsaether
Copy link
Member

Yes, only members of the org can set labels on PRs.

@sea212 sea212 deleted the sea212-benchmark-when-label-set branch July 14, 2021 08:58
@sea212 sea212 added s:accepted This pull request is ready for merge and removed s:review-needed The pull request requires reviews labels Jul 15, 2021
c410-f3r pushed a commit to c410-f3r/zeitgeist that referenced this pull request Sep 14, 2021
@sea212 @c410-f3r
[CICD] Trigger workflow on label and dispatch (zeitgeistpm#207)
179b2a0 
* Trigger workflow on label and dispatch

* Distinguish between different outcomes

* Fix syntax error

* Fix invalid step condition

* Fix invalid symbol

* Add status comment

* Add error message (test)

* Move exit action at the end

* Fix invalid GH token

* Integrate label mechanism into actual benchmarks

Additionally enforces senquential execution of benchmark workflows

* Add proper benchmark result message

This also serves to test if the workflow reacts properly to new commit (cancel, label and report)

* Test new ref fetch on PR

* Removed comments

* Replace local chain with dev chain

* Update weights

* Include complete benchmark output

* Test multiline error output

* Update weights

* Remove invalid local keyword

* Remove invalid local keyword

* Test successful run output

* Fix output formatting error

* Update weights

* Update weights

* Test multiline result text

* Test multiline result text

* Test multiline result text

* Test multiline result text

* Test multiline result text

* Shorten test text

* Move HTML into output variable

* Add function to construct html message

* Comment out line

* Replace test output with real output

* Uncomment actual benchmark command

* Activate benchmark error detection

* Update weights

* Reset weights

* Minor changes in step names

Co-authored-by: sea212 <sea212@users.noreply.github.com>
@sea212 sea212 mentioned this pull request Nov 17, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@c410-f3r c410-f3r c410-f3r approved these changes

@lsaether lsaether lsaether approved these changes

Assignees

@sea212 sea212

Labels
s:accepted This pull request is ready for merge
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[CICD] Trigger benchmarks after a specific label was set
3 participants
@sea212 @lsaether @c410-f3r