Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Add benchmarks to pipeline #1074

Merged
merged 35 commits into from
Apr 12, 2022
Merged

Add benchmarks to pipeline #1074

merged 35 commits into from
Apr 12, 2022

Conversation

alvicsam
Copy link
Contributor

@alvicsam alvicsam commented Mar 9, 2022

PR adds manual benchmarks job. It's divided to 2 jobs: benchmarks-build which builds binary on a faster ci runners and passes to benchmarks job which execute benchmarks.
The benchmarks job runs script benchmarks-ci.sh for statemine, statemint and westmint and creates PR on github with updated weights (example: #1138)

Part of https://github.com/paritytech/ci_cd/issues/260
Closes #631

@alvicsam alvicsam requested a review from a team as a code owner March 9, 2022 10:06
@alvicsam alvicsam added A3-inprogress B0-silent Changes should not be mentioned in any release notes labels Mar 23, 2022
@alvicsam alvicsam requested a review from NachoPal April 4, 2022 13:05
@alvicsam alvicsam changed the title [Do not merge] add benchmarks to pipeline Add benchmarks to pipeline Apr 4, 2022
.gitlab-ci.yml Outdated Show resolved Hide resolved
.gitlab-ci.yml Outdated Show resolved Hide resolved
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
@TriplEight TriplEight self-requested a review April 4, 2022 13:59
@alvicsam
Copy link
Contributor Author

alvicsam commented Apr 7, 2022

There is an additional condition that should be implemented (after discussing it with @NachoPal in element):

Weights should be pushed to the same branch where the binary for benchmarks was compiled.

Copy link
Contributor

@NachoPal NachoPal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @alvicsam . I realised that we'll need an input to select the branch we want to use for building the binary, running the benchmarks and opening a PR with the new weights.

The flow would be as follows:

  1. Select branch release-parachains-vX.Y.Z -> Click Run benchmarks button
  2. The binary is built, benchmarks are run.
  3. When the new weights are ready, a PR is open against release-parachains-vX.Y.Z
  4. We review the PR and merge it to release-parachains-vX.Y.Z

@alvicsam alvicsam requested a review from NachoPal April 12, 2022 07:46
@alvicsam
Copy link
Contributor Author

@NachoPal I changed benchmarks jobs. Now they will appear only on pipelines that are related to release-parachains-v* branches and behave as you described (run benchmarks, create PR against release-parachains-v*)

Copy link
Contributor

@NachoPal NachoPal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@NachoPal
Copy link
Contributor

What Benchmarking machine is used to run the benchmarks?

@alvicsam
Copy link
Contributor Author

bm4, bm5, bm6

@alvicsam alvicsam merged commit 058469b into master Apr 12, 2022
@alvicsam alvicsam deleted the as-benches branch April 12, 2022 11:53
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
B0-silent Changes should not be mentioned in any release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI/CD] Add benchmarking CI job
3 participants