Skip to content

ci: parallelize macrobenchmark experiments #6564

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Oct 3, 2025
Merged

ci: parallelize macrobenchmark experiments #6564

Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
58db7d1
test parallelizing macrobenchmarks
igoragoli Sep 29, 2025
f55f1cf
extend from macrobenchmarks
igoragoli Sep 29, 2025
7dc8339
change experiment to high-load on high-load-with-debug-and-without-ht…
igoragoli Sep 29, 2025
20cf818
simplify macrobenchmarks, parallelize experiments under the same jobs
igoragoli Sep 30, 2025
04c850d
test changing CI_JOB_NAME to see if results names are more readable
igoragoli Sep 30, 2025
6a7659a
test a cleaner approach for updating the CI_JOB_NAME
igoragoli Oct 1, 2025
b60e6da
test another cleaner approach for updating the CI_JOB_NAME
igoragoli Oct 1, 2025
1f18fc9
set CI_JOB_NAME directly on each benchmark group
igoragoli Oct 1, 2025
7affefe
add a TODO for reverting the benchmarking-platform branch
igoragoli Oct 1, 2025
a461519
update the TODO for reverting the benchmarking-platform branch
igoragoli Oct 2, 2025
166ed48
add only-tracing-wiht-runtime-metrics-enabled to check-slo-breaches "…
igoragoli Oct 2, 2025
c012e6e
Merge branch 'master' into augusto/test-rochdev-parallel-experiments
rochdev Oct 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 22 additions & 4 deletions .gitlab/macrobenchmarks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ include:
timeout: 1h
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/benchmarking-platform:js-hapi
script:
- git clone --branch js/hapi https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.ddbuild.io/DataDog/benchmarking-platform platform && cd platform
- bp-runner bp-runner.yml --debug -t
# TODO: Revert to js/hapi after https://github.com/DataDog/benchmarking-platform/pull/199 is merged
- git clone --branch rochdev/parallel-experiments https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.ddbuild.io/DataDog/benchmarking-platform platform && cd platform
- bp-runner bp-runner.$EXPERIMENT.yml --debug -t
artifacts:
name: "artifacts"
when: always
Expand Down Expand Up @@ -64,17 +65,34 @@ baseline:
extends: .macrobenchmarks
variables:
DD_BENCHMARKS_CONFIGURATION: baseline
# Must be set to ensure consistent benchmark result naming and tagging
CI_JOB_NAME: baseline
parallel:
matrix:
- EXPERIMENT: normal-operation
- EXPERIMENT: high-load

only-tracing:
extends: .macrobenchmarks
variables:
DD_BENCHMARKS_CONFIGURATION: only-tracing
# Must be set to ensure consistent benchmark result naming and tagging
CI_JOB_NAME: only-tracing
parallel:
matrix:
- EXPERIMENT: normal-operation
- EXPERIMENT: high-load

only-tracing-with-runtime-metrics-enabled:
extends: .macrobenchmarks
variables:
DD_BENCHMARKS_CONFIGURATION: only-tracing
DD_RUNTIME_METRICS_ENABLED: "true"
DD_BENCHMARKS_CONFIGURATION: only-tracing-with-runtime-metrics-enabled
# Must be set to ensure consistent benchmark result naming and tagging
CI_JOB_NAME: only-tracing-with-runtime-metrics-enabled
parallel:
matrix:
- EXPERIMENT: normal-operation
- EXPERIMENT: high-load

check-slo-breaches:
extends: .check-slo-breaches
Expand Down