-
Notifications
You must be signed in to change notification settings - Fork 469
ci: run benchmark configs in parallel #15364
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
|
1e08371 to
4d25fad
Compare
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 254 ± 7 ms. The average import time from base is: 259 ± 3 ms. The import time difference between this PR and base is: -5.3 ± 0.2 ms. Import time breakdownThe following import paths have shrunk:
|
e85a967 to
2ef221a
Compare
Performance SLOsComparing candidate APMLP-701/parallelize.benchmarks (1b46e93) with baseline main (09f2961) 📈 Performance Regressions (2 suites)📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 5.188µs (SLO: <10.000µs 📉 -48.1%) vs baseline: 📈 +20.9% Memory: ✅ 40.403MB (SLO: <41.000MB 🟡 -1.5%) vs baseline: +5.0% ✅ ospathbasename_noaspectTime: ✅ 1.079µs (SLO: <10.000µs 📉 -89.2%) vs baseline: -0.5% Memory: ✅ 40.364MB (SLO: <41.000MB 🟡 -1.6%) vs baseline: +5.0% ✅ ospathjoin_aspectTime: ✅ 6.238µs (SLO: <10.000µs 📉 -37.6%) vs baseline: +2.2% Memory: ✅ 40.305MB (SLO: <41.000MB 🟡 -1.7%) vs baseline: +4.8% ✅ ospathjoin_noaspectTime: ✅ 2.289µs (SLO: <10.000µs 📉 -77.1%) vs baseline: ~same Memory: ✅ 40.383MB (SLO: <41.000MB 🟡 -1.5%) vs baseline: +5.1% ✅ ospathnormcase_aspectTime: ✅ 3.551µs (SLO: <10.000µs 📉 -64.5%) vs baseline: +0.3% Memory: ✅ 40.324MB (SLO: <41.000MB 🟡 -1.6%) vs baseline: +4.9% ✅ ospathnormcase_noaspectTime: ✅ 0.563µs (SLO: <10.000µs 📉 -94.4%) vs baseline: -1.3% Memory: ✅ 40.364MB (SLO: <41.000MB 🟡 -1.6%) vs baseline: +4.9% ✅ ospathsplit_aspectTime: ✅ 4.882µs (SLO: <10.000µs 📉 -51.2%) vs baseline: -0.2% Memory: ✅ 40.344MB (SLO: <41.000MB 🟡 -1.6%) vs baseline: +4.8% ✅ ospathsplit_noaspectTime: ✅ 1.590µs (SLO: <10.000µs 📉 -84.1%) vs baseline: -0.1% Memory: ✅ 40.285MB (SLO: <41.000MB 🟡 -1.7%) vs baseline: +5.1% ✅ ospathsplitdrive_aspectTime: ✅ 3.704µs (SLO: <10.000µs 📉 -63.0%) vs baseline: +0.1% Memory: ✅ 40.423MB (SLO: <41.000MB 🟡 -1.4%) vs baseline: +5.1% ✅ ospathsplitdrive_noaspectTime: ✅ 0.694µs (SLO: <10.000µs 📉 -93.1%) vs baseline: -0.7% Memory: ✅ 40.324MB (SLO: <41.000MB 🟡 -1.6%) vs baseline: +4.8% ✅ ospathsplitext_aspectTime: ✅ 4.644µs (SLO: <10.000µs 📉 -53.6%) vs baseline: +2.0% Memory: ✅ 40.324MB (SLO: <41.000MB 🟡 -1.6%) vs baseline: +5.0% ✅ ospathsplitext_noaspectTime: ✅ 1.377µs (SLO: <10.000µs 📉 -86.2%) vs baseline: -0.1% Memory: ✅ 40.442MB (SLO: <41.000MB 🟡 -1.4%) vs baseline: +5.3% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.461µs (SLO: <20.000µs 📉 -82.7%) vs baseline: 📈 +15.3% Memory: ✅ 34.957MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +5.1% ✅ 1-count-metrics-100-timesTime: ✅ 199.347µs (SLO: <220.000µs -9.4%) vs baseline: -0.8% Memory: ✅ 34.878MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.3% ✅ 1-distribution-metric-1-timesTime: ✅ 3.317µs (SLO: <20.000µs 📉 -83.4%) vs baseline: -0.3% Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +4.3% ✅ 1-distribution-metrics-100-timesTime: ✅ 215.436µs (SLO: <230.000µs -6.3%) vs baseline: ~same Memory: ✅ 34.977MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +4.0% ✅ 1-gauge-metric-1-timesTime: ✅ 2.253µs (SLO: <20.000µs 📉 -88.7%) vs baseline: -1.3% Memory: ✅ 34.977MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +5.0% ✅ 1-gauge-metrics-100-timesTime: ✅ 137.147µs (SLO: <150.000µs -8.6%) vs baseline: ~same Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +4.1% ✅ 1-rate-metric-1-timesTime: ✅ 3.108µs (SLO: <20.000µs 📉 -84.5%) vs baseline: -0.7% Memory: ✅ 34.977MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +4.9% ✅ 1-rate-metrics-100-timesTime: ✅ 214.520µs (SLO: <250.000µs 📉 -14.2%) vs baseline: ~same Memory: ✅ 34.977MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +4.9% ✅ 100-count-metrics-100-timesTime: ✅ 20.413ms (SLO: <22.000ms -7.2%) vs baseline: +0.5% Memory: ✅ 35.252MB (SLO: <35.500MB 🟡 -0.7%) vs baseline: +4.8% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.249ms (SLO: <2.300ms -2.2%) vs baseline: -0.1% Memory: ✅ 35.271MB (SLO: <35.500MB 🟡 -0.6%) vs baseline: +5.3% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.409ms (SLO: <1.550ms -9.1%) vs baseline: -0.1% Memory: ✅ 35.271MB (SLO: <35.500MB 🟡 -0.6%) vs baseline: +4.7% ✅ 100-rate-metrics-100-timesTime: ✅ 2.226ms (SLO: <2.550ms 📉 -12.7%) vs baseline: +0.6% Memory: ✅ 35.330MB (SLO: <35.500MB 🟡 -0.5%) vs baseline: +5.0% ✅ flush-1-metricTime: ✅ 4.395µs (SLO: <20.000µs 📉 -78.0%) vs baseline: -0.7% Memory: ✅ 35.252MB (SLO: <35.500MB 🟡 -0.7%) vs baseline: +4.9% ✅ flush-100-metricsTime: ✅ 173.304µs (SLO: <250.000µs 📉 -30.7%) vs baseline: +0.2% Memory: ✅ 35.311MB (SLO: <35.500MB 🟡 -0.5%) vs baseline: +4.8% ✅ flush-1000-metricsTime: ✅ 2.182ms (SLO: <2.500ms 📉 -12.7%) vs baseline: -0.3% Memory: ✅ 36.156MB (SLO: <36.500MB 🟡 -0.9%) vs baseline: +5.0% 🟡 Near SLO Breach (18 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
14d4557 to
a49a085
Compare
a49a085 to
a1aafbe
Compare
3415391 to
6c93c0c
Compare
6c93c0c to
256965e
Compare
emmettbutler
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
|
This very elegant solution, really love it! |
This change parallelizes the microbenchmark jobs to make more efficient use of CI time and compute resources