ci: reduce flakiness from httplib https tests #15318
Conversation
brettlangdon
added
the
changelog/no-changelog
|
|
brettlangdon
force-pushed
the
brettlangdon/httplib.https
branch
from
58fe3e2 to
94d6bf1
Compare
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 266 ± 5 ms. The average import time from base is: 267 ± 5 ms. The import time difference between this PR and base is: -0.4 ± 0.2 ms. The difference is not statistically significant (z = -1.47). Import time breakdownThe following import paths have grown:
|
Performance SLOsComparing candidate brettlangdon/httplib.https (94d6bf1) with baseline main (dbe8eb7) 📈 Performance Regressions (2 suites)📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 4.290µs (SLO: <10.000µs 📉 -57.1%) vs baseline: +1.0% Memory: ✅ 39.872MB (SLO: <41.000MB -2.8%) vs baseline: +5.0% ✅ ospathbasename_noaspectTime: ✅ 1.084µs (SLO: <10.000µs 📉 -89.2%) vs baseline: +0.2% Memory: ✅ 39.833MB (SLO: <41.000MB -2.8%) vs baseline: +4.5% ✅ ospathjoin_aspectTime: ✅ 7.035µs (SLO: <10.000µs 📉 -29.7%) vs baseline: 📈 +13.7% Memory: ✅ 39.793MB (SLO: <41.000MB -2.9%) vs baseline: +4.6% ✅ ospathjoin_noaspectTime: ✅ 2.270µs (SLO: <10.000µs 📉 -77.3%) vs baseline: -0.6% Memory: ✅ 39.911MB (SLO: <41.000MB -2.7%) vs baseline: +4.9% ✅ ospathnormcase_aspectTime: ✅ 3.533µs (SLO: <10.000µs 📉 -64.7%) vs baseline: +0.2% Memory: ✅ 39.833MB (SLO: <41.000MB -2.8%) vs baseline: +5.0% ✅ ospathnormcase_noaspectTime: ✅ 0.566µs (SLO: <10.000µs 📉 -94.3%) vs baseline: -0.7% Memory: ✅ 39.892MB (SLO: <41.000MB -2.7%) vs baseline: +5.0% ✅ ospathsplit_aspectTime: ✅ 4.802µs (SLO: <10.000µs 📉 -52.0%) vs baseline: -0.9% Memory: ✅ 39.911MB (SLO: <41.000MB -2.7%) vs baseline: +5.0% ✅ ospathsplit_noaspectTime: ✅ 1.599µs (SLO: <10.000µs 📉 -84.0%) vs baseline: +0.5% Memory: ✅ 39.931MB (SLO: <41.000MB -2.6%) vs baseline: +5.1% ✅ ospathsplitdrive_aspectTime: ✅ 3.719µs (SLO: <10.000µs 📉 -62.8%) vs baseline: +0.1% Memory: ✅ 39.872MB (SLO: <41.000MB -2.8%) vs baseline: +5.0% ✅ ospathsplitdrive_noaspectTime: ✅ 0.710µs (SLO: <10.000µs 📉 -92.9%) vs baseline: +2.0% Memory: ✅ 39.911MB (SLO: <41.000MB -2.7%) vs baseline: +4.7% ✅ ospathsplitext_aspectTime: ✅ 4.596µs (SLO: <10.000µs 📉 -54.0%) vs baseline: +0.3% Memory: ✅ 39.833MB (SLO: <41.000MB -2.8%) vs baseline: +4.8% ✅ ospathsplitext_noaspectTime: ✅ 1.386µs (SLO: <10.000µs 📉 -86.1%) vs baseline: +1.5% Memory: ✅ 39.774MB (SLO: <41.000MB -3.0%) vs baseline: +4.6% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.296µs (SLO: <20.000µs 📉 -83.5%) vs baseline: 📈 +10.3% Memory: ✅ 34.564MB (SLO: <35.500MB -2.6%) vs baseline: +4.7% ✅ 1-count-metrics-100-timesTime: ✅ 201.750µs (SLO: <220.000µs -8.3%) vs baseline: -0.8% Memory: ✅ 34.524MB (SLO: <35.500MB -2.7%) vs baseline: +5.0% ✅ 1-distribution-metric-1-timesTime: ✅ 3.723µs (SLO: <20.000µs 📉 -81.4%) vs baseline: 📈 +10.5% Memory: ✅ 34.603MB (SLO: <35.500MB -2.5%) vs baseline: +5.0% ✅ 1-distribution-metrics-100-timesTime: ✅ 220.184µs (SLO: <220.000µs ~same) vs baseline: +0.5% Memory: ✅ 34.505MB (SLO: <35.500MB -2.8%) vs baseline: +4.7% ✅ 1-gauge-metric-1-timesTime: ✅ 2.424µs (SLO: <20.000µs 📉 -87.9%) vs baseline: 📈 +11.2% Memory: ✅ 34.583MB (SLO: <35.500MB -2.6%) vs baseline: +5.1% ✅ 1-gauge-metrics-100-timesTime: ✅ 136.946µs (SLO: <150.000µs -8.7%) vs baseline: ~same Memory: ✅ 34.524MB (SLO: <35.500MB -2.7%) vs baseline: +4.9% ✅ 1-rate-metric-1-timesTime: ✅ 3.096µs (SLO: <20.000µs 📉 -84.5%) vs baseline: -0.5% Memory: ✅ 34.524MB (SLO: <35.500MB -2.7%) vs baseline: +5.0% ✅ 1-rate-metrics-100-timesTime: ✅ 213.819µs (SLO: <250.000µs 📉 -14.5%) vs baseline: -1.0% Memory: ✅ 34.603MB (SLO: <35.500MB -2.5%) vs baseline: +5.2% ✅ 100-count-metrics-100-timesTime: ✅ 20.167ms (SLO: <22.000ms -8.3%) vs baseline: +0.4% Memory: ✅ 34.505MB (SLO: <35.500MB -2.8%) vs baseline: +4.7% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.271ms (SLO: <2.300ms 🟡 -1.3%) vs baseline: -0.5% Memory: ✅ 34.564MB (SLO: <35.500MB -2.6%) vs baseline: +4.7% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.413ms (SLO: <1.550ms -8.8%) vs baseline: +0.5% Memory: ✅ 34.505MB (SLO: <35.500MB -2.8%) vs baseline: +4.9% ✅ 100-rate-metrics-100-timesTime: ✅ 2.214ms (SLO: <2.550ms 📉 -13.2%) vs baseline: +0.5% Memory: ✅ 34.564MB (SLO: <35.500MB -2.6%) vs baseline: +4.9% ✅ flush-1-metricTime: ✅ 4.921µs (SLO: <20.000µs 📉 -75.4%) vs baseline: +7.2% Memory: ✅ 34.505MB (SLO: <35.500MB -2.8%) vs baseline: +4.8% ✅ flush-100-metricsTime: ✅ 175.640µs (SLO: <250.000µs 📉 -29.7%) vs baseline: -0.1% Memory: ✅ 34.426MB (SLO: <35.500MB -3.0%) vs baseline: +4.5% ✅ flush-1000-metricsTime: ✅ 2.116ms (SLO: <2.500ms 📉 -15.4%) vs baseline: -0.4% Memory: ✅ 35.389MB (SLO: <36.500MB -3.0%) vs baseline: +5.1% 🟡 Near SLO Breach (1 suite)🟡 otelspan - 22/22✅ add-eventTime: ✅ 38.613ms (SLO: <47.150ms 📉 -18.1%) vs baseline: +0.1% Memory: ✅ 38.884MB (SLO: <47.000MB 📉 -17.3%) vs baseline: +4.7% ✅ add-metricsTime: ✅ 257.139ms (SLO: <344.800ms 📉 -25.4%) vs baseline: +0.5% Memory: ✅ 43.273MB (SLO: <47.500MB -8.9%) vs baseline: +4.8% ✅ add-tagsTime: ✅ 315.198ms (SLO: <321.000ms 🟡 -1.8%) vs baseline: -0.3% Memory: ✅ 43.238MB (SLO: <47.500MB -9.0%) vs baseline: +4.4% ✅ get-contextTime: ✅ 79.102ms (SLO: <92.350ms 📉 -14.3%) vs baseline: +0.3% Memory: ✅ 39.317MB (SLO: <46.500MB 📉 -15.4%) vs baseline: +4.8% ✅ is-recordingTime: ✅ 36.171ms (SLO: <44.500ms 📉 -18.7%) vs baseline: +0.3% Memory: ✅ 38.828MB (SLO: <47.500MB 📉 -18.3%) vs baseline: +4.7% ✅ record-exceptionTime: ✅ 57.106ms (SLO: <67.650ms 📉 -15.6%) vs baseline: +0.2% Memory: ✅ 39.398MB (SLO: <47.000MB 📉 -16.2%) vs baseline: +4.8% ✅ set-statusTime: ✅ 42.849ms (SLO: <50.400ms 📉 -15.0%) vs baseline: +1.2% Memory: ✅ 38.769MB (SLO: <47.000MB 📉 -17.5%) vs baseline: +4.9% ✅ startTime: ✅ 35.325ms (SLO: <43.450ms 📉 -18.7%) vs baseline: -0.2% Memory: ✅ 38.753MB (SLO: <47.000MB 📉 -17.5%) vs baseline: +4.6% ✅ start-finishTime: ✅ 82.113ms (SLO: <88.000ms -6.7%) vs baseline: +0.5% Memory: ✅ 36.648MB (SLO: <46.500MB 📉 -21.2%) vs baseline: +5.1% ✅ start-finish-telemetryTime: ✅ 83.661ms (SLO: <89.000ms -6.0%) vs baseline: +0.3% Memory: ✅ 36.549MB (SLO: <46.500MB 📉 -21.4%) vs baseline: +4.6% ✅ update-nameTime: ✅ 36.942ms (SLO: <45.150ms 📉 -18.2%) vs baseline: +0.3% Memory: ✅ 38.833MB (SLO: <47.000MB 📉 -17.4%) vs baseline: +4.4%
|
## Description We are trying to hit a real third party service, which is now down because of cloudflare outage. We don't actually care that the request was successful, at least from a "200" vs "404" vs "500", we just want to be sure that using the HTTPS request mechanism does properly create a span. ## Testing <!-- Describe your testing strategy or note what tests are included --> ## Risks <!-- Note any risks associated with this change, or "None" if no risks --> ## Additional Notes <!-- Any other information that would be helpful for reviewers -->
Description
We are trying to hit a real third party service, which is now down because of cloudflare outage.
We don't actually care that the request was successful, at least from a "200" vs "404" vs "500", we just want to be sure that using the HTTPS request mechanism does properly create a span.
Testing
Risks
Additional Notes