Mark Grizzly HTTP integration test as flaky by PerfectSlayer · Pull Request #6949 · DataDog/dd-trace-java

PerfectSlayer · 2024-04-26T09:16:17Z

What Does This Do

This PR marks Grizzly HTTP  integration test as flaky.

Motivation

See #6933

Additional Notes

Jira ticket: APPSEC-11220

See #6933

pr-commenter · 2024-04-26T09:59:57Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bbujon/ci-flaky-integration-grizzly-http
git_commit_date	1714049499	1714122832
git_commit_sha	`ae1c4c9`	`d14cbd3`
release_version	1.34.0-SNAPSHOT~ae1c4c9475	1.34.0-SNAPSHOT~d14cbd3e74

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1714125455	1714125455
ci_job_id	497996578	497996578
ci_pipeline_id	32998810	32998810
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.34.0-SNAPSHOT~d14cbd3e74, baseline=1.34.0-SNAPSHOT~ae1c4c9475

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.084 s) : 0, 1084470
Total [baseline] (10.377 s) : 0, 10377366
Agent [candidate] (1.084 s) : 0, 1083741
Total [candidate] (10.36 s) : 0, 10360225
section appsec
Agent [baseline] (1.196 s) : 0, 1196334
Total [baseline] (10.498 s) : 0, 10498018
Agent [candidate] (1.19 s) : 0, 1190351
Total [candidate] (10.543 s) : 0, 10543143
section iast
Agent [baseline] (1.209 s) : 0, 1208568
Total [baseline] (10.781 s) : 0, 10781395
Agent [candidate] (1.2 s) : 0, 1199988
Total [candidate] (10.731 s) : 0, 10730682
section profiling
Agent [baseline] (1.267 s) : 0, 1267001
Total [baseline] (10.676 s) : 0, 10675811
Agent [candidate] (1.277 s) : 0, 1277274
Total [candidate] (10.663 s) : 0, 10662521

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.084 s	-
Agent	appsec	1.196 s	111.864 ms (10.3%)
Agent	iast	1.209 s	124.098 ms (11.4%)
Agent	profiling	1.267 s	182.532 ms (16.8%)
Total	tracing	10.377 s	-
Total	appsec	10.498 s	120.652 ms (1.2%)
Total	iast	10.781 s	404.029 ms (3.9%)
Total	profiling	10.676 s	298.446 ms (2.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.084 s	-
Agent	appsec	1.19 s	106.609 ms (9.8%)
Agent	iast	1.2 s	116.246 ms (10.7%)
Agent	profiling	1.277 s	193.533 ms (17.9%)
Total	tracing	10.36 s	-
Total	appsec	10.543 s	182.918 ms (1.8%)
Total	iast	10.731 s	370.457 ms (3.6%)
Total	profiling	10.663 s	302.296 ms (2.9%)

gantt
    title petclinic - break down per module: candidate=1.34.0-SNAPSHOT~d14cbd3e74, baseline=1.34.0-SNAPSHOT~ae1c4c9475

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (679.714 ms) : 0, 679714
BytebuddyAgent [candidate] (678.763 ms) : 0, 678763
GlobalTracer [baseline] (312.11 ms) : 0, 312110
GlobalTracer [candidate] (312.27 ms) : 0, 312270
AppSec [baseline] (49.744 ms) : 0, 49744
AppSec [candidate] (49.835 ms) : 0, 49835
Remote Config [baseline] (664.622 µs) : 0, 665
Remote Config [candidate] (658.348 µs) : 0, 658
Telemetry [baseline] (7.604 ms) : 0, 7604
Telemetry [candidate] (7.64 ms) : 0, 7640
section appsec
BytebuddyAgent [baseline] (700.34 ms) : 0, 700340
BytebuddyAgent [candidate] (696.235 ms) : 0, 696235
GlobalTracer [baseline] (293.274 ms) : 0, 293274
GlobalTracer [candidate] (291.947 ms) : 0, 291947
AppSec [baseline] (149.864 ms) : 0, 149864
AppSec [candidate] (149.318 ms) : 0, 149318
Remote Config [baseline] (622.975 µs) : 0, 623
Remote Config [candidate] (617.986 µs) : 0, 618
Telemetry [baseline] (7.91 ms) : 0, 7910
Telemetry [candidate] (8.204 ms) : 0, 8204
IAST [baseline] (19.429 ms) : 0, 19429
IAST [candidate] (19.291 ms) : 0, 19291
section iast
BytebuddyAgent [baseline] (800.761 ms) : 0, 800761
BytebuddyAgent [candidate] (795.229 ms) : 0, 795229
GlobalTracer [baseline] (290.539 ms) : 0, 290539
GlobalTracer [candidate] (288.488 ms) : 0, 288488
AppSec [baseline] (48.655 ms) : 0, 48655
AppSec [candidate] (48.911 ms) : 0, 48911
Remote Config [baseline] (577.669 µs) : 0, 578
Remote Config [candidate] (578.768 µs) : 0, 579
Telemetry [baseline] (6.633 ms) : 0, 6633
Telemetry [candidate] (7.419 ms) : 0, 7419
IAST [baseline] (26.763 ms) : 0, 26763
IAST [candidate] (25.036 ms) : 0, 25036
section profiling
BytebuddyAgent [baseline] (676.851 ms) : 0, 676851
BytebuddyAgent [candidate] (683.215 ms) : 0, 683215
GlobalTracer [baseline] (379.815 ms) : 0, 379815
GlobalTracer [candidate] (382.885 ms) : 0, 382885
AppSec [baseline] (50.176 ms) : 0, 50176
AppSec [candidate] (50.423 ms) : 0, 50423
Remote Config [baseline] (708.711 µs) : 0, 709
Remote Config [candidate] (717.31 µs) : 0, 717
Telemetry [baseline] (7.427 ms) : 0, 7427
Telemetry [candidate] (7.601 ms) : 0, 7601
ProfilingAgent [baseline] (95.639 ms) : 0, 95639
ProfilingAgent [candidate] (95.615 ms) : 0, 95615
Profiling [baseline] (95.663 ms) : 0, 95663
Profiling [candidate] (95.639 ms) : 0, 95639

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.34.0-SNAPSHOT~d14cbd3e74, baseline=1.34.0-SNAPSHOT~ae1c4c9475

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.082 s) : 0, 1082094
Total [baseline] (8.58 s) : 0, 8579958
Agent [candidate] (1.085 s) : 0, 1084627
Total [candidate] (8.565 s) : 0, 8564900
section iast
Agent [baseline] (1.197 s) : 0, 1197209
Total [baseline] (9.0 s) : 0, 8999677
Agent [candidate] (1.201 s) : 0, 1200936
Total [candidate] (9.014 s) : 0, 9014164
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.199 s) : 0, 1199296
Total [baseline] (8.996 s) : 0, 8995729
Agent [candidate] (1.197 s) : 0, 1196701
Total [candidate] (9.005 s) : 0, 9005140
section iast_TELEMETRY_OFF
Agent [baseline] (1.198 s) : 0, 1197826
Total [baseline] (9.001 s) : 0, 9001041
Agent [candidate] (1.197 s) : 0, 1197110
Total [candidate] (9.018 s) : 0, 9017658

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.082 s	-
Agent	iast	1.197 s	115.114 ms (10.6%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.199 s	117.202 ms (10.8%)
Agent	iast_TELEMETRY_OFF	1.198 s	115.732 ms (10.7%)
Total	tracing	8.58 s	-
Total	iast	9.0 s	419.719 ms (4.9%)
Total	iast_HARDCODED_SECRET_DISABLED	8.996 s	415.77 ms (4.8%)
Total	iast_TELEMETRY_OFF	9.001 s	421.083 ms (4.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.085 s	-
Agent	iast	1.201 s	116.31 ms (10.7%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.197 s	112.074 ms (10.3%)
Agent	iast_TELEMETRY_OFF	1.197 s	112.484 ms (10.4%)
Total	tracing	8.565 s	-
Total	iast	9.014 s	449.264 ms (5.2%)
Total	iast_HARDCODED_SECRET_DISABLED	9.005 s	440.24 ms (5.1%)
Total	iast_TELEMETRY_OFF	9.018 s	452.758 ms (5.3%)

gantt
    title insecure-bank - break down per module: candidate=1.34.0-SNAPSHOT~d14cbd3e74, baseline=1.34.0-SNAPSHOT~ae1c4c9475

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (677.824 ms) : 0, 677824
BytebuddyAgent [candidate] (679.583 ms) : 0, 679583
GlobalTracer [baseline] (311.799 ms) : 0, 311799
GlobalTracer [candidate] (312.17 ms) : 0, 312170
AppSec [baseline] (49.675 ms) : 0, 49675
AppSec [candidate] (49.834 ms) : 0, 49834
Remote Config [baseline] (662.279 µs) : 0, 662
Remote Config [candidate] (671.138 µs) : 0, 671
Telemetry [baseline] (7.561 ms) : 0, 7561
Telemetry [candidate] (7.743 ms) : 0, 7743
section iast
BytebuddyAgent [baseline] (792.814 ms) : 0, 792814
BytebuddyAgent [candidate] (795.523 ms) : 0, 795523
GlobalTracer [baseline] (288.214 ms) : 0, 288214
GlobalTracer [candidate] (288.822 ms) : 0, 288822
AppSec [baseline] (50.453 ms) : 0, 50453
AppSec [candidate] (49.761 ms) : 0, 49761
IAST [baseline] (24.163 ms) : 0, 24163
IAST [candidate] (23.645 ms) : 0, 23645
Remote Config [baseline] (572.624 µs) : 0, 573
Remote Config [candidate] (1.339 ms) : 0, 1339
Telemetry [baseline] (6.638 ms) : 0, 6638
Telemetry [candidate] (7.444 ms) : 0, 7444
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (795.079 ms) : 0, 795079
BytebuddyAgent [candidate] (793.333 ms) : 0, 793333
GlobalTracer [baseline] (288.348 ms) : 0, 288348
GlobalTracer [candidate] (287.988 ms) : 0, 287988
AppSec [baseline] (50.685 ms) : 0, 50685
AppSec [candidate] (51.355 ms) : 0, 51355
IAST [baseline] (22.564 ms) : 0, 22564
IAST [candidate] (21.709 ms) : 0, 21709
Remote Config [baseline] (571.753 µs) : 0, 572
Remote Config [candidate] (581.937 µs) : 0, 582
Telemetry [baseline] (7.418 ms) : 0, 7418
Telemetry [candidate] (7.371 ms) : 0, 7371
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (792.922 ms) : 0, 792922
BytebuddyAgent [candidate] (792.745 ms) : 0, 792745
GlobalTracer [baseline] (288.732 ms) : 0, 288732
GlobalTracer [candidate] (288.882 ms) : 0, 288882
AppSec [baseline] (49.107 ms) : 0, 49107
AppSec [candidate] (48.055 ms) : 0, 48055
IAST [baseline] (24.684 ms) : 0, 24684
IAST [candidate] (25.158 ms) : 0, 25158
Remote Config [baseline] (593.484 µs) : 0, 593
Remote Config [candidate] (576.85 µs) : 0, 577
Telemetry [baseline] (7.412 ms) : 0, 7412
Telemetry [candidate] (7.345 ms) : 0, 7345

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-04-26T09:28:28	2024-04-26T09:35:17
git_branch	master	bbujon/ci-flaky-integration-grizzly-http
git_commit_date	1714049499	1714122832
git_commit_sha	`ae1c4c9`	`d14cbd3`
release_version	1.34.0-SNAPSHOT~ae1c4c9475	1.34.0-SNAPSHOT~d14cbd3e74
start_time	2024-04-26T09:28:15	2024-04-26T09:35:03

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1714124462	1714124462
ci_job_id	497996579	497996579
ci_pipeline_id	32998810	32998810
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.34.0-SNAPSHOT~d14cbd3e74, baseline=1.34.0-SNAPSHOT~ae1c4c9475
    dateFormat X
    axisFormat %s
section baseline
no_agent (370.144 µs) : 351, 389
.   : milestone, 370,
iast (471.999 µs) : 451, 493
.   : milestone, 472,
iast_FULL (545.444 µs) : 524, 567
.   : milestone, 545,
iast_GLOBAL (495.526 µs) : 475, 516
.   : milestone, 496,
iast_HARDCODED_SECRET_DISABLED (471.712 µs) : 451, 492
.   : milestone, 472,
iast_INACTIVE (454.077 µs) : 433, 475
.   : milestone, 454,
iast_TELEMETRY_OFF (477.616 µs) : 456, 499
.   : milestone, 478,
tracing (446.71 µs) : 426, 468
.   : milestone, 447,
section candidate
no_agent (369.891 µs) : 350, 390
.   : milestone, 370,
iast (481.369 µs) : 460, 503
.   : milestone, 481,
iast_FULL (542.703 µs) : 521, 564
.   : milestone, 543,
iast_GLOBAL (507.393 µs) : 485, 530
.   : milestone, 507,
iast_HARDCODED_SECRET_DISABLED (476.38 µs) : 455, 498
.   : milestone, 476,
iast_INACTIVE (455.762 µs) : 434, 477
.   : milestone, 456,
iast_TELEMETRY_OFF (466.423 µs) : 446, 487
.   : milestone, 466,
tracing (443.549 µs) : 423, 464
.   : milestone, 444,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	370.144 µs [350.833 µs, 389.455 µs]	-
iast	471.999 µs [450.925 µs, 493.073 µs]	101.855 µs (27.5%)
iast_FULL	545.444 µs [523.614 µs, 567.274 µs]	175.3 µs (47.4%)
iast_GLOBAL	495.526 µs [475.306 µs, 515.747 µs]	125.382 µs (33.9%)
iast_HARDCODED_SECRET_DISABLED	471.712 µs [450.929 µs, 492.495 µs]	101.568 µs (27.4%)
iast_INACTIVE	454.077 µs [433.163 µs, 474.991 µs]	83.933 µs (22.7%)
iast_TELEMETRY_OFF	477.616 µs [456.242 µs, 498.989 µs]	107.472 µs (29.0%)
tracing	446.71 µs [425.757 µs, 467.663 µs]	76.566 µs (20.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	369.891 µs [350.223 µs, 389.559 µs]	-
iast	481.369 µs [459.854 µs, 502.883 µs]	111.477 µs (30.1%)
iast_FULL	542.703 µs [521.455 µs, 563.951 µs]	172.812 µs (46.7%)
iast_GLOBAL	507.393 µs [485.268 µs, 529.517 µs]	137.502 µs (37.2%)
iast_HARDCODED_SECRET_DISABLED	476.38 µs [454.711 µs, 498.05 µs]	106.489 µs (28.8%)
iast_INACTIVE	455.762 µs [434.403 µs, 477.121 µs]	85.871 µs (23.2%)
iast_TELEMETRY_OFF	466.423 µs [446.088 µs, 486.758 µs]	96.532 µs (26.1%)
tracing	443.549 µs [423.399 µs, 463.699 µs]	73.658 µs (19.9%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.34.0-SNAPSHOT~d14cbd3e74, baseline=1.34.0-SNAPSHOT~ae1c4c9475
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.36 ms) : 1341, 1380
.   : milestone, 1360,
appsec (1.721 ms) : 1697, 1746
.   : milestone, 1721,
appsec_no_iast (1.711 ms) : 1686, 1736
.   : milestone, 1711,
iast (1.486 ms) : 1463, 1510
.   : milestone, 1486,
profiling (1.508 ms) : 1483, 1534
.   : milestone, 1508,
tracing (1.461 ms) : 1436, 1486
.   : milestone, 1461,
section candidate
no_agent (1.363 ms) : 1343, 1384
.   : milestone, 1363,
appsec (1.699 ms) : 1674, 1724
.   : milestone, 1699,
appsec_no_iast (1.744 ms) : 1720, 1768
.   : milestone, 1744,
iast (1.498 ms) : 1476, 1520
.   : milestone, 1498,
profiling (1.548 ms) : 1522, 1574
.   : milestone, 1548,
tracing (1.477 ms) : 1452, 1501
.   : milestone, 1477,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.36 ms [1.341 ms, 1.38 ms]	-
appsec	1.721 ms [1.697 ms, 1.746 ms]	361.292 µs (26.6%)
appsec_no_iast	1.711 ms [1.686 ms, 1.736 ms]	350.922 µs (25.8%)
iast	1.486 ms [1.463 ms, 1.51 ms]	126.208 µs (9.3%)
profiling	1.508 ms [1.483 ms, 1.534 ms]	148.301 µs (10.9%)
tracing	1.461 ms [1.436 ms, 1.486 ms]	101.298 µs (7.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.363 ms [1.343 ms, 1.384 ms]	-
appsec	1.699 ms [1.674 ms, 1.724 ms]	335.333 µs (24.6%)
appsec_no_iast	1.744 ms [1.72 ms, 1.768 ms]	380.539 µs (27.9%)
iast	1.498 ms [1.476 ms, 1.52 ms]	134.667 µs (9.9%)
profiling	1.548 ms [1.522 ms, 1.574 ms]	184.509 µs (13.5%)
tracing	1.477 ms [1.452 ms, 1.501 ms]	113.087 µs (8.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bbujon/ci-flaky-integration-grizzly-http
git_commit_date	1714049499	1714122832
git_commit_sha	`ae1c4c9`	`d14cbd3`
release_version	1.34.0-SNAPSHOT~ae1c4c9475	1.34.0-SNAPSHOT~d14cbd3e74

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1714124950	1714124950
ci_job_id	497996580	497996580
ci_pipeline_id	32998810	32998810
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.34.0-SNAPSHOT~d14cbd3e74, baseline=1.34.0-SNAPSHOT~ae1c4c9475
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.04 s) : 15040000, 15040000
.   : milestone, 15040000,
appsec (15.305 s) : 15305000, 15305000
.   : milestone, 15305000,
iast (18.861 s) : 18861000, 18861000
.   : milestone, 18861000,
iast_GLOBAL (17.784 s) : 17784000, 17784000
.   : milestone, 17784000,
profiling (15.242 s) : 15242000, 15242000
.   : milestone, 15242000,
tracing (14.82 s) : 14820000, 14820000
.   : milestone, 14820000,
section candidate
no_agent (15.439 s) : 15439000, 15439000
.   : milestone, 15439000,
appsec (15.224 s) : 15224000, 15224000
.   : milestone, 15224000,
iast (18.637 s) : 18637000, 18637000
.   : milestone, 18637000,
iast_GLOBAL (17.943 s) : 17943000, 17943000
.   : milestone, 17943000,
profiling (15.902 s) : 15902000, 15902000
.   : milestone, 15902000,
tracing (15.011 s) : 15011000, 15011000
.   : milestone, 15011000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.04 s [15.04 s, 15.04 s]	-
appsec	15.305 s [15.305 s, 15.305 s]	265.0 ms (1.8%)
iast	18.861 s [18.861 s, 18.861 s]	3.821 s (25.4%)
iast_GLOBAL	17.784 s [17.784 s, 17.784 s]	2.744 s (18.2%)
profiling	15.242 s [15.242 s, 15.242 s]	202.0 ms (1.3%)
tracing	14.82 s [14.82 s, 14.82 s]	-220.0 ms (-1.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.439 s [15.439 s, 15.439 s]	-
appsec	15.224 s [15.224 s, 15.224 s]	-215.0 ms (-1.4%)
iast	18.637 s [18.637 s, 18.637 s]	3.198 s (20.7%)
iast_GLOBAL	17.943 s [17.943 s, 17.943 s]	2.504 s (16.2%)
profiling	15.902 s [15.902 s, 15.902 s]	463.0 ms (3.0%)
tracing	15.011 s [15.011 s, 15.011 s]	-428.0 ms (-2.8%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.34.0-SNAPSHOT~d14cbd3e74, baseline=1.34.0-SNAPSHOT~ae1c4c9475
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.46 ms) : 1449, 1472
.   : milestone, 1460,
appsec (2.193 ms) : 2159, 2227
.   : milestone, 2193,
iast (1.886 ms) : 1851, 1921
.   : milestone, 1886,
iast_GLOBAL (1.908 ms) : 1873, 1943
.   : milestone, 1908,
profiling (2.315 ms) : 2136, 2493
.   : milestone, 2315,
tracing (1.822 ms) : 1790, 1853
.   : milestone, 1822,
section candidate
no_agent (1.456 ms) : 1445, 1468
.   : milestone, 1456,
appsec (2.196 ms) : 2163, 2230
.   : milestone, 2196,
iast (1.878 ms) : 1843, 1913
.   : milestone, 1878,
iast_GLOBAL (1.923 ms) : 1887, 1959
.   : milestone, 1923,
profiling (1.85 ms) : 1818, 1883
.   : milestone, 1850,
tracing (1.82 ms) : 1789, 1851
.   : milestone, 1820,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.46 ms [1.449 ms, 1.472 ms]	-
appsec	2.193 ms [2.159 ms, 2.227 ms]	733.149 µs (50.2%)
iast	1.886 ms [1.851 ms, 1.921 ms]	425.85 µs (29.2%)
iast_GLOBAL	1.908 ms [1.873 ms, 1.943 ms]	447.893 µs (30.7%)
profiling	2.315 ms [2.136 ms, 2.493 ms]	854.38 µs (58.5%)
tracing	1.822 ms [1.79 ms, 1.853 ms]	361.315 µs (24.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.456 ms [1.445 ms, 1.468 ms]	-
appsec	2.196 ms [2.163 ms, 2.23 ms]	739.82 µs (50.8%)
iast	1.878 ms [1.843 ms, 1.913 ms]	421.777 µs (29.0%)
iast_GLOBAL	1.923 ms [1.887 ms, 1.959 ms]	466.297 µs (32.0%)
profiling	1.85 ms [1.818 ms, 1.883 ms]	394.02 µs (27.1%)
tracing	1.82 ms [1.789 ms, 1.851 ms]	363.738 µs (25.0%)

chore: Mark test as flaky

d14cbd3

See #6933

PerfectSlayer added inst: others All other instrumentations tag: flaky test Flaky tests tag: flaky test/disabled Disabled flaky tests labels Apr 26, 2024

PerfectSlayer requested a review from a team as a code owner April 26, 2024 09:16

PerfectSlayer requested review from am312 and dougqh April 26, 2024 09:16

PerfectSlayer changed the title ~~chore: Mark test as flaky~~ Mark Grizzly HTTP integration test as flaky Apr 26, 2024

smola approved these changes Apr 26, 2024

View reviewed changes

PerfectSlayer merged commit 8c1f7d8 into master Apr 29, 2024

PerfectSlayer deleted the bbujon/ci-flaky-integration-grizzly-http branch April 29, 2024 06:37

github-actions bot added this to the 1.34.0 milestone Apr 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mark Grizzly HTTP integration test as flaky#6949

Mark Grizzly HTTP integration test as flaky#6949
PerfectSlayer merged 1 commit intomasterfrom
bbujon/ci-flaky-integration-grizzly-http

PerfectSlayer commented Apr 26, 2024 •

edited by atlassian bot

Loading

Uh oh!

pr-commenter bot commented Apr 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

PerfectSlayer commented Apr 26, 2024 • edited by atlassian bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Uh oh!

pr-commenter bot commented Apr 26, 2024

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

PerfectSlayer commented Apr 26, 2024 •

edited by atlassian bot

Loading