Add latestDepTest for Jvm 21 on CI #6542

amarziali · 2024-01-24T13:29:52Z

What Does This Do

Add a latestDep pipeline on the CI build using a JVM 21

Motivation

Additional Notes

Jira ticket: [PROJ-IDENT]

pr-commenter · 2024-01-24T14:13:18Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	andrea.marziali/latestdep-21
git_commit_date	1706091852	1706102932
git_commit_sha	`98e7a43`	`d43489d`
release_version	1.29.0-SNAPSHOT~98e7a4379b	1.29.0-SNAPSHOT~d43489df91

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1706105557	1706105557
ci_job_id	417284292	417284292
ci_pipeline_id	27095480	27095480
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 8 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.29.0-SNAPSHOT~d43489df91, baseline=1.29.0-SNAPSHOT~98e7a4379b

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.06 s) : 0, 1059580
Total [baseline] (8.774 s) : 0, 8774087
Agent [candidate] (1.062 s) : 0, 1062003
Total [candidate] (8.756 s) : 0, 8756277
section iast
Agent [baseline] (1.185 s) : 0, 1185059
Total [baseline] (9.281 s) : 0, 9280907
Agent [candidate] (1.197 s) : 0, 1197083
Total [candidate] (9.262 s) : 0, 9261681
section iast_TELEMETRY_OFF
Agent [baseline] (1.177 s) : 0, 1177344
Total [baseline] (9.261 s) : 0, 9261422
Agent [candidate] (1.171 s) : 0, 1171334
Total [candidate] (9.299 s) : 0, 9298953

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.06 s	-
Agent	iast	1.185 s	125.478 ms (11.8%)
Agent	iast_TELEMETRY_OFF	1.177 s	117.763 ms (11.1%)
Total	tracing	8.774 s	-
Total	iast	9.281 s	506.82 ms (5.8%)
Total	iast_TELEMETRY_OFF	9.261 s	487.335 ms (5.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.062 s	-
Agent	iast	1.197 s	135.08 ms (12.7%)
Agent	iast_TELEMETRY_OFF	1.171 s	109.331 ms (10.3%)
Total	tracing	8.756 s	-
Total	iast	9.262 s	505.404 ms (5.8%)
Total	iast_TELEMETRY_OFF	9.299 s	542.676 ms (6.2%)

gantt
    title insecure-bank - break down per module: candidate=1.29.0-SNAPSHOT~d43489df91, baseline=1.29.0-SNAPSHOT~98e7a4379b

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (668.963 ms) : 0, 668963
BytebuddyAgent [candidate] (669.783 ms) : 0, 669783
GlobalTracer [baseline] (296.074 ms) : 0, 296074
GlobalTracer [candidate] (297.182 ms) : 0, 297182
AppSec [baseline] (51.928 ms) : 0, 51928
AppSec [candidate] (52.157 ms) : 0, 52157
Remote Config [baseline] (706.995 µs) : 0, 707
Remote Config [candidate] (698.848 µs) : 0, 699
Telemetry [baseline] (7.625 ms) : 0, 7625
Telemetry [candidate] (7.776 ms) : 0, 7776
section iast
BytebuddyAgent [baseline] (780.443 ms) : 0, 780443
BytebuddyAgent [candidate] (790.428 ms) : 0, 790428
GlobalTracer [baseline] (287.377 ms) : 0, 287377
GlobalTracer [candidate] (289.43 ms) : 0, 289430
AppSec [baseline] (55.843 ms) : 0, 55843
AppSec [candidate] (55.601 ms) : 0, 55601
IAST [baseline] (19.413 ms) : 0, 19413
IAST [candidate] (19.399 ms) : 0, 19399
Remote Config [baseline] (608.073 µs) : 0, 608
Remote Config [candidate] (588.044 µs) : 0, 588
Telemetry [baseline] (6.745 ms) : 0, 6745
Telemetry [candidate] (6.722 ms) : 0, 6722
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (773.631 ms) : 0, 773631
BytebuddyAgent [candidate] (767.907 ms) : 0, 767907
GlobalTracer [baseline] (287.74 ms) : 0, 287740
GlobalTracer [candidate] (287.32 ms) : 0, 287320
AppSec [baseline] (56.808 ms) : 0, 56808
AppSec [candidate] (56.446 ms) : 0, 56446
IAST [baseline] (17.603 ms) : 0, 17603
IAST [candidate] (18.4 ms) : 0, 18400
Remote Config [baseline] (622.505 µs) : 0, 623
Remote Config [candidate] (617.147 µs) : 0, 617
Telemetry [baseline] (6.366 ms) : 0, 6366
Telemetry [candidate] (6.402 ms) : 0, 6402

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.29.0-SNAPSHOT~d43489df91, baseline=1.29.0-SNAPSHOT~98e7a4379b

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.053 s) : 0, 1053169
Total [baseline] (9.3 s) : 0, 9299721
Agent [candidate] (1.056 s) : 0, 1055588
Total [candidate] (9.335 s) : 0, 9335259
section appsec
Agent [baseline] (1.151 s) : 0, 1151101
Total [baseline] (9.448 s) : 0, 9448098
Agent [candidate] (1.159 s) : 0, 1159198
Total [candidate] (9.45 s) : 0, 9450243
section iast
Agent [baseline] (1.174 s) : 0, 1173526
Total [baseline] (9.607 s) : 0, 9606602
Agent [candidate] (1.186 s) : 0, 1185802
Total [candidate] (9.603 s) : 0, 9602995
section profiling
Agent [baseline] (1.285 s) : 0, 1285106
Total [baseline] (9.62 s) : 0, 9619862
Agent [candidate] (1.286 s) : 0, 1286404
Total [candidate] (9.586 s) : 0, 9585839

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.053 s	-
Agent	appsec	1.151 s	97.933 ms (9.3%)
Agent	iast	1.174 s	120.358 ms (11.4%)
Agent	profiling	1.285 s	231.937 ms (22.0%)
Total	tracing	9.3 s	-
Total	appsec	9.448 s	148.376 ms (1.6%)
Total	iast	9.607 s	306.881 ms (3.3%)
Total	profiling	9.62 s	320.141 ms (3.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.056 s	-
Agent	appsec	1.159 s	103.61 ms (9.8%)
Agent	iast	1.186 s	130.214 ms (12.3%)
Agent	profiling	1.286 s	230.816 ms (21.9%)
Total	tracing	9.335 s	-
Total	appsec	9.45 s	114.984 ms (1.2%)
Total	iast	9.603 s	267.736 ms (2.9%)
Total	profiling	9.586 s	250.58 ms (2.7%)

gantt
    title petclinic - break down per module: candidate=1.29.0-SNAPSHOT~d43489df91, baseline=1.29.0-SNAPSHOT~98e7a4379b

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (663.948 ms) : 0, 663948
BytebuddyAgent [candidate] (665.539 ms) : 0, 665539
GlobalTracer [baseline] (295.199 ms) : 0, 295199
GlobalTracer [candidate] (295.435 ms) : 0, 295435
AppSec [baseline] (51.715 ms) : 0, 51715
AppSec [candidate] (52.086 ms) : 0, 52086
Remote Config [baseline] (683.852 µs) : 0, 684
Remote Config [candidate] (695.647 µs) : 0, 696
Telemetry [baseline] (7.513 ms) : 0, 7513
Telemetry [candidate] (7.562 ms) : 0, 7562
section appsec
BytebuddyAgent [baseline] (664.635 ms) : 0, 664635
BytebuddyAgent [candidate] (669.648 ms) : 0, 669648
GlobalTracer [baseline] (295.071 ms) : 0, 295071
GlobalTracer [candidate] (296.824 ms) : 0, 296824
AppSec [baseline] (149.759 ms) : 0, 149759
AppSec [candidate] (150.917 ms) : 0, 150917
Remote Config [baseline] (693.723 µs) : 0, 694
Remote Config [candidate] (704.067 µs) : 0, 704
Telemetry [baseline] (6.751 ms) : 0, 6751
Telemetry [candidate] (6.802 ms) : 0, 6802
section iast
BytebuddyAgent [baseline] (772.775 ms) : 0, 772775
BytebuddyAgent [candidate] (780.942 ms) : 0, 780942
GlobalTracer [baseline] (285.464 ms) : 0, 285464
GlobalTracer [candidate] (288.219 ms) : 0, 288219
AppSec [baseline] (54.565 ms) : 0, 54565
AppSec [candidate] (54.638 ms) : 0, 54638
Remote Config [baseline] (601.021 µs) : 0, 601
Remote Config [candidate] (604.916 µs) : 0, 605
Telemetry [baseline] (6.614 ms) : 0, 6614
Telemetry [candidate] (7.355 ms) : 0, 7355
IAST [baseline] (19.238 ms) : 0, 19238
IAST [candidate] (19.413 ms) : 0, 19413
section profiling
ProfilingAgent [baseline] (124.315 ms) : 0, 124315
ProfilingAgent [candidate] (124.299 ms) : 0, 124299
BytebuddyAgent [baseline] (667.242 ms) : 0, 667242
BytebuddyAgent [candidate] (667.671 ms) : 0, 667671
GlobalTracer [baseline] (378.026 ms) : 0, 378026
GlobalTracer [candidate] (378.59 ms) : 0, 378590
AppSec [baseline] (52.674 ms) : 0, 52674
AppSec [candidate] (52.909 ms) : 0, 52909
Remote Config [baseline] (670.916 µs) : 0, 671
Remote Config [candidate] (679.841 µs) : 0, 680
Telemetry [baseline] (7.563 ms) : 0, 7563
Telemetry [candidate] (7.577 ms) : 0, 7577
Profiling [baseline] (124.34 ms) : 0, 124340
Profiling [candidate] (124.324 ms) : 0, 124324

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-01-24T13:51:51	2024-01-24T14:08:24
git_branch	master	andrea.marziali/latestdep-21
git_commit_date	1706091852	1706102932
git_commit_sha	`98e7a43`	`d43489d`
release_version	1.29.0-SNAPSHOT~98e7a4379b	1.29.0-SNAPSHOT~d43489df91
start_time	2024-01-24T13:51:38	2024-01-24T14:08:11

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1706105557	1706105557
ci_job_id	417284292	417284292
ci_pipeline_id	27095480	27095480
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 14 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.29.0-SNAPSHOT~d43489df91, baseline=1.29.0-SNAPSHOT~98e7a4379b
    dateFormat X
    axisFormat %s
section baseline
no_agent (367.734 µs) : 347, 388
.   : milestone, 368,
iast (471.838 µs) : 451, 493
.   : milestone, 472,
iast_FULL (533.608 µs) : 513, 554
.   : milestone, 534,
iast_INACTIVE (439.838 µs) : 419, 460
.   : milestone, 440,
iast_TELEMETRY_OFF (469.644 µs) : 449, 490
.   : milestone, 470,
tracing (443.661 µs) : 422, 465
.   : milestone, 444,
section candidate
no_agent (360.734 µs) : 340, 382
.   : milestone, 361,
iast (469.962 µs) : 449, 491
.   : milestone, 470,
iast_FULL (542.208 µs) : 522, 563
.   : milestone, 542,
iast_INACTIVE (450.164 µs) : 429, 471
.   : milestone, 450,
iast_TELEMETRY_OFF (478.748 µs) : 458, 499
.   : milestone, 479,
tracing (439.393 µs) : 418, 461
.   : milestone, 439,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	367.734 µs [347.249 µs, 388.22 µs]	-
iast	471.838 µs [450.94 µs, 492.736 µs]	104.104 µs (28.3%)
iast_FULL	533.608 µs [513.005 µs, 554.211 µs]	165.874 µs (45.1%)
iast_INACTIVE	439.838 µs [419.349 µs, 460.327 µs]	72.104 µs (19.6%)
iast_TELEMETRY_OFF	469.644 µs [449.224 µs, 490.063 µs]	101.909 µs (27.7%)
tracing	443.661 µs [422.377 µs, 464.945 µs]	75.927 µs (20.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	360.734 µs [339.925 µs, 381.544 µs]	-
iast	469.962 µs [449.314 µs, 490.61 µs]	109.228 µs (30.3%)
iast_FULL	542.208 µs [521.853 µs, 562.563 µs]	181.474 µs (50.3%)
iast_INACTIVE	450.164 µs [429.385 µs, 470.943 µs]	89.43 µs (24.8%)
iast_TELEMETRY_OFF	478.748 µs [458.093 µs, 499.402 µs]	118.013 µs (32.7%)
tracing	439.393 µs [417.816 µs, 460.969 µs]	78.658 µs (21.8%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.29.0-SNAPSHOT~d43489df91, baseline=1.29.0-SNAPSHOT~98e7a4379b
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.341 ms) : 1322, 1359
.   : milestone, 1341,
appsec (1.774 ms) : 1749, 1799
.   : milestone, 1774,
iast (1.519 ms) : 1495, 1543
.   : milestone, 1519,
profiling (1.513 ms) : 1488, 1538
.   : milestone, 1513,
tracing (1.5 ms) : 1475, 1525
.   : milestone, 1500,
section candidate
no_agent (1.349 ms) : 1330, 1367
.   : milestone, 1349,
appsec (1.738 ms) : 1713, 1764
.   : milestone, 1738,
iast (1.518 ms) : 1493, 1542
.   : milestone, 1518,
profiling (1.521 ms) : 1494, 1547
.   : milestone, 1521,
tracing (1.473 ms) : 1449, 1498
.   : milestone, 1473,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.341 ms [1.322 ms, 1.359 ms]	-
appsec	1.774 ms [1.749 ms, 1.799 ms]	433.468 µs (32.3%)
iast	1.519 ms [1.495 ms, 1.543 ms]	178.345 µs (13.3%)
profiling	1.513 ms [1.488 ms, 1.538 ms]	172.66 µs (12.9%)
tracing	1.5 ms [1.475 ms, 1.525 ms]	159.139 µs (11.9%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.349 ms [1.33 ms, 1.367 ms]	-
appsec	1.738 ms [1.713 ms, 1.764 ms]	389.715 µs (28.9%)
iast	1.518 ms [1.493 ms, 1.542 ms]	169.113 µs (12.5%)
profiling	1.521 ms [1.494 ms, 1.547 ms]	171.955 µs (12.8%)
tracing	1.473 ms [1.449 ms, 1.498 ms]	124.843 µs (9.3%)

add latestDepTest for Jvm 21 on CI

d43489d

amarziali requested a review from a team as a code owner January 24, 2024 13:29

amarziali requested review from dougqh and am312 January 24, 2024 13:29

amarziali added the comp: testing Testing label Jan 24, 2024

PerfectSlayer approved these changes Jan 24, 2024

View reviewed changes

amarziali merged commit 3dc58c7 into master Jan 24, 2024
82 checks passed

amarziali deleted the andrea.marziali/latestdep-21 branch January 24, 2024 14:56

github-actions bot added this to the 1.29.0 milestone Jan 24, 2024

bm1549 mentioned this pull request Jan 29, 2024

Update JDK 21 to GA DataDog/documentation#21560

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add latestDepTest for Jvm 21 on CI #6542

Add latestDepTest for Jvm 21 on CI #6542

amarziali commented Jan 24, 2024

pr-commenter bot commented Jan 24, 2024

Add latestDepTest for Jvm 21 on CI #6542

Add latestDepTest for Jvm 21 on CI #6542

Conversation

amarziali commented Jan 24, 2024

What Does This Do

Motivation

Additional Notes

pr-commenter bot commented Jan 24, 2024

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary