Drop the requirement of jvm environment variables for `testJvm` constraint #9968

bric3 · 2025-11-14T11:45:46Z

What Does This Do

Allow our testJvmContraint plugin to let Gradle find the relevant JVM using the Gradle Toolchain.

Also, in CI, discover dynamically the JAVA_x_HOME environment variables exposed by the DataDog/dd-trace-java-docker-build image.

Motivation

Easier local setup. And follow-up to

Drop fromEnv toolchain restriction for local environment #9950

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

…ocal env, but keep them in CI

pr-commenter · 2025-11-14T12:30:03Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bdu/avoids-need-for-JVM-environment-variables
git_commit_date	1763657906	1763658312
git_commit_sha	`533755a`	`e5d8ca2`
release_version	1.57.0-SNAPSHOT~533755a5d1	1.56.0-SNAPSHOT~e5d8ca24c1

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1763661371	1763661371
ci_job_id	1248378102	1248378102
ci_pipeline_id	83420718	83420718
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-ftdjyd7n 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-ftdjyd7n 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 5 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.56.0-SNAPSHOT~e5d8ca24c1, baseline=1.57.0-SNAPSHOT~533755a5d1

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.107 s) : 0, 1106549
Total [baseline] (8.863 s) : 0, 8863456
Agent [candidate] (1.113 s) : 0, 1112806
Total [candidate] (8.87 s) : 0, 8870418
section iast
Agent [baseline] (1.251 s) : 0, 1251133
Total [baseline] (9.565 s) : 0, 9565315
Agent [candidate] (1.253 s) : 0, 1252701
Total [candidate] (9.608 s) : 0, 9608122

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.107 s	-
Agent	iast	1.251 s	144.584 ms (13.1%)
Total	tracing	8.863 s	-
Total	iast	9.565 s	701.859 ms (7.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.113 s	-
Agent	iast	1.253 s	139.895 ms (12.6%)
Total	tracing	8.87 s	-
Total	iast	9.608 s	737.704 ms (8.3%)

gantt
    title insecure-bank - break down per module: candidate=1.56.0-SNAPSHOT~e5d8ca24c1, baseline=1.57.0-SNAPSHOT~533755a5d1

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.46 ms) : 0, 1460
crashtracking [candidate] (1.466 ms) : 0, 1466
BytebuddyAgent [baseline] (710.883 ms) : 0, 710883
BytebuddyAgent [candidate] (716.673 ms) : 0, 716673
GlobalTracer [baseline] (250.824 ms) : 0, 250824
GlobalTracer [candidate] (251.299 ms) : 0, 251299
AppSec [baseline] (32.459 ms) : 0, 32459
AppSec [candidate] (32.316 ms) : 0, 32316
Debugger [baseline] (63.391 ms) : 0, 63391
Debugger [candidate] (63.321 ms) : 0, 63321
Remote Config [baseline] (631.149 µs) : 0, 631
Remote Config [candidate] (643.019 µs) : 0, 643
Telemetry [baseline] (8.264 ms) : 0, 8264
Telemetry [candidate] (8.263 ms) : 0, 8263
Flare Poller [baseline] (3.75 ms) : 0, 3750
Flare Poller [candidate] (3.696 ms) : 0, 3696
section iast
crashtracking [baseline] (1.485 ms) : 0, 1485
crashtracking [candidate] (1.477 ms) : 0, 1477
BytebuddyAgent [baseline] (841.321 ms) : 0, 841321
BytebuddyAgent [candidate] (842.75 ms) : 0, 842750
GlobalTracer [baseline] (239.076 ms) : 0, 239076
GlobalTracer [candidate] (239.187 ms) : 0, 239187
AppSec [baseline] (33.26 ms) : 0, 33260
AppSec [candidate] (33.539 ms) : 0, 33539
Debugger [baseline] (60.551 ms) : 0, 60551
Debugger [candidate] (60.183 ms) : 0, 60183
Remote Config [baseline] (553.404 µs) : 0, 553
Remote Config [candidate] (544.473 µs) : 0, 544
Telemetry [baseline] (7.609 ms) : 0, 7609
Telemetry [candidate] (7.614 ms) : 0, 7614
Flare Poller [baseline] (3.444 ms) : 0, 3444
Flare Poller [candidate] (3.457 ms) : 0, 3457
IAST [baseline] (28.716 ms) : 0, 28716
IAST [candidate] (28.614 ms) : 0, 28614

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.56.0-SNAPSHOT~e5d8ca24c1, baseline=1.57.0-SNAPSHOT~533755a5d1

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.103 s) : 0, 1103420
Total [baseline] (10.768 s) : 0, 10768363
Agent [candidate] (1.117 s) : 0, 1117157
Total [candidate] (10.9 s) : 0, 10899940
section appsec
Agent [baseline] (1.294 s) : 0, 1293789
Total [baseline] (11.224 s) : 0, 11223558
Agent [candidate] (1.284 s) : 0, 1284448
Total [candidate] (11.134 s) : 0, 11134259
section iast
Agent [baseline] (1.252 s) : 0, 1252050
Total [baseline] (11.303 s) : 0, 11303403
Agent [candidate] (1.243 s) : 0, 1243290
Total [candidate] (11.227 s) : 0, 11226789
section profiling
Agent [baseline] (1.235 s) : 0, 1235027
Total [baseline] (11.151 s) : 0, 11150502
Agent [candidate] (1.235 s) : 0, 1234675
Total [candidate] (11.199 s) : 0, 11198751

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.103 s	-
Agent	appsec	1.294 s	190.369 ms (17.3%)
Agent	iast	1.252 s	148.63 ms (13.5%)
Agent	profiling	1.235 s	131.607 ms (11.9%)
Total	tracing	10.768 s	-
Total	appsec	11.224 s	455.195 ms (4.2%)
Total	iast	11.303 s	535.04 ms (5.0%)
Total	profiling	11.151 s	382.139 ms (3.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.117 s	-
Agent	appsec	1.284 s	167.29 ms (15.0%)
Agent	iast	1.243 s	126.133 ms (11.3%)
Agent	profiling	1.235 s	117.517 ms (10.5%)
Total	tracing	10.9 s	-
Total	appsec	11.134 s	234.319 ms (2.1%)
Total	iast	11.227 s	326.849 ms (3.0%)
Total	profiling	11.199 s	298.812 ms (2.7%)

gantt
    title petclinic - break down per module: candidate=1.56.0-SNAPSHOT~e5d8ca24c1, baseline=1.57.0-SNAPSHOT~533755a5d1

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.447 ms) : 0, 1447
crashtracking [candidate] (1.479 ms) : 0, 1479
BytebuddyAgent [baseline] (708.168 ms) : 0, 708168
BytebuddyAgent [candidate] (718.223 ms) : 0, 718223
GlobalTracer [baseline] (249.647 ms) : 0, 249647
GlobalTracer [candidate] (252.509 ms) : 0, 252509
AppSec [baseline] (32.127 ms) : 0, 32127
AppSec [candidate] (32.546 ms) : 0, 32546
Debugger [baseline] (64.283 ms) : 0, 64283
Debugger [candidate] (64.545 ms) : 0, 64545
Remote Config [baseline] (631.286 µs) : 0, 631
Remote Config [candidate] (638.741 µs) : 0, 639
Telemetry [baseline] (8.396 ms) : 0, 8396
Telemetry [candidate] (8.319 ms) : 0, 8319
Flare Poller [baseline] (3.82 ms) : 0, 3820
Flare Poller [candidate] (3.717 ms) : 0, 3717
section appsec
crashtracking [baseline] (1.462 ms) : 0, 1462
crashtracking [candidate] (1.469 ms) : 0, 1469
BytebuddyAgent [baseline] (738.148 ms) : 0, 738148
BytebuddyAgent [candidate] (733.897 ms) : 0, 733897
GlobalTracer [baseline] (242.615 ms) : 0, 242615
GlobalTracer [candidate] (241.136 ms) : 0, 241136
AppSec [baseline] (176.061 ms) : 0, 176061
AppSec [candidate] (174.143 ms) : 0, 174143
Debugger [baseline] (62.347 ms) : 0, 62347
Debugger [candidate] (61.222 ms) : 0, 61222
Remote Config [baseline] (666.86 µs) : 0, 667
Remote Config [candidate] (661.508 µs) : 0, 662
Telemetry [baseline] (8.217 ms) : 0, 8217
Telemetry [candidate] (8.194 ms) : 0, 8194
Flare Poller [baseline] (3.944 ms) : 0, 3944
Flare Poller [candidate] (3.965 ms) : 0, 3965
IAST [baseline] (25.094 ms) : 0, 25094
IAST [candidate] (24.758 ms) : 0, 24758
section iast
crashtracking [baseline] (1.468 ms) : 0, 1468
crashtracking [candidate] (1.465 ms) : 0, 1465
BytebuddyAgent [baseline] (842.007 ms) : 0, 842007
BytebuddyAgent [candidate] (834.462 ms) : 0, 834462
GlobalTracer [baseline] (239.27 ms) : 0, 239270
GlobalTracer [candidate] (238.224 ms) : 0, 238224
AppSec [baseline] (33.138 ms) : 0, 33138
AppSec [candidate] (34.066 ms) : 0, 34066
Debugger [baseline] (60.732 ms) : 0, 60732
Debugger [candidate] (60.669 ms) : 0, 60669
Remote Config [baseline] (548.993 µs) : 0, 549
Remote Config [candidate] (536.818 µs) : 0, 537
Telemetry [baseline] (7.576 ms) : 0, 7576
Telemetry [candidate] (7.629 ms) : 0, 7629
Flare Poller [baseline] (3.474 ms) : 0, 3474
Flare Poller [candidate] (3.481 ms) : 0, 3481
IAST [baseline] (28.63 ms) : 0, 28630
IAST [candidate] (27.692 ms) : 0, 27692
section profiling
crashtracking [baseline] (1.447 ms) : 0, 1447
crashtracking [candidate] (1.445 ms) : 0, 1445
BytebuddyAgent [baseline] (735.59 ms) : 0, 735590
BytebuddyAgent [candidate] (735.478 ms) : 0, 735478
GlobalTracer [baseline] (223.388 ms) : 0, 223388
GlobalTracer [candidate] (223.229 ms) : 0, 223229
AppSec [baseline] (32.229 ms) : 0, 32229
AppSec [candidate] (32.326 ms) : 0, 32326
Debugger [baseline] (63.081 ms) : 0, 63081
Debugger [candidate] (63.257 ms) : 0, 63257
Remote Config [baseline] (674.202 µs) : 0, 674
Remote Config [candidate] (650.602 µs) : 0, 651
Telemetry [baseline] (8.051 ms) : 0, 8051
Telemetry [candidate] (7.971 ms) : 0, 7971
Flare Poller [baseline] (3.805 ms) : 0, 3805
Flare Poller [candidate] (3.806 ms) : 0, 3806
ProfilingAgent [baseline] (96.819 ms) : 0, 96819
ProfilingAgent [candidate] (96.801 ms) : 0, 96801
Profiling [baseline] (97.41 ms) : 0, 97410
Profiling [candidate] (97.383 ms) : 0, 97383

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bdu/avoids-need-for-JVM-environment-variables
git_commit_date	1763657906	1763658312
git_commit_sha	`533755a`	`e5d8ca2`
release_version	1.57.0-SNAPSHOT~533755a5d1	1.56.0-SNAPSHOT~e5d8ca24c1

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1763661856	1763661856
ci_job_id	1248378103	1248378103
ci_pipeline_id	83420718	83420718
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-7emukpvl 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-7emukpvl 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 2 performance regressions! Performance is the same for 17 metrics, 16 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load	worse [+59.998µs; +248.397µs] or [+2.316%; +9.587%]	unstable [+66.516µs; +990.819µs] or [+0.912%; +13.588%]	unstable [-186.330op/s; +153.813op/s] or [-14.066%; +11.611%]	2.745ms	7.820ms	1308.469op/s	2.591ms	7.292ms	1324.727op/s
scenario:load:petclinic:code_origins:high_load	worse [+0.703ms; +1.489ms] or [+4.012%; +8.498%]	same [-115.519µs; +1204.564µs] or [-0.396%; +4.125%]	unstable [-40.360op/s; +16.235op/s] or [-15.506%; +6.237%]	18.619ms	29.744ms	248.219op/s	17.523ms	29.200ms	260.281op/s
scenario:load:petclinic:profiling:high_load	better [-1.664ms; -0.689ms] or [-8.439%; -3.492%]	unsure [-2.159ms; -0.404ms] or [-6.913%; -1.295%]	unstable [-12.858op/s; +41.108op/s] or [-5.501%; +17.589%]	18.547ms	29.944ms	247.844op/s	19.724ms	31.225ms	233.719op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~e5d8ca24c1, baseline=1.57.0-SNAPSHOT~533755a5d1
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.185 ms) : 1174, 1196
.   : milestone, 1185,
iast (3.264 ms) : 3218, 3309
.   : milestone, 3264,
iast_FULL (5.825 ms) : 5766, 5884
.   : milestone, 5825,
iast_GLOBAL (3.352 ms) : 3307, 3396
.   : milestone, 3352,
profiling (2.061 ms) : 2043, 2079
.   : milestone, 2061,
tracing (1.874 ms) : 1857, 1890
.   : milestone, 1874,
section candidate
no_agent (1.204 ms) : 1192, 1216
.   : milestone, 1204,
iast (3.287 ms) : 3240, 3333
.   : milestone, 3287,
iast_FULL (5.863 ms) : 5791, 5936
.   : milestone, 5863,
iast_GLOBAL (3.503 ms) : 3453, 3554
.   : milestone, 3503,
profiling (2.014 ms) : 1997, 2031
.   : milestone, 2014,
tracing (1.829 ms) : 1812, 1846
.   : milestone, 1829,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.185 ms [1.174 ms, 1.196 ms]	-
iast	3.264 ms [3.218 ms, 3.309 ms]	2.079 ms (175.4%)
iast_FULL	5.825 ms [5.766 ms, 5.884 ms]	4.64 ms (391.6%)
iast_GLOBAL	3.352 ms [3.307 ms, 3.396 ms]	2.167 ms (182.9%)
profiling	2.061 ms [2.043 ms, 2.079 ms]	875.766 µs (73.9%)
tracing	1.874 ms [1.857 ms, 1.89 ms]	688.669 µs (58.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.204 ms [1.192 ms, 1.216 ms]	-
iast	3.287 ms [3.24 ms, 3.333 ms]	2.083 ms (173.0%)
iast_FULL	5.863 ms [5.791 ms, 5.936 ms]	4.66 ms (387.1%)
iast_GLOBAL	3.503 ms [3.453 ms, 3.554 ms]	2.3 ms (191.0%)
profiling	2.014 ms [1.997 ms, 2.031 ms]	809.853 µs (67.3%)
tracing	1.829 ms [1.812 ms, 1.846 ms]	625.505 µs (52.0%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~e5d8ca24c1, baseline=1.57.0-SNAPSHOT~533755a5d1
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.1 ms) : 18907, 19292
.   : milestone, 19100,
appsec (18.664 ms) : 18472, 18857
.   : milestone, 18664,
code_origins (17.927 ms) : 17745, 18109
.   : milestone, 17927,
iast (18.257 ms) : 18076, 18438
.   : milestone, 18257,
profiling (19.975 ms) : 19775, 20176
.   : milestone, 19975,
tracing (17.527 ms) : 17356, 17698
.   : milestone, 17527,
section candidate
no_agent (17.953 ms) : 17770, 18135
.   : milestone, 17953,
appsec (18.603 ms) : 18414, 18792
.   : milestone, 18603,
code_origins (18.802 ms) : 18615, 18989
.   : milestone, 18802,
iast (17.987 ms) : 17807, 18168
.   : milestone, 17987,
profiling (18.829 ms) : 18641, 19018
.   : milestone, 18829,
tracing (17.585 ms) : 17408, 17762
.   : milestone, 17585,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.1 ms [18.907 ms, 19.292 ms]	-
appsec	18.664 ms [18.472 ms, 18.857 ms]	-435.187 µs (-2.3%)
code_origins	17.927 ms [17.745 ms, 18.109 ms]	-1.173 ms (-6.1%)
iast	18.257 ms [18.076 ms, 18.438 ms]	-842.554 µs (-4.4%)
profiling	19.975 ms [19.775 ms, 20.176 ms]	875.742 µs (4.6%)
tracing	17.527 ms [17.356 ms, 17.698 ms]	-1.573 ms (-8.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	17.953 ms [17.77 ms, 18.135 ms]	-
appsec	18.603 ms [18.414 ms, 18.792 ms]	650.557 µs (3.6%)
code_origins	18.802 ms [18.615 ms, 18.989 ms]	849.858 µs (4.7%)
iast	17.987 ms [17.807 ms, 18.168 ms]	34.931 µs (0.2%)
profiling	18.829 ms [18.641 ms, 19.018 ms]	876.834 µs (4.9%)
tracing	17.585 ms [17.408 ms, 17.762 ms]	-367.04 µs (-2.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bdu/avoids-need-for-JVM-environment-variables
git_commit_date	1763657906	1763658312
git_commit_sha	`533755a`	`e5d8ca2`
release_version	1.57.0-SNAPSHOT~533755a5d1	1.56.0-SNAPSHOT~e5d8ca24c1

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1763661690	1763661690
ci_job_id	1248378104	1248378104
ci_pipeline_id	83420718	83420718
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-h8uinolj 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-h8uinolj 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:dacapo:tomcat:appsec	better [-1.361ms; -1.025ms] or [-37.073%; -27.915%]	2.478ms	3.671ms

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~e5d8ca24c1, baseline=1.57.0-SNAPSHOT~533755a5d1
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.026 s) : 15026000, 15026000
.   : milestone, 15026000,
appsec (14.572 s) : 14572000, 14572000
.   : milestone, 14572000,
iast (18.242 s) : 18242000, 18242000
.   : milestone, 18242000,
iast_GLOBAL (17.998 s) : 17998000, 17998000
.   : milestone, 17998000,
profiling (15.283 s) : 15283000, 15283000
.   : milestone, 15283000,
tracing (14.61 s) : 14610000, 14610000
.   : milestone, 14610000,
section candidate
no_agent (15.777 s) : 15777000, 15777000
.   : milestone, 15777000,
appsec (14.475 s) : 14475000, 14475000
.   : milestone, 14475000,
iast (18.475 s) : 18475000, 18475000
.   : milestone, 18475000,
iast_GLOBAL (18.249 s) : 18249000, 18249000
.   : milestone, 18249000,
profiling (14.915 s) : 14915000, 14915000
.   : milestone, 14915000,
tracing (14.901 s) : 14901000, 14901000
.   : milestone, 14901000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.026 s [15.026 s, 15.026 s]	-
appsec	14.572 s [14.572 s, 14.572 s]	-454.0 ms (-3.0%)
iast	18.242 s [18.242 s, 18.242 s]	3.216 s (21.4%)
iast_GLOBAL	17.998 s [17.998 s, 17.998 s]	2.972 s (19.8%)
profiling	15.283 s [15.283 s, 15.283 s]	257.0 ms (1.7%)
tracing	14.61 s [14.61 s, 14.61 s]	-416.0 ms (-2.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.777 s [15.777 s, 15.777 s]	-
appsec	14.475 s [14.475 s, 14.475 s]	-1.302 s (-8.3%)
iast	18.475 s [18.475 s, 18.475 s]	2.698 s (17.1%)
iast_GLOBAL	18.249 s [18.249 s, 18.249 s]	2.472 s (15.7%)
profiling	14.915 s [14.915 s, 14.915 s]	-862.0 ms (-5.5%)
tracing	14.901 s [14.901 s, 14.901 s]	-876.0 ms (-5.6%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~e5d8ca24c1, baseline=1.57.0-SNAPSHOT~533755a5d1
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (3.671 ms) : 3456, 3886
.   : milestone, 3671,
iast (2.22 ms) : 2156, 2285
.   : milestone, 2220,
iast_GLOBAL (2.259 ms) : 2194, 2324
.   : milestone, 2259,
profiling (2.089 ms) : 2035, 2144
.   : milestone, 2089,
tracing (2.054 ms) : 2003, 2105
.   : milestone, 2054,
section candidate
no_agent (1.479 ms) : 1468, 1491
.   : milestone, 1479,
appsec (2.478 ms) : 2426, 2530
.   : milestone, 2478,
iast (2.214 ms) : 2150, 2279
.   : milestone, 2214,
iast_GLOBAL (2.26 ms) : 2195, 2324
.   : milestone, 2260,
profiling (2.097 ms) : 2043, 2150
.   : milestone, 2097,
tracing (2.051 ms) : 2000, 2102
.   : milestone, 2051,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.476 ms [1.464 ms, 1.487 ms]	-
appsec	3.671 ms [3.456 ms, 3.886 ms]	2.195 ms (148.8%)
iast	2.22 ms [2.156 ms, 2.285 ms]	744.853 µs (50.5%)
iast_GLOBAL	2.259 ms [2.194 ms, 2.324 ms]	783.778 µs (53.1%)
profiling	2.089 ms [2.035 ms, 2.144 ms]	613.765 µs (41.6%)
tracing	2.054 ms [2.003 ms, 2.105 ms]	578.708 µs (39.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.479 ms [1.468 ms, 1.491 ms]	-
appsec	2.478 ms [2.426 ms, 2.53 ms]	998.944 µs (67.5%)
iast	2.214 ms [2.15 ms, 2.279 ms]	735.284 µs (49.7%)
iast_GLOBAL	2.26 ms [2.195 ms, 2.324 ms]	780.529 µs (52.8%)
profiling	2.097 ms [2.043 ms, 2.15 ms]	617.771 µs (41.8%)
tracing	2.051 ms [2.0 ms, 2.102 ms]	571.638 µs (38.6%)

AlexeyKuznetsov-DD · 2025-11-14T13:22:22Z

buildSrc/src/main/kotlin/datadog/gradle/plugin/testJvmConstraints/TestJvmSpec.kt

-      if (javaVersions.isEmpty()) {
-        throw GradleException("No valid JAVA_X_HOME environment variables found.")
+    when (testJvm) {
+      "stable" -> {


Could we rename stable -> latest? or similar? stable is kind of misleading to me, WDYT?

I'm hesitant to that right now, as there were prior discussions on this.

Maybe we can have both, like lastStable :D

AlexeyKuznetsov-DD · 2025-11-14T13:24:26Z

buildSrc/src/main/kotlin/datadog/gradle/plugin/testJvmConstraints/TestJvmSpec.kt

+ * Handles the `testJvm` property to resolve a Java launcher for testing.
+ *
+ * The `testJvm` property can be set via command line or environment variable to specify
+ * which JVM to use for running tests. E.g.
+ *
+ * ```shell
+ * ./gradlew test -DtestJvm=ZULU11
+ * ```


I wish we have TESTING.md with all this useful documentation from this class :)

Exactly, I started the kdoc here, so we can reuse this text for such a file.

buildSrc/src/main/kotlin/datadog/gradle/plugin/testJvmConstraints/TestJvmSpec.kt

sarahchen6 · 2025-11-14T18:14:09Z

Nice!

Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com>

…-JVM-environment-variables

…raint (#9968) * chore: Drop org.gradle.java.installations.fromEnv toolchain restriction * chore: Drop org.gradle.java.installations toolchain restriction for local env, but keep them in CI * chore: Drop org.gradle.java.installations toolchain restriction for local env, but keep them in CI * feat: Allow discovery of testJvm without environment variables * feat: Only rely on Gradle's toolchain to run tests * fix: Auto discover JAVA_xx_HOME variables from the build image * typo: Use `-P` Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com> --------- Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com>

bric3 added 6 commits November 13, 2025 13:45

chore: Drop org.gradle.java.installations.fromEnv toolchain restriction

35a24a3

chore: Drop org.gradle.java.installations toolchain restriction for l…

470c6e7

…ocal env, but keep them in CI

chore: Drop org.gradle.java.installations toolchain restriction for l…

c889363

…ocal env, but keep them in CI

feat: Allow discovery of testJvm without environment variables

1104f19

feat: Only rely on Gradle's toolchain to run tests

a690268

fix: Auto discover JAVA_xx_HOME variables from the build image

501af3d

bric3 requested review from AlexeyKuznetsov-DD and sarahchen6 November 14, 2025 11:45

bric3 changed the base branch from master to bdu/drop-fromEnv-toolchain-restriction November 14, 2025 11:46

AlexeyKuznetsov-DD reviewed Nov 14, 2025

View reviewed changes

AlexeyKuznetsov-DD approved these changes Nov 14, 2025

View reviewed changes

bric3 added type: enhancement Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Nov 14, 2025

bric3 marked this pull request as ready for review November 14, 2025 13:37

bric3 requested review from a team as code owners November 14, 2025 13:37

bric3 requested review from randomanderson and ygree and removed request for a team November 14, 2025 13:37

bric3 mentioned this pull request Nov 14, 2025

Drop fromEnv toolchain restriction for local environment #9950

Merged

sarahchen6 reviewed Nov 14, 2025

View reviewed changes

buildSrc/src/main/kotlin/datadog/gradle/plugin/testJvmConstraints/TestJvmSpec.kt Outdated Show resolved Hide resolved

sarahchen6 approved these changes Nov 14, 2025

View reviewed changes

randomanderson removed their request for review November 17, 2025 18:25

Base automatically changed from bdu/drop-fromEnv-toolchain-restriction to master November 20, 2025 16:58

bric3 and others added 2 commits November 20, 2025 17:59

typo: Use -P

2214cb2

Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com>

Merge remote-tracking branch 'origin/master' into bdu/avoids-need-for…

e5d8ca2

…-JVM-environment-variables

bric3 enabled auto-merge (squash) November 20, 2025 17:05

bric3 merged commit bb3503a into master Nov 20, 2025
537 of 538 checks passed

bric3 deleted the bdu/avoids-need-for-JVM-environment-variables branch November 20, 2025 19:50

github-actions bot added this to the 1.57.0 milestone Nov 20, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Drop the requirement of jvm environment variables for `testJvm` constraint #9968

Drop the requirement of jvm environment variables for `testJvm` constraint #9968

Uh oh!

bric3 commented Nov 14, 2025

Uh oh!

pr-commenter bot commented Nov 14, 2025 •

edited

Loading

Uh oh!

AlexeyKuznetsov-DD Nov 14, 2025

Uh oh!

bric3 Nov 14, 2025

Uh oh!

AlexeyKuznetsov-DD Nov 14, 2025

Uh oh!

bric3 Nov 14, 2025

Uh oh!

Uh oh!

sarahchen6 commented Nov 14, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Drop the requirement of jvm environment variables for testJvm constraint #9968

Drop the requirement of jvm environment variables for testJvm constraint #9968

Uh oh!

Conversation

bric3 commented Nov 14, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

AlexeyKuznetsov-DD Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

bric3 Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

AlexeyKuznetsov-DD Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

bric3 Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sarahchen6 commented Nov 14, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Drop the requirement of jvm environment variables for `testJvm` constraint #9968

Drop the requirement of jvm environment variables for `testJvm` constraint #9968

pr-commenter bot commented Nov 14, 2025 •

edited

Loading