Collections JMH Benchmarks by dougqh · Pull Request #10674 · DataDog/dd-trace-java

dougqh · 2026-02-24T20:46:56Z

What Does This Do

Benchmarks used to demonstrate most performant approach to common tracer programming problems
Each benchmark includes a header comment that includes benchmark results and the conclusions drawn from those results.

Motivation

The benchmarks are intended to serve as a reference for developers contributing to dd-java-agent.
The benchmarks are intended to be a living example that will be updated as new solutions are found / created.

Additional Notes

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

… situations

github-actions · 2026-02-24T20:47:10Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

dougqh · 2026-02-24T20:49:56Z

internal-api/src/jmh/java/datadog/trace/util/CaseInsensitiveMapBenchmark.java

+@Fork(2)
+@Warmup(iterations = 2)
+@Measurement(iterations = 3)
+@Threads(8)


I've chosen to run the benchmarks with 8 threads because I found that the total system impact was often hidden when only using a single thread.

With 8 threads, solutions that perform more allocation tend to fall behind because of stop-the-world pauses caused by garbage collection.

The thinking is that in a real world system, the tracer is often running inside many application threads, so designing for many threads typically makes sense.

dougqh · 2026-02-24T20:50:37Z

internal-api/src/jmh/java/datadog/trace/util/HashingBenchmark.java

- * common situation where the JIT's escape analysis is unable to elide the var-arg array allocation.
+ *
+ *
+ * <ul>


Tweaked this benchmark to match comment to adhere to the same format as the others

pr-commenter · 2026-02-24T21:29:26Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	dougqh/collection-benchmarks
git_commit_date	1773689566	1773689747
git_commit_sha	`e6b242f`	`06f307b`
release_version	1.61.0-SNAPSHOT~e6b242f8b5	1.60.0-SNAPSHOT~06f307bd15

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1773691492	1773691492
ci_job_id	1510411258	1510411258
ci_pipeline_id	102787658	102787658
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-3pb0b4ah 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-3pb0b4ah 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 67 metrics, 4 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~06f307bd15, baseline=1.61.0-SNAPSHOT~e6b242f8b5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.058 s) : 0, 1058473
Total [baseline] (8.819 s) : 0, 8818779
Agent [candidate] (1.058 s) : 0, 1058365
Total [candidate] (8.802 s) : 0, 8802221
section iast
Agent [baseline] (1.227 s) : 0, 1226760
Total [baseline] (9.566 s) : 0, 9565890
Agent [candidate] (1.225 s) : 0, 1225211
Total [candidate] (9.53 s) : 0, 9529871

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.058 s	-
Agent	iast	1.227 s	168.287 ms (15.9%)
Total	tracing	8.819 s	-
Total	iast	9.566 s	747.111 ms (8.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.058 s	-
Agent	iast	1.225 s	166.846 ms (15.8%)
Total	tracing	8.802 s	-
Total	iast	9.53 s	727.65 ms (8.3%)

gantt
    title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~06f307bd15, baseline=1.61.0-SNAPSHOT~e6b242f8b5

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.21 ms) : 0, 1210
crashtracking [candidate] (1.192 ms) : 0, 1192
BytebuddyAgent [baseline] (627.063 ms) : 0, 627063
BytebuddyAgent [candidate] (626.988 ms) : 0, 626988
AgentMeter [baseline] (29.044 ms) : 0, 29044
AgentMeter [candidate] (28.989 ms) : 0, 28989
GlobalTracer [baseline] (256.659 ms) : 0, 256659
GlobalTracer [candidate] (257.146 ms) : 0, 257146
AppSec [baseline] (31.674 ms) : 0, 31674
AppSec [candidate] (31.634 ms) : 0, 31634
Debugger [baseline] (59.459 ms) : 0, 59459
Debugger [candidate] (59.267 ms) : 0, 59267
Remote Config [baseline] (590.245 µs) : 0, 590
Remote Config [candidate] (586.737 µs) : 0, 587
Telemetry [baseline] (8.01 ms) : 0, 8010
Telemetry [candidate] (8.028 ms) : 0, 8028
Flare Poller [baseline] (8.755 ms) : 0, 8755
Flare Poller [candidate] (8.571 ms) : 0, 8571
section iast
crashtracking [baseline] (1.195 ms) : 0, 1195
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (795.052 ms) : 0, 795052
BytebuddyAgent [candidate] (794.838 ms) : 0, 794838
AgentMeter [baseline] (11.321 ms) : 0, 11321
AgentMeter [candidate] (11.285 ms) : 0, 11285
GlobalTracer [baseline] (247.319 ms) : 0, 247319
GlobalTracer [candidate] (247.084 ms) : 0, 247084
AppSec [baseline] (26.523 ms) : 0, 26523
AppSec [candidate] (26.506 ms) : 0, 26506
Debugger [baseline] (70.729 ms) : 0, 70729
Debugger [candidate] (70.146 ms) : 0, 70146
Remote Config [baseline] (536.074 µs) : 0, 536
Remote Config [candidate] (516.212 µs) : 0, 516
Telemetry [baseline] (9.229 ms) : 0, 9229
Telemetry [candidate] (9.136 ms) : 0, 9136
Flare Poller [baseline] (3.364 ms) : 0, 3364
Flare Poller [candidate] (3.285 ms) : 0, 3285
IAST [baseline] (25.338 ms) : 0, 25338
IAST [candidate] (25.282 ms) : 0, 25282

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~06f307bd15, baseline=1.61.0-SNAPSHOT~e6b242f8b5

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.071 s) : 0, 1070744
Total [baseline] (11.002 s) : 0, 11001780
Agent [candidate] (1.063 s) : 0, 1062576
Total [candidate] (11.009 s) : 0, 11009493
section appsec
Agent [baseline] (1.251 s) : 0, 1250861
Total [baseline] (11.196 s) : 0, 11196409
Agent [candidate] (1.253 s) : 0, 1252926
Total [candidate] (11.209 s) : 0, 11209013
section iast
Agent [baseline] (1.227 s) : 0, 1226940
Total [baseline] (11.42 s) : 0, 11420128
Agent [candidate] (1.227 s) : 0, 1226991
Total [candidate] (11.316 s) : 0, 11315794
section profiling
Agent [baseline] (1.181 s) : 0, 1181292
Total [baseline] (11.072 s) : 0, 11071717
Agent [candidate] (1.18 s) : 0, 1179877
Total [candidate] (11.056 s) : 0, 11055936

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.071 s	-
Agent	appsec	1.251 s	180.117 ms (16.8%)
Agent	iast	1.227 s	156.196 ms (14.6%)
Agent	profiling	1.181 s	110.548 ms (10.3%)
Total	tracing	11.002 s	-
Total	appsec	11.196 s	194.628 ms (1.8%)
Total	iast	11.42 s	418.348 ms (3.8%)
Total	profiling	11.072 s	69.937 ms (0.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.063 s	-
Agent	appsec	1.253 s	190.351 ms (17.9%)
Agent	iast	1.227 s	164.415 ms (15.5%)
Agent	profiling	1.18 s	117.301 ms (11.0%)
Total	tracing	11.009 s	-
Total	appsec	11.209 s	199.52 ms (1.8%)
Total	iast	11.316 s	306.3 ms (2.8%)
Total	profiling	11.056 s	46.443 ms (0.4%)

gantt
    title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~06f307bd15, baseline=1.61.0-SNAPSHOT~e6b242f8b5

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.222 ms) : 0, 1222
crashtracking [candidate] (1.197 ms) : 0, 1197
BytebuddyAgent [baseline] (633.044 ms) : 0, 633044
BytebuddyAgent [candidate] (628.925 ms) : 0, 628925
AgentMeter [baseline] (29.412 ms) : 0, 29412
AgentMeter [candidate] (29.133 ms) : 0, 29133
GlobalTracer [baseline] (258.672 ms) : 0, 258672
GlobalTracer [candidate] (257.299 ms) : 0, 257299
AppSec [baseline] (31.888 ms) : 0, 31888
AppSec [candidate] (31.608 ms) : 0, 31608
Debugger [baseline] (60.53 ms) : 0, 60530
Debugger [candidate] (60.184 ms) : 0, 60184
Remote Config [baseline] (598.355 µs) : 0, 598
Remote Config [candidate] (592.042 µs) : 0, 592
Telemetry [baseline] (8.115 ms) : 0, 8115
Telemetry [candidate] (8.034 ms) : 0, 8034
Flare Poller [baseline] (11.14 ms) : 0, 11140
Flare Poller [candidate] (9.563 ms) : 0, 9563
section appsec
crashtracking [baseline] (1.187 ms) : 0, 1187
crashtracking [candidate] (1.205 ms) : 0, 1205
BytebuddyAgent [baseline] (659.969 ms) : 0, 659969
BytebuddyAgent [candidate] (662.217 ms) : 0, 662217
AgentMeter [baseline] (12.119 ms) : 0, 12119
AgentMeter [candidate] (12.063 ms) : 0, 12063
GlobalTracer [baseline] (259.685 ms) : 0, 259685
GlobalTracer [candidate] (259.428 ms) : 0, 259428
IAST [baseline] (24.436 ms) : 0, 24436
IAST [candidate] (24.417 ms) : 0, 24417
AppSec [baseline] (178.091 ms) : 0, 178091
AppSec [candidate] (177.929 ms) : 0, 177929
Debugger [baseline] (66.566 ms) : 0, 66566
Debugger [candidate] (66.639 ms) : 0, 66639
Remote Config [baseline] (612.284 µs) : 0, 612
Remote Config [candidate] (611.989 µs) : 0, 612
Telemetry [baseline] (8.376 ms) : 0, 8376
Telemetry [candidate] (8.436 ms) : 0, 8436
Flare Poller [baseline] (3.633 ms) : 0, 3633
Flare Poller [candidate] (3.683 ms) : 0, 3683
section iast
crashtracking [baseline] (1.184 ms) : 0, 1184
crashtracking [candidate] (1.182 ms) : 0, 1182
BytebuddyAgent [baseline] (794.907 ms) : 0, 794907
BytebuddyAgent [candidate] (795.116 ms) : 0, 795116
AgentMeter [baseline] (11.293 ms) : 0, 11293
AgentMeter [candidate] (11.34 ms) : 0, 11340
GlobalTracer [baseline] (247.31 ms) : 0, 247310
GlobalTracer [candidate] (247.303 ms) : 0, 247303
IAST [baseline] (25.386 ms) : 0, 25386
IAST [candidate] (25.333 ms) : 0, 25333
AppSec [baseline] (26.569 ms) : 0, 26569
AppSec [candidate] (26.519 ms) : 0, 26519
Debugger [baseline] (71.078 ms) : 0, 71078
Debugger [candidate] (71.034 ms) : 0, 71034
Remote Config [baseline] (535.254 µs) : 0, 535
Remote Config [candidate] (527.919 µs) : 0, 528
Telemetry [baseline] (9.264 ms) : 0, 9264
Telemetry [candidate] (9.291 ms) : 0, 9291
Flare Poller [baseline] (3.397 ms) : 0, 3397
Flare Poller [candidate] (3.382 ms) : 0, 3382
section profiling
crashtracking [baseline] (1.168 ms) : 0, 1168
crashtracking [candidate] (1.166 ms) : 0, 1166
BytebuddyAgent [baseline] (681.363 ms) : 0, 681363
BytebuddyAgent [candidate] (680.668 ms) : 0, 680668
AgentMeter [baseline] (8.663 ms) : 0, 8663
AgentMeter [candidate] (8.575 ms) : 0, 8575
GlobalTracer [baseline] (215.515 ms) : 0, 215515
GlobalTracer [candidate] (215.395 ms) : 0, 215395
AppSec [baseline] (32.318 ms) : 0, 32318
AppSec [candidate] (32.428 ms) : 0, 32428
Debugger [baseline] (62.803 ms) : 0, 62803
Debugger [candidate] (66.061 ms) : 0, 66061
Remote Config [baseline] (580.195 µs) : 0, 580
Remote Config [candidate] (573.444 µs) : 0, 573
Telemetry [baseline] (10.03 ms) : 0, 10030
Telemetry [candidate] (7.637 ms) : 0, 7637
Flare Poller [baseline] (4.243 ms) : 0, 4243
Flare Poller [candidate] (3.474 ms) : 0, 3474
ProfilingAgent [baseline] (94.01 ms) : 0, 94010
ProfilingAgent [candidate] (93.344 ms) : 0, 93344
Profiling [baseline] (94.576 ms) : 0, 94576
Profiling [candidate] (93.898 ms) : 0, 93898

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	dougqh/collection-benchmarks
git_commit_date	1773689566	1773689747
git_commit_sha	`e6b242f`	`06f307b`
release_version	1.61.0-SNAPSHOT~e6b242f8b5	1.60.0-SNAPSHOT~06f307bd15

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1773691959	1773691959
ci_job_id	1510411262	1510411262
ci_pipeline_id	102787658	102787658
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-2apy5n9e 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-2apy5n9e 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 1 performance regressions! Performance is the same for 17 metrics, 16 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:petclinic:no_agent:high_load	better [-2.982ms; -1.486ms] or [-15.818%; -7.882%]	unstable [-5.276ms; -1.914ms] or [-16.797%; -6.093%]	unstable [-0.232op/s; +59.669op/s] or [-0.095%; +24.543%]	16.619ms	27.812ms	272.844op/s	18.853ms	31.407ms	243.125op/s
scenario:load:petclinic:tracing:high_load	worse [+0.401ms; +1.933ms] or [+2.264%; +10.919%]	unsure [+0.262ms; +1.813ms] or [+0.901%; +6.241%]	unstable [-37.591op/s; +19.091op/s] or [-14.627%; +7.428%]	18.867ms	30.095ms	247.750op/s	17.700ms	29.057ms	257.000op/s
scenario:load:petclinic:appsec:high_load	better [-1460.918µs; -452.187µs] or [-7.559%; -2.340%]	same [-1433.286µs; +240.573µs] or [-4.686%; +0.786%]	unstable [-17.171op/s; +37.046op/s] or [-7.187%; +15.505%]	18.371ms	29.993ms	248.875op/s	19.328ms	30.589ms	238.938op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~06f307bd15, baseline=1.61.0-SNAPSHOT~e6b242f8b5
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.19 ms) : 1178, 1202
.   : milestone, 1190,
iast (3.141 ms) : 3099, 3183
.   : milestone, 3141,
iast_FULL (5.722 ms) : 5665, 5779
.   : milestone, 5722,
iast_GLOBAL (3.601 ms) : 3534, 3667
.   : milestone, 3601,
profiling (2.17 ms) : 2150, 2190
.   : milestone, 2170,
tracing (1.789 ms) : 1774, 1804
.   : milestone, 1789,
section candidate
no_agent (1.163 ms) : 1152, 1175
.   : milestone, 1163,
iast (3.292 ms) : 3243, 3340
.   : milestone, 3292,
iast_FULL (5.824 ms) : 5765, 5882
.   : milestone, 5824,
iast_GLOBAL (3.612 ms) : 3562, 3662
.   : milestone, 3612,
profiling (2.08 ms) : 2060, 2100
.   : milestone, 2080,
tracing (1.771 ms) : 1757, 1785
.   : milestone, 1771,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.19 ms [1.178 ms, 1.202 ms]	-
iast	3.141 ms [3.099 ms, 3.183 ms]	1.951 ms (163.9%)
iast_FULL	5.722 ms [5.665 ms, 5.779 ms]	4.532 ms (380.8%)
iast_GLOBAL	3.601 ms [3.534 ms, 3.667 ms]	2.411 ms (202.5%)
profiling	2.17 ms [2.15 ms, 2.19 ms]	979.509 µs (82.3%)
tracing	1.789 ms [1.774 ms, 1.804 ms]	598.98 µs (50.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.163 ms [1.152 ms, 1.175 ms]	-
iast	3.292 ms [3.243 ms, 3.34 ms]	2.128 ms (182.9%)
iast_FULL	5.824 ms [5.765 ms, 5.882 ms]	4.66 ms (400.6%)
iast_GLOBAL	3.612 ms [3.562 ms, 3.662 ms]	2.448 ms (210.4%)
profiling	2.08 ms [2.06 ms, 2.1 ms]	916.139 µs (78.7%)
tracing	1.771 ms [1.757 ms, 1.785 ms]	607.645 µs (52.2%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~06f307bd15, baseline=1.61.0-SNAPSHOT~e6b242f8b5
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.197 ms) : 18998, 19396
.   : milestone, 19197,
appsec (19.537 ms) : 19339, 19735
.   : milestone, 19537,
code_origins (17.632 ms) : 17459, 17804
.   : milestone, 17632,
iast (17.54 ms) : 17370, 17710
.   : milestone, 17540,
profiling (18.69 ms) : 18503, 18878
.   : milestone, 18690,
tracing (18.16 ms) : 17981, 18338
.   : milestone, 18160,
section candidate
no_agent (17.095 ms) : 16928, 17263
.   : milestone, 17095,
appsec (18.751 ms) : 18562, 18941
.   : milestone, 18751,
code_origins (17.908 ms) : 17732, 18085
.   : milestone, 17908,
iast (17.947 ms) : 17770, 18124
.   : milestone, 17947,
profiling (18.938 ms) : 18747, 19128
.   : milestone, 18938,
tracing (18.838 ms) : 18648, 19029
.   : milestone, 18838,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.197 ms [18.998 ms, 19.396 ms]	-
appsec	19.537 ms [19.339 ms, 19.735 ms]	339.899 µs (1.8%)
code_origins	17.632 ms [17.459 ms, 17.804 ms]	-1.565 ms (-8.2%)
iast	17.54 ms [17.37 ms, 17.71 ms]	-1.657 ms (-8.6%)
profiling	18.69 ms [18.503 ms, 18.878 ms]	-506.902 µs (-2.6%)
tracing	18.16 ms [17.981 ms, 18.338 ms]	-1.038 ms (-5.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	17.095 ms [16.928 ms, 17.263 ms]	-
appsec	18.751 ms [18.562 ms, 18.941 ms]	1.656 ms (9.7%)
code_origins	17.908 ms [17.732 ms, 18.085 ms]	813.365 µs (4.8%)
iast	17.947 ms [17.77 ms, 18.124 ms]	851.788 µs (5.0%)
profiling	18.938 ms [18.747 ms, 19.128 ms]	1.843 ms (10.8%)
tracing	18.838 ms [18.648 ms, 19.029 ms]	1.743 ms (10.2%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	dougqh/collection-benchmarks
git_commit_date	1773689566	1773689747
git_commit_sha	`e6b242f`	`06f307b`
release_version	1.61.0-SNAPSHOT~e6b242f8b5	1.60.0-SNAPSHOT~06f307bd15

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1773691697	1773691697
ci_job_id	1510411265	1510411265
ci_pipeline_id	102787658	102787658
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-x1nehsuy 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-x1nehsuy 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~06f307bd15, baseline=1.61.0-SNAPSHOT~e6b242f8b5
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.934 s) : 14934000, 14934000
.   : milestone, 14934000,
appsec (14.991 s) : 14991000, 14991000
.   : milestone, 14991000,
iast (18.119 s) : 18119000, 18119000
.   : milestone, 18119000,
iast_GLOBAL (17.944 s) : 17944000, 17944000
.   : milestone, 17944000,
profiling (14.848 s) : 14848000, 14848000
.   : milestone, 14848000,
tracing (15.145 s) : 15145000, 15145000
.   : milestone, 15145000,
section candidate
no_agent (15.004 s) : 15004000, 15004000
.   : milestone, 15004000,
appsec (15.135 s) : 15135000, 15135000
.   : milestone, 15135000,
iast (18.299 s) : 18299000, 18299000
.   : milestone, 18299000,
iast_GLOBAL (17.905 s) : 17905000, 17905000
.   : milestone, 17905000,
profiling (14.596 s) : 14596000, 14596000
.   : milestone, 14596000,
tracing (14.954 s) : 14954000, 14954000
.   : milestone, 14954000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.934 s [14.934 s, 14.934 s]	-
appsec	14.991 s [14.991 s, 14.991 s]	57.0 ms (0.4%)
iast	18.119 s [18.119 s, 18.119 s]	3.185 s (21.3%)
iast_GLOBAL	17.944 s [17.944 s, 17.944 s]	3.01 s (20.2%)
profiling	14.848 s [14.848 s, 14.848 s]	-86.0 ms (-0.6%)
tracing	15.145 s [15.145 s, 15.145 s]	211.0 ms (1.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.004 s [15.004 s, 15.004 s]	-
appsec	15.135 s [15.135 s, 15.135 s]	131.0 ms (0.9%)
iast	18.299 s [18.299 s, 18.299 s]	3.295 s (22.0%)
iast_GLOBAL	17.905 s [17.905 s, 17.905 s]	2.901 s (19.3%)
profiling	14.596 s [14.596 s, 14.596 s]	-408.0 ms (-2.7%)
tracing	14.954 s [14.954 s, 14.954 s]	-50.0 ms (-0.3%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~06f307bd15, baseline=1.61.0-SNAPSHOT~e6b242f8b5
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (2.521 ms) : 2466, 2576
.   : milestone, 2521,
iast (2.257 ms) : 2188, 2326
.   : milestone, 2257,
iast_GLOBAL (2.303 ms) : 2234, 2373
.   : milestone, 2303,
profiling (2.091 ms) : 2037, 2146
.   : milestone, 2091,
tracing (2.059 ms) : 2005, 2112
.   : milestone, 2059,
section candidate
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (3.818 ms) : 3596, 4039
.   : milestone, 3818,
iast (2.25 ms) : 2181, 2319
.   : milestone, 2250,
iast_GLOBAL (2.289 ms) : 2219, 2358
.   : milestone, 2289,
profiling (2.104 ms) : 2048, 2161
.   : milestone, 2104,
tracing (2.066 ms) : 2012, 2119
.   : milestone, 2066,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.476 ms [1.464 ms, 1.487 ms]	-
appsec	2.521 ms [2.466 ms, 2.576 ms]	1.045 ms (70.8%)
iast	2.257 ms [2.188 ms, 2.326 ms]	780.819 µs (52.9%)
iast_GLOBAL	2.303 ms [2.234 ms, 2.373 ms]	827.506 µs (56.1%)
profiling	2.091 ms [2.037 ms, 2.146 ms]	615.465 µs (41.7%)
tracing	2.059 ms [2.005 ms, 2.112 ms]	583.04 µs (39.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.46 ms, 1.483 ms]	-
appsec	3.818 ms [3.596 ms, 4.039 ms]	2.346 ms (159.4%)
iast	2.25 ms [2.181 ms, 2.319 ms]	778.449 µs (52.9%)
iast_GLOBAL	2.289 ms [2.219 ms, 2.358 ms]	816.74 µs (55.5%)
profiling	2.104 ms [2.048 ms, 2.161 ms]	632.53 µs (43.0%)
tracing	2.066 ms [2.012 ms, 2.119 ms]	593.741 µs (40.3%)

amarziali

thanks. That would help for future choices. I believe this should be documented but I'm not sure we already have a good place for that kind of docs (could be helpful having a dedicated md page for that kind of perf things)

dd-octo-sts · 2026-03-10T13:46:28Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-03-10T13:46:32Z

View all feedbacks in Devflow UI.

2026-03-10 13:46:32 UTC ℹ️ Start processing command /merge

2026-03-10 13:46:37 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

2026-03-10 15:42:03 UTC ℹ️ MergeQueue: Readding this merge request to the queue because another merge request processed with yours failed. No action is needed from your side.

2026-03-10 15:45:38 UTC ❌ MergeQueue: This merge request was updated

This PR is rejected because it was updated

dd-octo-sts · 2026-03-10T18:07:51Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-03-10T18:07:56Z

View all feedbacks in Devflow UI.

2026-03-10 18:07:55 UTC ℹ️ Start processing command /merge

2026-03-10 18:07:59 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

2026-03-10 19:12:50 UTC ❌ MergeQueue: This merge request was updated

This PR is rejected because it was updated

dd-octo-sts · 2026-03-12T00:02:48Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-03-12T00:02:52Z

View all feedbacks in Devflow UI.

2026-03-12 00:02:52 UTC ℹ️ Start processing command /merge

2026-03-12 00:02:56 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

2026-03-12 02:03:59 UTC ❌ MergeQueue: The build pipeline has timeout

The merge request has been interrupted because the build 102047087 took longer than expected. The current limit for the base branch 'master' is 120 minutes.

dd-octo-sts · 2026-03-13T11:29:16Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-03-13T11:29:20Z

View all feedbacks in Devflow UI.

2026-03-13 11:29:20 UTC ℹ️ Start processing command /merge

2026-03-13 11:29:25 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

2026-03-13 13:59:57 UTC ❌ MergeQueue: The build pipeline has timeout

The merge request has been interrupted because the build 102345410 took longer than expected. The current limit for the base branch 'master' is 120 minutes.

bric3 · 2026-03-13T12:44:41Z

Not sure why check_base fails only in the devflow/merge pipeline, and not in this PR pipeline. The error on the devflow pipeline:

> Task :internal-api:compileJmhJava FAILED
.../dd-trace-java/internal-api/src/jmh/java/datadog/trace/util/UnsynchronizedMapBenchmark.java:3: error: package datadog.collections.tagmap11_prototype2 does not exist
import datadog.collections.tagmap11_prototype2.TagMap;
                                              ^
...dd-trace-java/internal-api/src/jmh/java/datadog/trace/util/UnsynchronizedMapBenchmark.java:251: error: cannot find symbol
  static TagMap _create_tagMap() {
         ^
  symbol:   class TagMap
  location: class datadog.trace.util.UnsynchronizedMapBenchmark
.../dd-trace-java/internal-api/src/jmh/java/datadog/trace/util/UnsynchronizedMapBenchmark.java:260: error: cannot find symbol
  public TagMap create_tagMap() {
         ^
  symbol:   class TagMap
  location: class datadog.trace.util.UnsynchronizedMapBenchmark
.../dd-trace-java/internal-api/src/jmh/java/datadog/trace/util/UnsynchronizedMapBenchmark.java:265: error: cannot find symbol
  public TagMap create_tagMap_via_ledger() {
         ^
  symbol:   class TagMap
  location: class datadog.trace.util.UnsynchronizedMapBenchmark
.../dd-trace-java/internal-api/src/jmh/java/datadog/trace/util/UnsynchronizedMapBenchmark.java:273: error: cannot find symbol
  static final TagMap TAG_MAP = _create_tagMap();
               ^
  symbol:   class TagMap
  location: class datadog.trace.util.UnsynchronizedMapBenchmark
.../dd-trace-java/internal-api/src/jmh/java/datadog/trace/util/UnsynchronizedMapBenchmark.java:305: error: cannot find symbol
  public TagMap clone_tagMap() {
         ^
  symbol:   class TagMap
  location: class datadog.trace.util.UnsynchronizedMapBenchmark
.../dd-trace-java/internal-api/src/jmh/java/datadog/trace/util/UnsynchronizedMapBenchmark.java:252: error: cannot find symbol
    TagMap map = TagMap.create();
    ^
  symbol:   class TagMap
  location: class datadog.trace.util.UnsynchronizedMapBenchmark
.../dd-trace-java/internal-api/src/jmh/java/datadog/trace/util/UnsynchronizedMapBenchmark.java:252: error: cannot find symbol
    TagMap map = TagMap.create();
                 ^
  symbol:   variable TagMap
  location: class datadog.trace.util.UnsynchronizedMapBenchmark
.../dd-trace-java/internal-api/src/jmh/java/datadog/trace/util/UnsynchronizedMapBenchmark.java:266: error: package TagMap does not exist
    TagMap.Ledger ledger = TagMap.ledger();
          ^
.../dd-trace-java/internal-api/src/jmh/java/datadog/trace/util/UnsynchronizedMapBenchmark.java:266: error: cannot find symbol
    TagMap.Ledger ledger = TagMap.ledger();
                           ^
  symbol:   variable TagMap
  location: class datadog.trace.util.UnsynchronizedMapBenchmark
.../dd-trace-java/internal-api/src/jmh/java/datadog/trace/util/UnsynchronizedMapBenchmark.java:287: error: package TagMap does not exist
    for (TagMap.EntryReader entry : TAG_MAP) {
               ^
11 errors
> Task :components:environment:spotbugsMain
> Task :telemetry:spotbugsMain
> Task :components:json:spotbugsMain
> Task :dd-trace-core:compileTestGroovy
> Task :dd-java-agent:shadowJar
[Incubating] Problems report is available at: file:///.../dd-trace-java/workspace/build/reports/problems/problems-report.html
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':internal-api:compileJmhJava'.
> Compilation failed; see the compiler output below.
  .../dd-trace-java/internal-api/src/jmh/java/datadog/trace/util/UnsynchronizedMapBenchmark.java:3: error: package datadog.collections.tagmap11_prototype2 does not exist
  import datadog.collections.tagmap11_prototype2.TagMap;
                                                ^
  11 errors

dougqh · 2026-03-13T19:24:25Z

Not sure why check_base fails only in the devflow/merge pipeline, and not in this PR pipeline. The error on the devflow pipeline:

That's weird. I'll look into it.
I'll admit that I had some copy & paste mistakes previously, because I often develop my prototypes & benchmarks in a stand-alone project.

bric3 · 2026-03-16T09:20:56Z

@dougqh I think the only issue is with datadog.collections.tagmap11_prototype2.TagMap. Should it be replaced by the existing TagMap ?

Btw the compilation task is ./gradlew :internal-api:compileJmhJava

dougqh · 2026-03-16T13:40:40Z

@dougqh I think the only issue is with datadog.collections.tagmap11_prototype2.TagMap. Should it be replaced by the existing TagMap ?

Btw the compilation task is ./gradlew :internal-api:compileJmhJava

Thanks, I'll take care of that.

dd-octo-sts · 2026-03-17T09:13:26Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-03-17T09:13:30Z

View all feedbacks in Devflow UI.

2026-03-17 09:13:30 UTC ℹ️ Start processing command /merge

2026-03-17 09:13:34 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 3h (p90).

2026-03-17 10:13:09 UTC ℹ️ MergeQueue: This merge request was merged

dougqh added 2 commits February 24, 2026 15:43

Adding benchmarks to illustrate recommended approach to common tracer…

73b19dd

… situations

spotless

bba3921

dougqh requested a review from a team as a code owner February 24, 2026 20:46

dougqh requested a review from amarziali February 24, 2026 20:46

dougqh commented Feb 24, 2026

View reviewed changes

Merge branch 'master' into dougqh/collection-benchmarks

53901eb

dougqh added comp: core Tracer core tag: performance Performance related changes tag: diagnostics Diagnostics related changes type: enhancement Enhancements and improvements labels Feb 26, 2026

Merge branch 'master' into dougqh/collection-benchmarks

cf3b9cb

amarziali approved these changes Mar 2, 2026

View reviewed changes

bric3 changed the title ~~Collection Benchmarks~~ Collection of Benchmarks Mar 2, 2026

bric3 changed the title ~~Collection of Benchmarks~~ Collection of JMH Benchmarks Mar 2, 2026

bric3 changed the title ~~Collection of JMH Benchmarks~~ Collections JMH Benchmarks Mar 3, 2026

dougqh added this pull request to the merge queue Mar 10, 2026

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 10, 2026

Merge branch 'master' into dougqh/collection-benchmarks

707e4e2

dougqh added this pull request to the merge queue Mar 10, 2026

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 10, 2026

Merge branch 'master' into dougqh/collection-benchmarks

43f7eaf

dougqh enabled auto-merge March 10, 2026 19:12

Merge branch 'master' into dougqh/collection-benchmarks

42b4b7e

Merge branch 'master' into dougqh/collection-benchmarks

8829765

dougqh enabled auto-merge March 11, 2026 23:12

dougqh added this pull request to the merge queue Mar 12, 2026

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 12, 2026

Merge branch 'master' into dougqh/collection-benchmarks

ac7b262

dougqh enabled auto-merge March 12, 2026 11:44

Merge branch 'master' into dougqh/collection-benchmarks

a9eaa92

dougqh added this pull request to the merge queue Mar 13, 2026

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 13, 2026

Merge branch 'master' into dougqh/collection-benchmarks

dfb3ee9

bric3 added the tag: no release notes Changes to exclude from release notes label Mar 16, 2026

dougqh added 2 commits March 16, 2026 15:12

Fixing copy paste error from stand-alone TagMap prototyping project

96eb551

Merge branch 'master' into dougqh/collection-benchmarks

982a58b

dougqh enabled auto-merge March 16, 2026 19:20

Merge branch 'master' into dougqh/collection-benchmarks

06f307b

dougqh added this pull request to the merge queue Mar 17, 2026

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 17, 2026

gh-worker-dd-mergequeue-cf854d bot merged commit fc35e2e into master Mar 17, 2026
748 of 750 checks passed

gh-worker-dd-mergequeue-cf854d bot deleted the dougqh/collection-benchmarks branch March 17, 2026 10:13

github-actions bot added this to the 1.61.0 milestone Mar 17, 2026

Conversation

dougqh commented Feb 24, 2026

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

github-actions bot commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dougqh Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

dougqh Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

pr-commenter bot commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

amarziali left a comment

Choose a reason for hiding this comment

Uh oh!

dd-octo-sts bot commented Mar 10, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 bot commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

dd-octo-sts bot commented Mar 10, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 bot commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

dd-octo-sts bot commented Mar 12, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 bot commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

dd-octo-sts bot commented Mar 13, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 bot commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

bric3 commented Mar 13, 2026

Uh oh!

dougqh commented Mar 13, 2026

Uh oh!

bric3 commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dougqh commented Mar 16, 2026

Uh oh!

dd-octo-sts bot commented Mar 17, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Feb 24, 2026 •

edited

Loading

pr-commenter bot commented Feb 24, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 bot commented Mar 10, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 bot commented Mar 10, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 bot commented Mar 12, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 bot commented Mar 13, 2026 •

edited

Loading

bric3 commented Mar 16, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 bot commented Mar 17, 2026 •

edited

Loading