Wire -Pjmh.includes and -PtestJvm into internal-api JMH config by dougqh · Pull Request #11703 · DataDog/dd-trace-java

dougqh · 2026-06-22T23:58:16Z

What This Does

Wires two previously-ignored Gradle properties into the internal-api JMH configuration:

-Pjmh.includes=<pattern> — filters which benchmarks run. Previously silently ignored by the me.champeau.jmh plugin, requiring a full fat-jar build to run a single benchmark.
-PtestJvm=<version> — selects the JVM used for benchmark execution. Previously ignored, defaulting to whatever JVM the Gradle daemon happened to use.

Usage:

./gradlew :internal-api:jmh -Pjmh.includes=AtomicsBenchmark -PtestJvm=17

Motivation

Running a single benchmark without this fix required building the full fat-jar and invoking it manually. The JVM defaulted to the daemon JVM rather than the requested version, making results non-reproducible across machines.

Additional Notes

The same pattern applies to other modules that use me.champeau.jmh (there are ~10 others). This PR fixes internal-api where current benchmark development is happening; other modules can adopt the pattern as needed.

tag: no release note

🤖 Generated with Claude Code

Without this, -Pjmh.includes is silently ignored by the me.champeau.jmh plugin, requiring a full fat-jar build to run a single benchmark. -PtestJvm was also ignored for JMH execution, defaulting to the Gradle daemon JVM regardless of the requested version. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

dd-octo-sts · 2026-06-22T23:58:26Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

dd-octo-sts · 2026-06-23T00:25:00Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.91 s	13.93 s	[-0.8%; +0.6%] (no difference)
startup:insecure-bank:tracing:Agent	12.83 s	12.98 s	[-1.8%; -0.5%] (maybe better)
startup:petclinic:appsec:Agent	16.94 s	16.69 s	[+0.7%; +2.4%] (maybe worse)
startup:petclinic:iast:Agent	16.84 s	17.01 s	[-1.9%; -0.1%] (maybe better)
startup:petclinic:profiling:Agent	16.91 s	16.80 s	[-0.4%; +1.7%] (no difference)
startup:petclinic:sca:Agent	16.94 s	16.67 s	[+0.6%; +2.6%] (maybe worse)
startup:petclinic:tracing:Agent	16.05 s	16.26 s	[-2.4%; -0.1%] (maybe better)

Commit: 2845c650 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

dd-octo-sts · 2026-06-23T18:06:30Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-06-23T18:06:34Z

View all feedbacks in Devflow UI.

2026-06-23 18:06:34 UTC ℹ️ Start processing command /merge

2026-06-23 18:06:39 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

2026-06-23 19:15:24 UTC ℹ️ MergeQueue: This merge request was merged

The internal-api/build.gradle.kts -Pjmh.includes / -PtestJvm wiring belongs in its dedicated PR (#11703), not here. Revert it so this PR is just the benchmark. Also correct the results header to "MacBook M1 Pro Max". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Align TagMapAccessBenchmark with UnsynchronizedMapBenchmark at @fork(2) for steadier numbers (results to be refreshed on the next run). Also revert the internal-api/build.gradle.kts -Pjmh.includes / -PtestJvm wiring, which belongs in its dedicated PR (#11703), not here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Atomics benchmark Benchmark to show cost of Atomic vs AtomicFieldUpdater Merge branch 'master' into dougqh/atomics-benchmark spotless Merge branch 'master' into dougqh/atomics-benchmark Update internal-api/src/jmh/java/datadog/trace/util/AtomicsBenchmark.java Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com> Fix data race in AtomicsBenchmark scaffolding index sharedLookupIndex was a plain static int incremented by all 8 JMH threads without synchronization — a data race that could cause contention measurements to reflect arbitrary holder access patterns. Move index to @State(Scope.Thread) so each thread has its own cursor. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Wire -Pjmh.includes and -PtestJvm into internal-api JMH config Without this, -Pjmh.includes is silently ignored by the me.champeau.jmh plugin, requiring a full fat-jar build to run a single benchmark. -PtestJvm was also ignored for JMH execution, defaulting to the Gradle daemon JVM regardless of the requested version. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Update AtomicsBenchmark results with corrected Java 17 numbers Re-run after fixing the shared-index data race. The incrementAndGet results increased dramatically (~16M → ~91-121M ops/s) because the prior run was measuring contention on the shared index rather than the atomics themselves. Notable: atomic_incrementAndGet (121M) now outperforms atomicFieldUpdater_incrementAndGet (91M) for the increment op; the AFU advantage is in construction (2215M vs 1872M, saving 16 B/op). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Clarify AtomicsBenchmark recommendation: tradeoff depends on access pattern AFU wins on construction (16 B vs 32 B/op); AtomicInteger wins on incrementAndGet (~33% faster). Updated Javadoc to present both sides rather than a blanket recommendation. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Add VarHandle note to AtomicsBenchmark Javadoc VarHandle (Java 9+) is the modern replacement for AtomicIntegerFieldUpdater and would likely close the incrementAndGet performance gap. Not usable here because internal-api targets Java 8. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Merge branch 'master' into dougqh/atomics-benchmark Drop JMH gradle wiring from atomics PR; fix results header machine name The internal-api/build.gradle.kts -Pjmh.includes / -PtestJvm wiring belongs in its dedicated PR (#11703), not here. Revert it so this PR is just the benchmark. Also correct the results header to "MacBook M1 Pro Max". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>

dougqh requested a review from a team as a code owner June 22, 2026 23:58

dougqh requested a review from mhlidd June 22, 2026 23:58

dd-octo-sts Bot added the tag: ai generated Largely based on code generated by an AI or LLM label Jun 22, 2026

This comment has been minimized.

Sign in to view

mhlidd approved these changes Jun 23, 2026

View reviewed changes

dougqh added type: enhancement Enhancements and improvements comp: core Tracer core labels Jun 23, 2026

Merge branch 'master' into dougqh/jmh-gradle-improvements

2845c65

dougqh mentioned this pull request Jun 23, 2026

Atomics benchmark #10862

Merged

dougqh added this pull request to the merge queue Jun 23, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 23, 2026

gh-worker-dd-mergequeue-cf854d Bot merged commit e500e6a into master Jun 23, 2026
583 checks passed

gh-worker-dd-mergequeue-cf854d Bot deleted the dougqh/jmh-gradle-improvements branch June 23, 2026 19:15

github-actions Bot added this to the 1.64.0 milestone Jun 23, 2026

dougqh mentioned this pull request Jun 23, 2026

Overhaul map JMH benchmarks: remove thread contention and split by use case #11679

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wire -Pjmh.includes and -PtestJvm into internal-api JMH config#11703

Wire -Pjmh.includes and -PtestJvm into internal-api JMH config#11703
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits into
masterfrom
dougqh/jmh-gradle-improvements

dougqh commented Jun 22, 2026

Uh oh!

dd-octo-sts Bot commented Jun 22, 2026 •

edited

Loading

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Jun 23, 2026 •

edited

Loading

Uh oh!

dd-octo-sts Bot commented Jun 23, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Jun 23, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

dougqh commented Jun 22, 2026

What This Does

Motivation

Additional Notes

Uh oh!

dd-octo-sts Bot commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

dd-octo-sts Bot commented Jun 23, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dd-octo-sts Bot commented Jun 22, 2026 •

edited

Loading

dd-octo-sts Bot commented Jun 23, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented Jun 23, 2026 •

edited

Loading