Skip to content

fix(profiler): guard line-number-table copy against stale jmethodID#647

Merged
jbachorik merged 13 commits into
mainfrom
jb/resolve_method
Jul 14, 2026
Merged

fix(profiler): guard line-number-table copy against stale jmethodID#647
jbachorik merged 13 commits into
mainfrom
jb/resolve_method

Conversation

@jbachorik

@jbachorik jbachorik commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?:
Guards the JVMTI line-number-table copy in Lookup::fillJavaMethodInfo (flightRecorder.cpp) with SafeAccess::isReadableRange() before memcpy-ing it, instead of dereferencing the pointer unconditionally. Also bounds line_number_table_size before it's trusted to compute a byte range: sizes above MAX_LINE_NUMBER_TABLE_ENTRIES (65535, the JVM spec's max code_length) or negative are rejected outright, since a corrupted pointer from a stale jmethodID is just as likely to be paired with a corrupted size. Adds a LINE_NUMBER_TABLE_UNREADABLE counter for observability when either guard trips.

Motivation:

Production crash report: SIGSEGV in __memcpy_evex_unaligned_erms, inside Lookup::resolveMethod (inlined fillJavaMethodInfo), during FlightRecorder::dump. Confirmed on stock HotSpot JDK 11.

fillJavaMethodInfo calls GetLineNumberTable() on a jmethodID captured asynchronously at sample time, which can point at an already-unloaded class by the time the dump thread processes it (the TOCTOU race already documented a few lines above this call, re: GetMethodDeclaringClass). The sibling class_name/method_name/method_sig strings returned by the preceding JVMTI calls are already probed with SafeAccess::isReadableRange for exactly this reason (#537 / ef13aa29f), but the line-number-table pointer and its paired size were copied/trusted unguarded. This PR closes that gap.

Additional Notes:

  • Reproducer (lineNumberTableCopy_ut.cpp) isolates the copy step from JVMTI/JNI (impractical to fake a live JVM in a plain gtest) by exercising the identical copy pattern against a real mmap'd-then-munmap'd page, standing in for "GetLineNumberTable() returned a bad pointer" regardless of the precise reason. Verified independently outside gtest too: the unguarded pattern segfaults (exit 139) and the guarded pattern exits cleanly.
  • Also adds JMethodIDInvalidationStressTest.java, a JUnit stress test that races class-unload churn (via disposable classloaders + a dedicated GC-pressure thread) against concurrent dump()/resolveMethod() calls, and asserts via native whitebox counters that a stale jmethodID was actually observed and safely guarded during the run — not just that the JVM didn't crash. The counter-delta check is a JUnit assumption (skip, not fail) since hitting the race within the window is host/JVM-discretionary.
  • This branch also carries an unrelate addition to AGENTS.md (a short "CI / Automation Script Changes" guardrails section) picked up from other work on this branch; it is not part of the PROF-15385 fix itself but the gap was identified during the reviews and the addition is cheap/small enough not to disturb the main PR.

How to test the change?:

  • ./gradlew :ddprof-lib:gtestDebug_lineNumberTableCopy_ut — 3 tests: unguarded pattern crashes as expected (EXPECT_DEATH), guarded pattern skips the copy safely when the source is unmapped/oversized, guarded pattern still copies correctly for valid data.
  • ./gradlew testDebug -Ptests=JMethodIDInvalidationStressTest — stress test racing class-unload churn against profiler dump/resolve.

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles
    credentials of any kind, I've requested a security review (run the dd:platform-security-review
    skill, or file a request via the PSEC review form).
    bewaire also runs automatically on every PR.
  • This PR doesn't touch any of that.
  • JIRA: PROF-15385

Unsure? Have a question? Request a review!

@jbachorik jbachorik added the AI label Jul 10, 2026
@dd-octo-sts

dd-octo-sts Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #29345212506 | Commit: 0f996f3 | Duration: 17m 10s (longest job)

All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - - -
8-ibm - - -
8-j9 - -
8-librca - -
8-orcl - - -
11 - - -
11-j9 - -
11-librca - -
17 - -
17-graal - -
17-j9 - -
17-librca - -
21 - -
21-graal - -
21-librca - -
25 - -
25-graal - -
25-librca - -

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-07-14 15:44:48 UTC

@dd-octo-sts

dd-octo-sts Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 36f4468)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124068415 Commit: 36f4468c5b43f9d55e01d3b6645365fc3bfb46f2

⚠️ Significant outliers

  • 🟢 fj-kmeans (JDK 21): runtime -4% (2825→2711 ms)
  • 🟢 future-genetic (JDK 25): runtime -4.7% (2126→2027 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10326 ms (21 iters) ✅ 10289 ms (21 iters) ≈ -0.4% (±11.2%) — / —
akka-uct 25 ✅ 8965 ms (24 iters) ✅ 8824 ms (24 iters) ≈ -1.6% (±9.7%) — / —
finagle-chirper 21 ✅ 6017 ms (33 iters) ✅ 6003 ms (33 iters) ≈ -0.2% (±25.4%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5479 ms (36 iters) ✅ 5525 ms (36 iters) ≈ +0.8% (±24.9%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2825 ms (66 iters) ✅ 2711 ms (69 iters) 🟢 -4% — / —
fj-kmeans 25 ✅ 2813 ms (66 iters) ✅ 2820 ms (66 iters) ≈ +0.2% (±2.6%) — / —
future-genetic 25 ✅ 2126 ms (87 iters) ✅ 2027 ms (91 iters) 🟢 -4.7% — / —
naive-bayes 21 ✅ 1264 ms (135 iters) ✅ 1289 ms (133 iters) ≈ +2% (±33.3%) — / —
naive-bayes 25 ✅ 1009 ms (169 iters) ✅ 1018 ms (168 iters) ≈ +0.9% (±32.2%) — / —
reactors 21 ✅ 16193 ms (15 iters) ✅ 16743 ms (15 iters) ≈ +3.4% (±7.3%) — / —
reactors 25 ✅ 18502 ms (15 iters) ✅ 18476 ms (15 iters) ≈ -0.1% (±4.8%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 1 / 1 1952 / 1969 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 1 / 1 2077 / 2368 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 3 / 4 8413 / 8843 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ ✅ / 1 8590 / 8627 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 1 / 1 1261 / 1237 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 1 / 4 2893 / 2959 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 2 / 1 2868 / 2862 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 1 / 2 3473 / 3538 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 2 / 5 3507 / 3483 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ 1 / 2 1661 / 1582 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 1975 / 1867 ✅ / ✅ ✅ / ✅

@datadog-datadog-us1-prod

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit e5ef83e)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124087303 Commit: e5ef83e27614101e69defb0c599585f0e28b1ebd

⚠️ Significant outliers

  • 🔴 future-genetic (JDK 21): runtime +4% (2048→2129 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10327 ms (21 iters) ✅ 10396 ms (21 iters) ≈ +0.7% (±10.9%) — / —
akka-uct 25 ✅ 8806 ms (24 iters) ✅ 8888 ms (24 iters) ≈ +0.9% (±11%) — / —
finagle-chirper 21 ✅ 5982 ms (33 iters) ✅ 6066 ms (33 iters) ≈ +1.4% (±25.2%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5471 ms (36 iters) ✅ 5502 ms (36 iters) ≈ +0.6% (±24.4%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2715 ms (68 iters) ✅ 2762 ms (68 iters) ≈ +1.7% (±2.7%) — / —
fj-kmeans 25 ✅ 2849 ms (66 iters) ✅ 2840 ms (66 iters) ≈ -0.3% (±2.6%) — / —
future-genetic 21 ✅ 2048 ms (91 iters) ✅ 2129 ms (87 iters) 🔴 +4% — / —
future-genetic 25 ✅ 2085 ms (89 iters) ✅ 2033 ms (92 iters) ≈ -2.5% (±2.6%) — / —
naive-bayes 21 ✅ 1269 ms (135 iters) ✅ 1265 ms (135 iters) ≈ -0.3% (±32.8%) — / —
naive-bayes 25 ✅ 1004 ms (170 iters) ✅ 1017 ms (168 iters) ≈ +1.3% (±31.7%) — / —
reactors 21 ✅ 16602 ms (15 iters) ✅ 15968 ms (15 iters) ≈ -3.8% (±7.8%) — / —
reactors 25 ✅ 18322 ms (15 iters) ✅ 18907 ms (15 iters) ≈ +3.2% (±4.6%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 4 / 4 2184 / 2291 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 4 / 2 8712 / 8658 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ ✅ / ✅ 8439 / 8434 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 1 / 1 1239 / 1267 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 2 / 3 1288 / 1274 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 1 / 1 2973 / 2888 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 2 / 1 2926 / 2837 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 11 / 3 3532 / 3526 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 6 / 3 3456 / 3462 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ 1 / 2 1715 / 1428 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ ✅ / 1 1909 / 1913 ✅ / ✅ ✅ / ✅

@jbachorik

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e5ef83e276

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ddprof-lib/src/main/cpp/flightRecorder.cpp Outdated
@dd-octo-sts

dd-octo-sts Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Reliability & Chaos Results

All reliability & chaos checks passed Pipeline: https://gitlab.ddbuild.io/DataDog/java-profiler/-/pipelines/124389630

@dd-octo-sts

dd-octo-sts Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 30240b5)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124264493 Commit: 30240b542b872a175b8d5278f225769d4e436e28

⚠️ Significant outliers

  • 🔴 fj-kmeans (JDK 21): runtime +5.7% (2700→2855 ms)
  • 🔴 future-genetic (JDK 25): runtime +2.7% (2064→2119 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10246 ms (21 iters) ✅ 10423 ms (21 iters) ≈ +1.7% (±11%) — / —
akka-uct 25 ✅ 8914 ms (24 iters) ✅ 8770 ms (24 iters) ≈ -1.6% (±9.5%) — / —
finagle-chirper 21 ✅ 5984 ms (33 iters) ✅ 5971 ms (33 iters) ≈ -0.2% (±25.6%) ⚠️ W:4 / ⚠️ W:3
finagle-chirper 25 ✅ 5551 ms (36 iters) ✅ 5523 ms (36 iters) ≈ -0.5% (±24.4%) ⚠️ W:3 / ⚠️ W:4
fj-kmeans 21 ✅ 2700 ms (70 iters) ✅ 2855 ms (66 iters) 🔴 +5.7% — / —
fj-kmeans 25 ✅ 2833 ms (66 iters) ✅ 2827 ms (66 iters) ≈ -0.2% (±2.6%) — / —
future-genetic 21 ✅ 2125 ms (87 iters) ✅ 2090 ms (90 iters) ≈ -1.6% (±2.8%) — / —
future-genetic 25 ✅ 2064 ms (90 iters) ✅ 2119 ms (88 iters) 🔴 +2.7% — / —
naive-bayes 21 ✅ 1253 ms (137 iters) ✅ 1270 ms (134 iters) ≈ +1.4% (±33.4%) — / —
naive-bayes 25 ✅ 1015 ms (169 iters) ✅ 1023 ms (168 iters) ≈ +0.8% (±31.7%) — / —
reactors 21 ✅ 16578 ms (15 iters) ✅ 16507 ms (15 iters) ≈ -0.4% (±7.5%) — / —
reactors 25 ✅ 18580 ms (15 iters) ✅ 18707 ms (15 iters) ≈ +0.7% (±4.7%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 2 / 2 2041 / 1997 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 3 / 2 2237 / 2223 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 5 / 4 8657 / 8429 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 2 / 1 8454 / 8533 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 2 / ✅ 1305 / 1290 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 4 / 2 1302 / 1246 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 1 / ✅ 3043 / 2981 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ ✅ / 2 2984 / 2830 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 5 / 8 3573 / 3525 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 4 / 4 3491 / 3514 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ 3 / 3 1688 / 1828 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ ✅ / ✅ 1887 / 1943 ✅ / ✅ ✅ / ✅

@dd-octo-sts

dd-octo-sts Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 50fdaf5)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124272420 Commit: 50fdaf5b318db079c2a3bb9dd95e68ade4f57c54

⚠️ Significant outliers

  • 🔴 future-genetic (JDK 21): runtime +2.9% (2033→2092 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10256 ms (21 iters) ✅ 10414 ms (21 iters) ≈ +1.5% (±12.2%) — / —
akka-uct 25 ✅ 8870 ms (24 iters) ✅ 8851 ms (24 iters) ≈ -0.2% (±10%) — / —
finagle-chirper 21 ✅ 5890 ms (33 iters) ✅ 5948 ms (33 iters) ≈ +1% (±24.7%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5500 ms (36 iters) ✅ 5489 ms (36 iters) ≈ -0.2% (±24.1%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2758 ms (68 iters) ✅ 2758 ms (68 iters) ≈ 0% (±2.7%) — / —
fj-kmeans 25 ✅ 2807 ms (67 iters) ✅ 2812 ms (66 iters) ≈ +0.2% (±2.6%) — / —
future-genetic 21 ✅ 2033 ms (91 iters) ✅ 2092 ms (89 iters) 🔴 +2.9% — / —
future-genetic 25 ✅ 2092 ms (88 iters) ✅ 2070 ms (89 iters) ≈ -1.1% (±2.6%) — / —
naive-bayes 21 ✅ 1287 ms (133 iters) ✅ 1246 ms (137 iters) ≈ -3.2% (±32.1%) — / —
naive-bayes 25 ✅ 1018 ms (168 iters) ✅ 1007 ms (170 iters) ≈ -1.1% (±31.3%) — / —
reactors 21 ✅ 16641 ms (15 iters) ✅ 16053 ms (15 iters) ≈ -3.5% (±8.5%) — / —
reactors 25 ✅ 18163 ms (15 iters) ✅ 18590 ms (15 iters) ≈ +2.4% (±5.4%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 1 / 2 1978 / 1925 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 1 / 3 2362 / 2350 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 3 / 1 8426 / 8414 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 8204 / 8727 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 4 / 1 1279 / 1286 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 2 / 5 1272 / 1269 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ ✅ / 1 3030 / 3054 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 2 / 1 2889 / 2862 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 2 / 5 3496 / 3533 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 3 / 6 3484 / 3505 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ ✅ / 1 1735 / 1587 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ ✅ / 2 1865 / 1879 ✅ / ✅ ✅ / ✅

@jbachorik jbachorik marked this pull request as ready for review July 13, 2026 12:05
@jbachorik jbachorik requested a review from a team as a code owner July 13, 2026 12:05

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 940a14e3f1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ddprof-lib/src/test/cpp/lineNumberTableCopy_ut.cpp
@dd-octo-sts

dd-octo-sts Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 940a14e)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124292080 Commit: 940a14e3f13afc617960977c2fdb64c2446c0b13

✅ Within expected boundaries

No significant runtime deltas (all within run-to-run noise) and no internal-counter outliers.

Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10222 ms (21 iters) ✅ 10417 ms (21 iters) ≈ +1.9% (±11.4%) — / —
finagle-chirper 21 ✅ 5958 ms (33 iters) ✅ 6028 ms (33 iters) ≈ +1.2% (±25.5%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5511 ms (36 iters) ✅ 5472 ms (36 iters) ≈ -0.7% (±24.7%) ⚠️ W:4 / ⚠️ W:3
fj-kmeans 21 ✅ 2711 ms (69 iters) ✅ 2754 ms (68 iters) ≈ +1.6% (±2.7%) — / —
fj-kmeans 25 ✅ 2867 ms (66 iters) ✅ 2863 ms (66 iters) ≈ -0.1% (±2.6%) — / —
future-genetic 21 ✅ 2047 ms (90 iters) ✅ 2065 ms (89 iters) ≈ +0.9% (±2.6%) — / —
future-genetic 25 ✅ 2078 ms (90 iters) ✅ 2052 ms (90 iters) ≈ -1.3% (±2.6%) — / —
naive-bayes 21 ✅ 1276 ms (134 iters) ✅ 1273 ms (134 iters) ≈ -0.2% (±32.8%) — / —
naive-bayes 25 ✅ 1026 ms (167 iters) ✅ 1009 ms (170 iters) ≈ -1.7% (±31.2%) — / —
reactors 25 ✅ 18431 ms (15 iters) ✅ 18546 ms (15 iters) ≈ +0.6% (±3.9%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ ✅ / ✅ 2005 / 2026 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 3 / 6 8842 / 8416 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 1 / 2 8303 / 8377 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 4 / 2 1289 / 1249 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 1 / 1 1284 / 1291 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 5 / 3 2893 / 2929 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 4 / 1 2950 / 2886 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 3 / 3 3507 / 3502 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 3 / 3 3471 / 3498 ✅ / ✅ ✅ / ✅
reactors 21 · / ✅ · / ✅ · / 2 · / 1879 · / ✅ · / ✅
reactors 25 ✅ / ✅ ✅ / ✅ ✅ / 1 1881 / 1878 ✅ / ✅ ✅ / ✅

@jbachorik

jbachorik commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator Author

Added commit 95a3498: gates FrameType::isRawPointer() on VM::isHotspot().

Unrelated root cause to the line-number-table fix above, found while investigating a J9 debug-lane CI abort in GetLineNumberTableLeakTest: the raw-pointer bci bit (bit 30) only means "jmethodID slot replaced by a VMMethod*" on HotSpot. On OpenJ9, an unencoded, VM-supplied bci can coincidentally have bit 30 set, which was misread as our raw-pointer encoding and routed into JVMSupport::resolve(), asserting/aborting on any non-HotSpot VM.

Fixing in the PR because the failure was virtually blocking getting the clean CI run.

Reproduced on the exact CI toolchain (JDK21 build JDK, Semeru 8.0.462/OpenJ9 0.53.0 test JDK): unpatched code crashed 1/2 iterations of GetLineNumberTableLeakTest; with the fix, 15/15 passed cleanly.

@dd-octo-sts

dd-octo-sts Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 60f1f83)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124300350 Commit: 60f1f839facc0a145054bc31b293d9adce478a05

✅ Within expected boundaries

No significant runtime deltas (all within run-to-run noise) and no internal-counter outliers.

Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10274 ms (21 iters) ✅ 10376 ms (21 iters) ≈ +1% (±11%) — / —
akka-uct 25 ✅ 8760 ms (24 iters) ✅ 8785 ms (24 iters) ≈ +0.3% (±10%) — / —
finagle-chirper 21 ✅ 5951 ms (33 iters) ✅ 5936 ms (33 iters) ≈ -0.3% (±25%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5456 ms (36 iters) ✅ 5484 ms (36 iters) ≈ +0.5% (±24.7%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2685 ms (71 iters) ✅ 2757 ms (68 iters) ≈ +2.7% (±2.7%) — / —
fj-kmeans 25 ✅ 2824 ms (66 iters) ✅ 2822 ms (66 iters) ≈ -0.1% (±2.6%) — / —
future-genetic 21 ✅ 2068 ms (90 iters) ✅ 2035 ms (91 iters) ≈ -1.6% (±2.5%) — / —
future-genetic 25 ✅ 2027 ms (91 iters) ✅ 2010 ms (92 iters) ≈ -0.8% (±2.7%) — / —
naive-bayes 21 ✅ 1264 ms (135 iters) ✅ 1264 ms (135 iters) ≈ 0% (±32.7%) — / —
naive-bayes 25 ✅ 1024 ms (167 iters) ✅ 1028 ms (167 iters) ≈ +0.4% (±31.8%) — / —
reactors 21 ✅ 16402 ms (15 iters) ✅ 15931 ms (15 iters) ≈ -2.9% (±7.1%) — / ⚠️ W:1
reactors 25 ✅ 18692 ms (15 iters) ✅ 18189 ms (15 iters) ≈ -2.7% (±4.1%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 3 / 5 1966 / 2049 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ ✅ / 3 2311 / 2330 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 7 / 4 8457 / 8096 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 2 / 4 8234 / 8254 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 3 / 2 1296 / 1309 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 1 / 1 1310 / 1292 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ ✅ / 3 2981 / 2886 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 2 / 1 2885 / 2909 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 4 / 1 3522 / 3485 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 4 / 1 3457 / 3524 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ ✅ / ✅ 1689 / 1502 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ ✅ / ✅ 1880 / 1853 ✅ / ✅ ✅ / ✅

@dd-octo-sts

dd-octo-sts Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 9b861da)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124308819 Commit: 9b861da01542ca14d00b08c75d07f3a1bcfaee95

⚠️ Significant outliers

  • 🔴 future-genetic (JDK 25): runtime +4.1% (2045→2129 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10371 ms (21 iters) ✅ 10358 ms (21 iters) ≈ -0.1% (±11.1%) — / —
akka-uct 25 ✅ 8922 ms (24 iters) ✅ 8802 ms (24 iters) ≈ -1.3% (±10.7%) — / —
finagle-chirper 21 ✅ 5949 ms (33 iters) ✅ 5981 ms (33 iters) ≈ +0.5% (±25.1%) ⚠️ W:4 / ⚠️ W:3
finagle-chirper 25 ✅ 5485 ms (36 iters) ✅ 5469 ms (36 iters) ≈ -0.3% (±23.6%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2715 ms (70 iters) ✅ 2696 ms (70 iters) ≈ -0.7% (±2.7%) — / —
fj-kmeans 25 ✅ 2821 ms (66 iters) ✅ 2802 ms (66 iters) ≈ -0.7% (±2.6%) — / —
future-genetic 21 ✅ 2125 ms (87 iters) ✅ 2147 ms (86 iters) ≈ +1% (±2.8%) — / —
future-genetic 25 ✅ 2045 ms (91 iters) ✅ 2129 ms (87 iters) 🔴 +4.1% — / —
naive-bayes 21 ✅ 1258 ms (136 iters) ✅ 1239 ms (138 iters) ≈ -1.5% (±32.4%) — / —
naive-bayes 25 ✅ 1023 ms (167 iters) ✅ 1000 ms (170 iters) ≈ -2.2% (±31%) — / —
reactors 21 ✅ 15794 ms (15 iters) ✅ 16692 ms (15 iters) ≈ +5.7% (±8%) — / —
reactors 25 ✅ 18882 ms (15 iters) ✅ 18768 ms (15 iters) ≈ -0.6% (±3.1%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ ✅ / 5 2009 / 1899 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 2 / 3 2191 / 2266 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 2 / 6 8352 / 8248 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ ✅ / 1 8470 / 8222 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ ✅ / 1 1284 / 1294 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ ✅ / 1 1268 / 1264 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 2 / 3 3021 / 3009 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 2 / 1 2932 / 2903 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 4 / 1 3534 / 3563 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 2 / 4 3449 / 3449 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ 1 / 1 1511 / 1853 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ 2 / 2 1947 / 1908 ✅ / ✅ ✅ / ✅

@dd-octo-sts

dd-octo-sts Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 4a9fba4)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124315244 Commit: 4a9fba47dbdd37f2a908bf3fc9b46289ba110f23

⚠️ Significant outliers

  • 🔴 fj-kmeans (JDK 21): runtime +4.7% (2714→2841 ms)
  • 🔴 future-genetic (JDK 21): runtime +3.7% (2084→2161 ms)
  • 🔴 future-genetic (JDK 25): runtime +4.6% (1975→2065 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10260 ms (21 iters) ✅ 10237 ms (21 iters) ≈ -0.2% (±11.6%) — / —
akka-uct 25 ✅ 8875 ms (24 iters) ✅ 8843 ms (24 iters) ≈ -0.4% (±9.5%) — / —
finagle-chirper 21 ✅ 6017 ms (33 iters) ✅ 6005 ms (33 iters) ≈ -0.2% (±25.5%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5423 ms (36 iters) ✅ 5454 ms (36 iters) ≈ +0.6% (±24.2%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2714 ms (69 iters) ✅ 2841 ms (66 iters) 🔴 +4.7% — / —
fj-kmeans 25 ✅ 2819 ms (66 iters) ✅ 2822 ms (66 iters) ≈ +0.1% (±2.6%) — / —
future-genetic 21 ✅ 2084 ms (90 iters) ✅ 2161 ms (86 iters) 🔴 +3.7% — / —
future-genetic 25 ✅ 1975 ms (93 iters) ✅ 2065 ms (89 iters) 🔴 +4.6% — / —
naive-bayes 21 ✅ 1224 ms (139 iters) ✅ 1272 ms (134 iters) ≈ +3.9% (±33.2%) — / —
naive-bayes 25 ✅ 1011 ms (169 iters) ✅ 1003 ms (170 iters) ≈ -0.8% (±31.3%) — / —
reactors 21 ✅ 16324 ms (15 iters) ✅ 16116 ms (15 iters) ≈ -1.3% (±7.5%) — / —
reactors 25 ✅ 18316 ms (15 iters) ✅ 18171 ms (15 iters) ≈ -0.8% (±4.5%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 1 / 1 1979 / 1997 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ ✅ / 1 2117 / 2196 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 3 / 2 8475 / 8556 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ ✅ / 2 8375 / 8142 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 5 / 4 1259 / 1253 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 2 / 4 1258 / 1260 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ ✅ / 3 3054 / 2990 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 2 / 2 2774 / 2868 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 2 / 7 3520 / 3509 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 2 / 4 3509 / 3493 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ ✅ / 2 1543 / 1657 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ 3 / 2 1834 / 1837 ✅ / ✅ ✅ / ✅

@rkennke rkennke left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a thorough pass — the core fix is correct, minimal, and consistent with the #537 string-probe hardening; the death-test's volatile-sink-vs-DCE handling and the dual SIGSEGV/SIGBUS registration are nice touches. Two small things inline, plus one question:

The isRawPointer HotSpot gating looks like a genuinely separate defect from the line-number-table memcpy crash — different symptom, different VM (OpenJ9 vs stock HotSpot), different file, and the title/JIRA (PROF-15385) only mention the memcpy. Was there a reason to fold it in here rather than split it into its own PR? Both trace back to stale-jmethodID handling so I'm fine either way — just want to make sure the J9 fix gets its own visibility/traceability.

Comment thread ddprof-lib/src/main/cpp/flightRecorder.cpp Outdated
Comment thread ddprof-lib/src/main/cpp/counters.h Outdated
@dd-octo-sts

dd-octo-sts Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 4a30289)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124389768 Commit: 4a3028975f537e577fd56ed2168a22a4cee9f805

⚠️ Significant outliers

  • 🔴 fj-kmeans (JDK 21): runtime +3.2% (2740→2828 ms)
  • 🔴 future-genetic (JDK 21): runtime +3.4% (2073→2144 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10481 ms (21 iters) ✅ 10325 ms (21 iters) ≈ -1.5% (±11.6%) — / —
akka-uct 25 ✅ 8706 ms (24 iters) ✅ 8864 ms (24 iters) ≈ +1.8% (±10.8%) — / —
finagle-chirper 21 ✅ 5972 ms (33 iters) ✅ 5995 ms (33 iters) ≈ +0.4% (±25.3%) ⚠️ W:3 / ⚠️ W:4
finagle-chirper 25 ✅ 5413 ms (36 iters) ✅ 5436 ms (36 iters) ≈ +0.4% (±24.4%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2740 ms (69 iters) ✅ 2828 ms (66 iters) 🔴 +3.2% — / —
fj-kmeans 25 ✅ 2743 ms (68 iters) ✅ 2813 ms (66 iters) ≈ +2.6% (±2.8%) — / —
future-genetic 21 ✅ 2073 ms (89 iters) ✅ 2144 ms (87 iters) 🔴 +3.4% — / —
future-genetic 25 ✅ 2079 ms (89 iters) ✅ 2076 ms (89 iters) ≈ -0.1% (±2.6%) — / —
naive-bayes 21 ✅ 1281 ms (133 iters) ✅ 1264 ms (136 iters) ≈ -1.3% (±32%) — / —
naive-bayes 25 ✅ 1006 ms (169 iters) ✅ 1016 ms (168 iters) ≈ +1% (±32.1%) — / —
reactors 21 ✅ 17040 ms (15 iters) ✅ 15890 ms (15 iters) ≈ -6.7% (±8.5%) — / —
reactors 25 ✅ 18695 ms (15 iters) ✅ 18872 ms (15 iters) ≈ +0.9% (±5.5%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 1 / 1 1997 / 1940 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ ✅ / 2 2319 / 2386 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 5 / 1 8773 / 8404 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 8178 / 8303 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 4 / 4 1305 / 1272 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 2 / 2 1280 / 1267 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 1 / 2 2933 / 2947 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ ✅ / ✅ 2869 / 2898 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 4 / 3 3463 / 3525 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 3 / 6 3424 / 3498 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ 3 / ✅ 1696 / 1602 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ ✅ / 2 1845 / 1904 ✅ / ✅ ✅ / ✅

@jbachorik

jbachorik commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator Author

@rkennke The reason for folding in the unrelated fix was that I was not able to get clean CI pass for this change. And jumping one more hoop to have the rather straight-forward fix felt a bit heavy. I might have gotten impatient but I was chasing weird CI failures in a kind of circular mode the whole day :(
If you feel strongly about this I can extract just the raw-pointer fix to a a PR and then wait with this PR until the previous PR gets merged.

Actually, extracted to a separate PR (#658) for posterity

@dd-octo-sts

dd-octo-sts Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit a37e7ec)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124483891 Commit: a37e7ecfed8e576aba503fa9a2d0dc608823198a

✅ Within expected boundaries

No significant runtime deltas (all within run-to-run noise) and no internal-counter outliers.

Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10229 ms (21 iters) ✅ 10204 ms (21 iters) ≈ -0.2% (±11%) — / —
akka-uct 25 ✅ 8810 ms (24 iters) ✅ 8861 ms (24 iters) ≈ +0.6% (±9.8%) — / —
finagle-chirper 21 ✅ 5996 ms (33 iters) ✅ 5886 ms (33 iters) ≈ -1.8% (±25%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5469 ms (36 iters) ✅ 5469 ms (36 iters) ≈ 0% (±24.4%) ⚠️ W:4 / ⚠️ W:3
fj-kmeans 21 ✅ 2786 ms (67 iters) ✅ 2742 ms (68 iters) ≈ -1.6% (±2.7%) — / —
fj-kmeans 25 ✅ 2765 ms (68 iters) ✅ 2812 ms (66 iters) ≈ +1.7% (±2.7%) — / —
future-genetic 21 ✅ 2090 ms (88 iters) ✅ 2095 ms (88 iters) ≈ +0.2% (±2.8%) — / —
future-genetic 25 ✅ 2082 ms (90 iters) ✅ 2081 ms (89 iters) ≈ -0% (±2.5%) — / —
naive-bayes 21 ✅ 1275 ms (134 iters) ✅ 1275 ms (134 iters) ≈ 0% (±33%) — / —
naive-bayes 25 ✅ 1024 ms (167 iters) ✅ 1014 ms (169 iters) ≈ -1% (±31.5%) — / —
reactors 21 ✅ 15996 ms (15 iters) ✅ 15620 ms (16 iters) ≈ -2.4% (±6.1%) — / —
reactors 25 ✅ 18343 ms (15 iters) ✅ 18732 ms (15 iters) ≈ +2.1% (±3.6%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ ✅ / 3 2032 / 2012 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ ✅ / 2 2236 / 2319 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 1 / 4 8550 / 8450 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 8345 / 8195 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 4 / 1 1263 / 1258 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ ✅ / 1 1301 / 1251 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 1 / 1 2925 / 3124 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 3 / 2 2875 / 2845 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ ✅ / 6 3459 / 3475 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 3 / 5 3464 / 3460 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ ✅ / 2 1551 / 1733 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ 2 / ✅ 1914 / 1888 ✅ / ✅ ✅ / ✅

jbachorik and others added 7 commits July 14, 2026 12:17
fillJavaMethodInfo() memcpy'd the JVMTI-returned line-number table
without probing it first, unlike the sibling class/method name+sig
strings hardened in #537. Crash reported in production as SIGSEGV in
Lookup::resolveMethod on stock HotSpot JDK 11.
…hodID-churn stress test

The EXPECT_DEATH reproducer silently passed on -O3 builds because the
optimizer eliminated the dead memcpy/free once owned_table's contents
were never observed; a forced volatile read fixes that. Also adds a
JUnit stress test that races class-unload churn against profiler
dump/resolve and asserts native whitebox counters actually observed a
stale jmethodID, not just that the JVM didn't crash.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fixes review comment: the counter-delta check is best-effort — a
healthy host that doesn't race unload tightly enough shouldn't flake
CI. Switch it to Assumptions.assumeTrue so a missed race reports as
skipped rather than failed.
init/analyze must stay on the same codeql-action version or analyze
rejects init's config; group them in dependabot.yml so bumps land
together atomically instead of drifting across separate PRs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Mirrors safefetch_ut.cpp: the safefetch trampoline can fault with
SIGBUS instead of SIGSEGV depending on platform, so both handlers
must be registered to avoid crashing the gtest binary.
…/memcpy()

A corrupted GetLineNumberTable() pointer for a stale jmethodID could pair
with a corrupted size, driving an unbounded byte count into the readability
probe. Cap it at 65535 (JVM spec's max code_length) and treat oversized
values like an unreadable pointer.
…ment

A corrupted negative size out-param fell through both branches,
leaving line_number_table non-null and untrusted for Deallocate().
@jbachorik jbachorik force-pushed the jb/resolve_method branch from a37e7ec to de7bcd8 Compare July 14, 2026 10:18
@dd-octo-sts

dd-octo-sts Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit de7bcd8)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124497122 Commit: de7bcd83d0babd25a16f0f822c3eca1ae009728d

⚠️ Significant outliers

  • 🔴 reactors (JDK 21): runtime +10.7% (15705→17378 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10395 ms (21 iters) ✅ 10325 ms (21 iters) ≈ -0.7% (±11.9%) — / —
fj-kmeans 21 ✅ 2825 ms (66 iters) ✅ 2799 ms (66 iters) ≈ -0.9% (±2.5%) — / —
fj-kmeans 25 ✅ 2837 ms (66 iters) ✅ 2805 ms (66 iters) ≈ -1.1% (±2.6%) — / —
future-genetic 21 ✅ 2078 ms (89 iters) ✅ 2091 ms (88 iters) ≈ +0.6% (±2.8%) — / —
future-genetic 25 ✅ 2020 ms (92 iters) ✅ 2066 ms (89 iters) ≈ +2.3% (±2.5%) — / —
reactors 21 ✅ 15705 ms (16 iters) ✅ 17378 ms (15 iters) 🔴 +10.7% — / —
reactors 25 ✅ 18547 ms (15 iters) ✅ 17977 ms (15 iters) ≈ -3.1% (±4.8%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 1 / 1 1954 / 1985 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 1 / 3 2359 / 2267 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 4 / ✅ 8994 / ✅ ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 2 / 1 8626 / 8459 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 5 / 1 1256 / 1249 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ ✅ / 2 1274 / 1263 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 2 / 1 2920 / 2864 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 3 / 1 2849 / 2852 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 3 / 1 3468 / 3467 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ ✅ / 3 1597 / 1889 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 1861 / 1823 ✅ / ✅ ✅ / ✅

@dd-octo-sts

dd-octo-sts Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 6c0bae9)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124510325 Commit: 6c0bae9b79d37c251488b9b9ca6cd7a84d6036bb

⚠️ Significant outliers

  • 🔴 fj-kmeans (JDK 21): runtime +4.3% (2697→2812 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10227 ms (21 iters) ✅ 10299 ms (21 iters) ≈ +0.7% (±11.6%) — / —
akka-uct 25 ✅ 8837 ms (24 iters) ✅ 8984 ms (24 iters) ≈ +1.7% (±10.6%) — / —
finagle-chirper 21 ✅ 5988 ms (33 iters) ✅ 5949 ms (33 iters) ≈ -0.7% (±24.9%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5491 ms (36 iters) ✅ 5491 ms (36 iters) ≈ 0% (±24.9%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2697 ms (70 iters) ✅ 2812 ms (66 iters) 🔴 +4.3% — / —
fj-kmeans 25 ✅ 2773 ms (68 iters) ✅ 2765 ms (68 iters) ≈ -0.3% (±2.8%) — / —
future-genetic 21 ✅ 2124 ms (87 iters) ✅ 2085 ms (89 iters) ≈ -1.8% (±2.7%) — / —
future-genetic 25 ✅ 2083 ms (89 iters) ✅ 2044 ms (91 iters) ≈ -1.9% (±2.6%) — / —
naive-bayes 21 ✅ 1302 ms (132 iters) ✅ 1243 ms (137 iters) ≈ -4.5% (±31.8%) — / —
naive-bayes 25 ✅ 1005 ms (170 iters) ✅ 1023 ms (167 iters) ≈ +1.8% (±32.5%) — / —
reactors 21 ✅ 15518 ms (16 iters) ✅ 15636 ms (15 iters) ≈ +0.8% (±8.2%) — / —
reactors 25 ✅ 18182 ms (15 iters) ✅ 18242 ms (15 iters) ≈ +0.3% (±4.2%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 3 / 4 1877 / 2019 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 3 / 3 2344 / 2393 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 4 / ✅ 8459 / 8370 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ ✅ / 1 8274 / 8291 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 2 / 3 1289 / 1249 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 2 / 3 1277 / 1272 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 1 / ✅ 2992 / 3025 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 2 / 3 2947 / 2968 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ ✅ / 4 3514 / 3484 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 6 / 4 3475 / 3491 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ ✅ / 1 1540 / 1529 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ 2 / ✅ 1858 / 1791 ✅ / ✅ ✅ / ✅

Comment thread ddprof-lib/src/main/cpp/flightRecorder.cpp Outdated
…opy()

isReadableRange() + memcpy still races if the probed memory changes
mapping between the check and the copy. Use SafeAccess::safeCopy()
instead, which fault-protects each read as it happens.
@jbachorik jbachorik requested a review from zhengyu123 July 14, 2026 14:23
@dd-octo-sts

dd-octo-sts Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit f4ee43a)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124544793 Commit: f4ee43ac48f07c97b3e9424fd7f2d60494f1d25c

⚠️ Significant outliers

  • 🟢 future-genetic (JDK 25): runtime -4.1% (2104→2017 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10336 ms (21 iters) ✅ 10190 ms (21 iters) ≈ -1.4% (±10.9%) — / —
akka-uct 25 ✅ 8817 ms (24 iters) ✅ 8964 ms (24 iters) ≈ +1.7% (±10.1%) — / —
finagle-chirper 21 ✅ 5942 ms (33 iters) ✅ 5963 ms (33 iters) ≈ +0.4% (±25.3%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5453 ms (36 iters) ✅ 5455 ms (36 iters) ≈ +0% (±24.1%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2767 ms (67 iters) ✅ 2714 ms (69 iters) ≈ -1.9% (±2.7%) — / —
fj-kmeans 25 ✅ 2843 ms (66 iters) ✅ 2838 ms (66 iters) ≈ -0.2% (±2.7%) — / —
future-genetic 21 ✅ 2098 ms (88 iters) ✅ 2078 ms (89 iters) ≈ -1% (±2.8%) — / —
future-genetic 25 ✅ 2104 ms (88 iters) ✅ 2017 ms (92 iters) 🟢 -4.1% — / —
naive-bayes 21 ✅ 1259 ms (136 iters) ✅ 1302 ms (131 iters) ≈ +3.4% (±33.5%) — / —
naive-bayes 25 ✅ 1015 ms (169 iters) ✅ 1014 ms (169 iters) ≈ -0.1% (±31.3%) — / —
reactors 21 ✅ 16885 ms (15 iters) ✅ 16926 ms (15 iters) ≈ +0.2% (±7.8%) — / —
reactors 25 ✅ 18291 ms (15 iters) ✅ 18680 ms (15 iters) ≈ +2.1% (±5.2%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 1 / 2 2014 / 1910 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 3 / 2 1971 / 2318 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ ✅ / 1 8420 / 8573 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ ✅ / 1 1241 / 1257 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 2 / ✅ 1281 / 1295 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 3 / 1 3004 / 2971 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 2 / 1 2883 / 2848 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 2 / 4 3533 / 3533 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 5 / 3 3487 / 3515 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ 3 / 2 1619 / 1734 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ 1 / 1 1866 / 1865 ✅ / ✅ ✅ / ✅

Comment thread ddprof-lib/src/main/cpp/flightRecorder.cpp Outdated

@rkennke rkennke left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thank you!

@jbachorik jbachorik requested a review from zhengyu123 July 14, 2026 15:29
@dd-octo-sts

dd-octo-sts Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 47b8b58)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124565177 Commit: 47b8b58785c907e41dc50eba88b531eeca531762

⚠️ Significant outliers

  • 🔴 future-genetic (JDK 21): runtime +6.8% (2091→2233 ms)
  • 🟢 future-genetic (JDK 25): runtime -2.8% (2064→2007 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10221 ms (21 iters) ✅ 10315 ms (21 iters) ≈ +0.9% (±10.8%) — / —
akka-uct 25 ✅ 8917 ms (24 iters) ✅ 8942 ms (24 iters) ≈ +0.3% (±10.6%) — / —
finagle-chirper 21 ✅ 6035 ms (33 iters) ✅ 5963 ms (33 iters) ≈ -1.2% (±25.4%) ⚠️ W:5 / ⚠️ W:3
finagle-chirper 25 ✅ 5486 ms (36 iters) ✅ 5544 ms (36 iters) ≈ +1.1% (±24.6%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2771 ms (67 iters) ✅ 2772 ms (67 iters) ≈ +0% (±2.7%) — / —
fj-kmeans 25 ✅ 2787 ms (68 iters) ✅ 2841 ms (66 iters) ≈ +1.9% (±2.8%) — / —
future-genetic 21 ✅ 2091 ms (88 iters) ✅ 2233 ms (83 iters) 🔴 +6.8% — / —
future-genetic 25 ✅ 2064 ms (90 iters) ✅ 2007 ms (92 iters) 🟢 -2.8% — / —
naive-bayes 21 ✅ 1264 ms (136 iters) ✅ 1290 ms (133 iters) ≈ +2.1% (±32.6%) — / —
naive-bayes 25 ✅ 975 ms (174 iters) ✅ 1045 ms (164 iters) ≈ +7.2% (±33.1%) — / —
reactors 21 ✅ 15750 ms (15 iters) ✅ 16108 ms (15 iters) ≈ +2.3% (±6%) — / —
reactors 25 ✅ 18027 ms (15 iters) ✅ 18694 ms (15 iters) ≈ +3.7% (±4.5%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 2 / 1 2089 / 1945 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 3 / 1 2180 / 2162 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 3 / 3 8336 / 8737 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 8531 / 8773 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 2 / 4 1269 / 1270 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 2 / ✅ 1293 / 1271 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 1 / 2 2931 / 3002 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 1 / 3 2898 / 2793 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 2 / 2 3435 / 3496 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ ✅ / ✅ 1512 / 1619 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ 1 / 2 1865 / 1881 ✅ / ✅ ✅ / ✅

Comment thread ddprof-lib/src/main/cpp/flightRecorder.cpp
@jbachorik jbachorik merged commit 2c58b48 into main Jul 14, 2026
105 checks passed
@jbachorik jbachorik deleted the jb/resolve_method branch July 14, 2026 17:19
@jbachorik

Copy link
Copy Markdown
Collaborator Author

Thanks for the reviews!

@github-actions github-actions Bot added this to the 1.47.0 milestone Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants