Skip to content

Store known span tags densely in TagMap by tag-id (phase 2)#12045

Draft
dougqh wants to merge 1 commit into
dougqh/tagsetfrom
dougqh/dense-store-v2
Draft

Store known span tags densely in TagMap by tag-id (phase 2)#12045
dougqh wants to merge 1 commit into
dougqh/tagsetfrom
dougqh/dense-store-v2

Conversation

@dougqh

@dougqh dougqh commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

What

Adds a dense store for known span tags in TagMap: tags resolved to a stable tag-id via KnownTagCodec.keyOf are held in insertion-ordered parallel arrays (knownIds/knownValues) with no per-tag Entry object — eliminating the TagMap$Entry allocation that macro JFR profiling flagged as the #1 tracer allocation lever.

  • Dormant until a resolver registers (KnownTags), so production is byte-identical until the generator lands — all-known maps allocate zero buckets.
  • Iteration/serialization emit dense entries through a reused flyweight (EntryReadingHelper) — no per-entry Entry alloc on the read/serialize path either.
  • Read-through is chain-aware: dense entries participate in the multi-level ancestor union via parentDenseVisible (mirrors parentEntryVisible), nearest-level-wins with tombstone/shadow checks. Disjointness (known tags never bucket) keeps the two stores independent by construction.

Why

Removes TagMap$Entry allocation for known tags (the macro alloc win — see the tracer-overhead JFR profiling). CPU is neutral/parity; the headline is allocation on the app thread.

Stack

Sits on dougqh/tagset (StringIndex / #11660 base). Supersedes the old dense PR #11814.

Test

TagMapDenseForkedTest, TagMapDenseFuzzForkedTest, KnownTagsTest green; spotbugsMain + spotlessJavaCheck clean.

🤖 Generated with Claude Code

Known tags (keyOf resolves to a stored id) are held in insertion-ordered
parallel arrays (knownIds/knownValues) with NO per-tag Entry object — the
allocation lever. Lazily allocated on the first known-tag write; custom tags
stay in the hash buckets. Disjoint by construction (known-ness is global), so
read-through shadow checks stay within-region and the bucket path is unchanged.

- KnownTagCodec (id encoding + resolver) + hand-written KnownTags (keyOf
  substrate over StringIndex). Off-by-default: dormant until a resolver
  registers, so production is byte-identical.
- CoreTracer flips it live behind `-Ddd.trace.dense.tags.enabled` for A/B.
- Sizing is a generous fixed stopgap (KNOWN_INIT_CAP=12, the per-type max);
  exact per-type sizing comes with the tag registry.

Reconciled onto the level-split stack (fold + read-through + StringIndex);
built on the folded final-class TagMap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dougqh dougqh added comp: core Tracer core tag: no release notes Changes to exclude from release notes type: refactoring tag: ai generated Largely based on code generated by an AI or LLM labels Jul 23, 2026
@datadog-official

datadog-official Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Bits has a CI fix ready

🟢 Investigated · 🟢 Fix prepared · ⚪ Validation skipped · 🟠 Ready

CoreTracer.java contained an overlong dense-tag initialization comment that violated the Java formatter. Wrapped the comment without changing runtime behavior.

Commit fix to this PR


View in Datadog | Reviewed commit d758efe · Any feedback? Reach out in #deveng-pr-agent

@datadog-official

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.95 s 14.01 s [-1.1%; +0.3%] (no difference)
startup:insecure-bank:tracing:Agent 12.97 s 12.97 s [-0.9%; +0.8%] (no difference)
startup:petclinic:appsec:Agent 16.91 s 16.63 s [+0.7%; +2.6%] (maybe worse)
startup:petclinic:iast:Agent 16.86 s 16.89 s [-1.0%; +0.6%] (no difference)
startup:petclinic:profiling:Agent 16.61 s 16.38 s [+0.3%; +2.5%] (maybe worse)
startup:petclinic:sca:Agent 16.84 s 16.85 s [-1.0%; +1.0%] (no difference)
startup:petclinic:tracing:Agent 16.24 s 16.29 s [-1.2%; +0.6%] (no difference)

Commit: d758efe1 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@dougqh dougqh changed the title Store known span tags densely in TagMap by tag-id Store known span tags densely in TagMap by tag-id (phase 2) Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant