Skip to content

fix(metrics): match V3 resource mapping with Agent#1770

Merged
rayz merged 2 commits into
tobz/datadog-metrics-v3-payload-supportfrom
rayz/metrics-v3-resource-mapping
May 29, 2026
Merged

fix(metrics): match V3 resource mapping with Agent#1770
rayz merged 2 commits into
tobz/datadog-metrics-v3-payload-supportfrom
rayz/metrics-v3-resource-mapping

Conversation

@rayz
Copy link
Copy Markdown
Contributor

@rayz rayz commented May 29, 2026

Summary

Match the Datadog Agent’s V3 resource mapping behavior for series and sketches.

For V3 series, this promotes resource-style tags into V3 resources:

  • device:<value> becomes a device resource and is removed from tags.
  • Valid dd.internal.resource:<type>:<name> tags become resources and are removed from tags.
  • Malformed dd.internal.resource:* tags are dropped.
  • Resource order matches the Agent: host, device, then promoted resources.

For V3 sketches, this keeps device:* and dd.internal.resource:* as normal tags and only emits host as a resource, matching the Agent sketch path.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

How did you test this PR?

References

@dd-octo-sts dd-octo-sts Bot added area/core Core functionality, event model, etc. area/io General I/O and networking. area/components Sources, transforms, and destinations. area/ci CI/CD, automated testing, etc. area/test All things testing: unit/integration, correctness, SMP regression, etc. encoder/datadog-metrics Datadog Metrics encoder. forwarder/datadog Datadog forwarder. labels May 29, 2026
@rayz rayz changed the base branch from main to tobz/datadog-metrics-v3-payload-support May 29, 2026 17:36
…support' into rayz/metrics-v3-resource-mapping
@dd-octo-sts dd-octo-sts Bot removed area/core Core functionality, event model, etc. area/io General I/O and networking. area/ci CI/CD, automated testing, etc. area/test All things testing: unit/integration, correctness, SMP regression, etc. forwarder/datadog Datadog forwarder. labels May 29, 2026
@rayz rayz marked this pull request as ready for review May 29, 2026 17:50
@rayz rayz requested a review from a team as a code owner May 29, 2026 17:50
Copilot AI review requested due to automatic review settings May 29, 2026 17:50
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented May 29, 2026

Regression Detector (Agent Data Plane)

Run ID: 47ad2953-685d-4842-9f60-22460b5ecb09
Baseline: 1ad7cde2 · Comparison: 2c514b83 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (35)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
dsd_uds_512kb_3k_contexts_cpu (erratic) cpu ⚪ +6.05 metrics profiles logs
quality_gates_rss_idle memory ⚪ +2.06 metrics profiles logs
dsd_uds_10mb_3k_contexts_memory memory ⚪ +1.20 metrics profiles logs
quality_gates_rss_dsd_low memory ⚪ +1.04 metrics profiles logs
otlp_ingest_metrics_5mb_cpu (erratic) cpu ⚪ +0.90 metrics profiles logs
dsd_uds_1mb_3k_contexts_memory memory ⚪ +0.88 metrics profiles logs
otlp_ingest_traces_5mb_throughput throughput ⚪ -0.83 metrics profiles logs
quality_gates_rss_dsd_medium memory ⚪ +0.82 metrics profiles logs
dsd_uds_512kb_3k_contexts_memory memory ⚪ +0.72 metrics profiles logs
dsd_uds_100mb_3k_contexts_memory memory ⚪ +0.65 metrics profiles logs
otlp_ingest_logs_5mb_cpu (ignored) cpu ⚪ +0.63 metrics profiles logs
dsd_uds_100mb_3k_contexts_cpu (erratic) cpu ⚪ +0.60 metrics profiles logs
dsd_uds_500mb_3k_contexts_memory memory ⚪ +0.59 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_cpu (erratic) cpu ⚪ +0.47 metrics profiles logs
otlp_ingest_traces_5mb_cpu (erratic) cpu ⚪ +0.43 metrics profiles logs
dsd_uds_500mb_3k_contexts_cpu (erratic) cpu ⚪ +0.36 metrics profiles logs
otlp_ingest_metrics_5mb_memory memory ⚪ +0.35 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_memory memory ⚪ +0.30 metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory ⚪ +0.16 metrics profiles logs
otlp_ingest_traces_5mb_memory memory ⚪ +0.14 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_memory memory ⚪ +0.11 metrics profiles logs
dsd_uds_10mb_3k_contexts_throughput throughput ⚪ -0.01 metrics profiles logs
dsd_uds_1mb_3k_contexts_throughput throughput ⚪ -0.00 metrics profiles logs
dsd_uds_512kb_3k_contexts_throughput throughput ⚪ -0.00 metrics profiles logs
dsd_uds_100mb_3k_contexts_throughput throughput ⚪ +0.00 metrics profiles logs
quality_gates_rss_dsd_heavy memory ⚪ -0.01 metrics profiles logs
otlp_ingest_metrics_5mb_throughput throughput ⚪ +0.02 metrics profiles logs
otlp_ingest_logs_5mb_throughput (ignored) throughput ⚪ +0.03 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_throughput throughput ⚪ +0.03 metrics profiles logs
dsd_uds_500mb_3k_contexts_throughput throughput ⚪ +0.08 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_throughput throughput ⚪ +0.09 metrics profiles logs
dsd_uds_10mb_3k_contexts_cpu (erratic) cpu ⚪ -0.68 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_cpu (erratic) cpu ⚪ -1.06 metrics profiles logs
otlp_ingest_logs_5mb_memory (ignored) memory ⚪ -6.23 metrics profiles logs
dsd_uds_1mb_3k_contexts_cpu (erratic) cpu 🟢 -10.78 metrics profiles logs
Bounds Checks: ✅ Passed (5)
experiment check replicates observed links
quality_gates_rss_dsd_heavy memory_usage 10/10 ✅ 122 MiB ≤ 140 MiB metrics profiles logs
quality_gates_rss_dsd_low memory_usage 10/10 ✅ 40.2 MiB ≤ 50 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 10/10 ✅ 60.5 MiB ≤ 75 MiB metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory_usage 10/10 ✅ 177 MiB ≤ 200 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 27.2 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Aligns Saluki’s Datadog V3 metrics encoding with the Datadog Agent’s resource mapping semantics, especially around promoting device:* and dd.internal.resource:* tags into V3 “resources” for series while keeping sketch behavior unchanged.

Changes:

  • Updated V3 series encoding to promote device:* and well-formed dd.internal.resource:<type>:<name> tags into resources (and remove them from tags), with resource ordering matching the Agent.
  • Updated V3 sketch encoding to not promote device:* / dd.internal.resource:* tags (keeps them as tags) and only emit host as a resource.
  • Added unit tests asserting tag/resource behavior and ordering for series vs sketches.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +1255 to +1257
fn is_v3_series_resource_tag(tag: &Tag) -> bool {
tag.name() == "dd.internal.resource" && tag.value().is_some()
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot sounds on point here, but I think all that matters is how Agent is doing it, which I'll leave to you to double check and fix here if necessary.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems weird but it is what the Agent does. Agent only treats tags as internal resource tags if they start with dd.internal.resource: so dd.internal.resource would be kept

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2c514b8369

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +1255 to +1257
fn is_v3_series_resource_tag(tag: &Tag) -> bool {
tag.name() == "dd.internal.resource" && tag.value().is_some()
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Drop malformed internal resource tags from series

When a V3 series includes a malformed reserved tag with no value, such as the bare tag dd.internal.resource, this predicate returns false and the filter above keeps it as a normal metric tag. The previous starts_with("dd.internal.resource") filter dropped these reserved-prefix tags, and the commit message says malformed dd.internal.resource:* tags should be dropped; leaking them into the V3 tagset makes ADP diverge from the Agent behavior for malformed resource markers.

Useful? React with 👍 / 👎.

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented May 29, 2026

Binary Size Analysis (Agent Data Plane)

Baseline: 1ad7cde · Comparison: 2c514b8 · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 37.65 MiB (baseline) vs 37.97 MiB (comparison)
Size Change: +327.95 KiB (+0.85%)

✅ Binary size difference within threshold

Changes by Module
Module File Size Symbols
saluki_components::encoders::datadog +83.20 KiB 356
core +53.09 KiB 14500
anyhow +39.34 KiB 1699
alloc +37.12 KiB 2408
hyper +27.84 KiB 584
saluki_components::sources::otlp -21.76 KiB 236
hashbrown +21.32 KiB 1115
figment +20.92 KiB 700
[sections] +17.78 KiB 9
piecemeal -14.05 KiB 31
axum +13.12 KiB 428
chrono -13.08 KiB 23
tokio +11.44 KiB 4515
http +11.34 KiB 418
serde_core +10.89 KiB 889
hyper_util -10.21 KiB 139
http_body_util -9.12 KiB 208
h2 +9.03 KiB 779
saluki_components::common::datadog +8.70 KiB 440
tracing -8.11 KiB 175
Detailed Symbol Changes
    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +1.6%  +327Ki  +1.6%  +259Ki    [47045 Others]
  [NEW]  +136Ki  [NEW]  +136Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::hf265beba717a8ffb
  [NEW] +66.7Ki  [NEW] +66.5Ki    saluki_core::topology::built::BuiltTopology::spawn::_{{closure}}::hef6f38503239c166
  [NEW] +66.1Ki  [NEW] +66.0Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::hdcdf70adbde87ffc
  [NEW] +58.7Ki  [NEW] +58.5Ki    agent_data_plane::internal::env::workload::build_collector::_{{closure}}::ha56b75595a60c965
  [NEW] +58.2Ki  [NEW] +58.0Ki    saluki_core::topology::blueprint::TopologyBlueprint::build::_{{closure}}::hacbbeaf29f1d76dc
  [NEW] +57.1Ki  [NEW] +56.9Ki    agent_data_plane::internal::env::ADPEnvironmentProvider::from_configuration::_{{closure}}::hb77df6cc347f39b1
  [NEW] +56.7Ki  [NEW] +56.5Ki    agent_data_plane::cli::debug::handle_debug_command::_{{closure}}::ha2e6796c57380dae
  [NEW] +56.0Ki  [NEW] +55.8Ki    agent_data_plane::cli::dogstatsd::handle_dogstatsd_command::_{{closure}}::hb45f834f247c7f00
  [NEW] +49.8Ki  [NEW] +49.5Ki    agent_data_plane::main::_{{closure}}::hafe55c655b44bc0f
  [NEW] +49.7Ki  [NEW] +49.6Ki    core::ops::function::FnOnce::call_once::he29f19ec791745c2
  [DEL] -48.9Ki  [DEL] -48.8Ki    core::ops::function::FnOnce::call_once::h669ff363e842dff3
  [DEL] -49.7Ki  [DEL] -49.4Ki    agent_data_plane::main::_{{closure}}::h7d49a9469c34b899
  [DEL] -56.5Ki  [DEL] -56.3Ki    agent_data_plane::cli::dogstatsd::handle_dogstatsd_command::_{{closure}}::he79b0808df75cda1
  [DEL] -56.6Ki  [DEL] -56.4Ki    agent_data_plane::cli::debug::handle_debug_command::_{{closure}}::h7c655b6902e2f89a
  [DEL] -57.1Ki  [DEL] -56.9Ki    agent_data_plane::internal::env::ADPEnvironmentProvider::from_configuration::_{{closure}}::ha0920a2b79b02cce
  [DEL] -57.8Ki  [DEL] -57.6Ki    saluki_core::topology::blueprint::TopologyBlueprint::build::_{{closure}}::h32219546faa6f65a
  [DEL] -58.4Ki  [DEL] -58.2Ki    agent_data_plane::internal::env::workload::build_collector::_{{closure}}::ha8e6b4b93de14a84
  [DEL] -66.2Ki  [DEL] -66.0Ki    saluki_core::topology::built::BuiltTopology::spawn::_{{closure}}::hfe1e9d26bba4c0cd
  [DEL] -66.3Ki  [DEL] -66.1Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h9611a56813a60eb8
  [DEL]  -136Ki  [DEL]  -136Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::h4399bd59161f275e
  +0.9%  +327Ki  +0.8%  +260Ki    TOTAL

@datadog-datadog-prod-us1-2

This comment has been minimized.

@rayz rayz merged commit f27ed86 into tobz/datadog-metrics-v3-payload-support May 29, 2026
76 checks passed
@rayz rayz deleted the rayz/metrics-v3-resource-mapping branch May 29, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/components Sources, transforms, and destinations. encoder/datadog-metrics Datadog Metrics encoder.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants