Skip to content

chore(dogstatsd): add support for dogstatsd_context_expiry_seconds#1650

Merged
tobz merged 1 commit into
mainfrom
tobz/align-dsd-context-expiry
May 14, 2026
Merged

chore(dogstatsd): add support for dogstatsd_context_expiry_seconds#1650
tobz merged 1 commit into
mainfrom
tobz/align-dsd-context-expiry

Conversation

@tobz
Copy link
Copy Markdown
Member

@tobz tobz commented May 14, 2026

Summary

This PR adds support for dogstatsd_context_expiry_seconds, which controls how long contexts stay cached for in the context resolver for the DogStatsD source.

Prior to this PR, we simply hardcoded the expiration to 30 seconds: close enough to the default of 20 seconds in the Datadog Agent, but not the same and not configurable. We now have both the proper default value of 20 seconds and we read the value from the configuration if present. Huzzah!

I made some small cleanup tweaks -- ordering of constants, deduplicating variable usages, etc -- as I was going through the code. They're small, and should be very obvious.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

How did you test this PR?

  • Existing configuration smoke test.

References

DADP-72

Closes #1340

@tobz tobz requested a review from a team as a code owner May 14, 2026 00:25
@tobz tobz added the type/chore Updates to dependencies or general "administrative" tasks necessary to maintain the codebase/repo. label May 14, 2026
@dd-octo-sts dd-octo-sts Bot added area/components Sources, transforms, and destinations. source/dogstatsd DogStatsD source. area/docs Reference documentation. labels May 14, 2026
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented May 14, 2026

Binary Size Analysis (Agent Data Plane)

Target: fb3fdf0 (baseline) vs 6e1ca86 (comparison) diff
Analysis Type: Stripped binaries (debug symbols excluded)
Baseline Size: 37.15 MiB
Comparison Size: 37.15 MiB
Size Change: -1.80 KiB (-0.00%)
Pass/Fail Threshold: +5%
Result: PASSED ✅

Changes by Module

Module File Size Symbols
tower -4.69 KiB 15
saluki_io::net::util +4.13 KiB 3
core -3.15 KiB 644
saluki_io::deser::framing +3.15 KiB 1
http_body_util +2.65 KiB 14
saluki_components::sources::dogstatsd -2.40 KiB 23
crossbeam_channel -2.08 KiB 4
tower_layer +1.89 KiB 1
anon.fd7b529fe56f46398ecdea2e5e277011.5.llvm.6425510092197803827 -1.69 KiB 1
anon.bb6a30149cebda15d8b61928541f6dfa.456.llvm.9067471501927015186 +1.69 KiB 1
std +1.64 KiB 6
[Unmapped] -1.58 KiB 1
anon.022733de89c1b9e4dcd1337b503abb98.1039.llvm.16171459405870212930 +1.51 KiB 1
h2 +1.49 KiB 4
anon.dd5b1f3bfca0f8b909ebdc618a7bf277.21.llvm.1618916015308137990 -1.42 KiB 1
anon.76b10c6ba1f5f88986f9dfd595c03364.22.llvm.18017217637076799424 -1.35 KiB 1
anon.f06a3103d0444795408a52e3b4e4e6de.6.llvm.12077248601313944555 +1.34 KiB 1
anon.b187eb6dc8b3f143c80dbecbb91de4c1.66.llvm.16087103420286002926 -1.33 KiB 1
anon.d2678bcbaf4238fcd297e08325cf4495.35.llvm.5207802880310005362 +1.32 KiB 1
anon.7085673481cc83f1b54ec9b36a638a73.3.llvm.16819013267556482521 +1.32 KiB 1

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW] +18.2Ki  [NEW] +18.0Ki    _<saluki_components::sources::dogstatsd::DogStatsDConfiguration as saluki_core::components::sources::builder::SourceBuilder>::build::_{{closure}}::h417fb6c2bd9965f4
  [NEW] +4.83Ki  [NEW] +4.64Ki    _<saluki_io::net::util::middleware::retry_circuit_breaker::ResponseFuture<P,F,Request> as core::future::future::Future>::poll::h54feb35fa34fbefe
  [NEW] +4.05Ki  [NEW] +3.70Ki    core::ptr::drop_in_place<saluki_components::sources::dogstatsd::DogStatsDConfiguration>::hf5c0f524c13de77e
  +0.0% +1.60Ki  +0.1% +3.28Ki    [4284 Others]
  [NEW] +3.15Ki  [NEW] +2.97Ki    _<saluki_io::deser::framing::NestedFramer<Inner,Outer> as saluki_io::deser::framing::Framer>::next_frame::h9872b4af11b6685d
  [NEW] +2.92Ki  [NEW] +2.80Ki    saluki_components::sources::dogstatsd::resolver::ContextResolvers::new::h4aead2e78f0554a7
  [NEW] +1.97Ki  [NEW]    +315    core::ptr::drop_in_place<tokio::sync::mpsc::bounded::Permit<saluki_core::topology::interconnect::event_buffer::FixedSizeEventBuffer<1024_usize>>>::h048fd97e097085bc
  +296% +1.89Ki  +385% +1.89Ki    _<tower_layer::stack::Stack<Inner,Outer> as tower_layer::Layer<S>>::layer::hb9e23bcd228e099f
  [NEW] +1.69Ki  [NEW]     +97    anon.bb6a30149cebda15d8b61928541f6dfa.456.llvm.9067471501927015186
   +17% +1.64Ki   +18% +1.64Ki    _<std::io::error::ErrorKind as core::fmt::Debug>::fmt::h6f31d7a057898f75
 -46.4% -1.58Ki  [ = ]       0    [Unmapped]
  [DEL] -1.61Ki  [DEL] -1.46Ki    _<tower::util::map_request::MapRequestLayer<F> as tower_layer::Layer<S>>::layer::hfc72de87e1a2f5cc
 -90.8% -1.64Ki -95.7% -1.64Ki    _<&T as core::fmt::Debug>::fmt::h6defbe7231ccc0b3
  [DEL] -1.69Ki  [DEL]     -97    anon.fd7b529fe56f46398ecdea2e5e277011.5.llvm.6425510092197803827
  [DEL] -1.97Ki  [DEL]    -315    core::ptr::drop_in_place<tokio::sync::mpsc::bounded::Permit<saluki_components::sources::otlp::OtlpResource>>::hec5850e6645f78db
 -71.0% -2.07Ki -74.3% -2.07Ki    crossbeam_channel::flavors::array::Channel<T>::recv::_{{closure}}::hc32a41b3424d63c5
  [DEL] -2.94Ki  [DEL] -2.83Ki    saluki_components::sources::dogstatsd::resolver::ContextResolvers::new::h3a99a60c1ef84507
  [DEL] -3.18Ki  [DEL] -3.02Ki    _<saluki_components::sources::dogstatsd::framer::DsdFramer as saluki_io::deser::framing::Framer>::next_frame::h5b5492ef1d4669f6
  [DEL] -4.05Ki  [DEL] -3.70Ki    core::ptr::drop_in_place<saluki_components::sources::dogstatsd::DogStatsDConfiguration>::h3c9cf99721606cf6
  [DEL] -4.79Ki  [DEL] -4.64Ki    _<tower::limit::concurrency::future::ResponseFuture<F> as core::future::future::Future>::poll::hc82a8bc58d98f9b2
  [DEL] -18.2Ki  [DEL] -18.0Ki    _<saluki_components::sources::dogstatsd::DogStatsDConfiguration as saluki_core::components::sources::builder::SourceBuilder>::build::_{{closure}}::h11f7e705f538dbbd
  -0.0% -1.80Ki  +0.0% +1.57Ki    TOTAL

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented May 14, 2026

Regression Detector (Agent Data Plane)

Run ID: 0d750231-4e98-41b1-ab83-1b953b40ff70
Baseline: fb3fdf06 · Comparison: 6e1ca867 · Diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (35)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
dsd_uds_10mb_3k_contexts_cpu (erratic) cpu ⚪ +2.30 metrics profiles logs
dsd_uds_100mb_3k_contexts_cpu (erratic) cpu ⚪ +1.70 metrics profiles logs
dsd_uds_512kb_3k_contexts_memory memory ⚪ +1.15 metrics profiles logs
dsd_uds_1mb_3k_contexts_memory memory ⚪ +1.06 metrics profiles logs
dsd_uds_500mb_3k_contexts_cpu (erratic) cpu ⚪ +0.63 metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory ⚪ +0.42 metrics profiles logs
otlp_ingest_metrics_5mb_cpu (erratic) cpu ⚪ +0.32 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_cpu (erratic) cpu ⚪ +0.31 metrics profiles logs
quality_gates_rss_dsd_heavy memory ⚪ +0.19 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_memory memory ⚪ +0.17 metrics profiles logs
dsd_uds_10mb_3k_contexts_memory memory ⚪ +0.06 metrics profiles logs
otlp_ingest_logs_5mb_cpu (ignored) cpu ⚪ +0.06 metrics profiles logs
dsd_uds_100mb_3k_contexts_throughput throughput ⚪ -0.01 metrics profiles logs
dsd_uds_1mb_3k_contexts_throughput throughput ⚪ -0.00 metrics profiles logs
dsd_uds_512kb_3k_contexts_throughput throughput ⚪ -0.00 metrics profiles logs
quality_gates_rss_dsd_medium memory ⚪ +0.00 metrics profiles logs
otlp_ingest_logs_5mb_throughput (ignored) throughput ⚪ +0.01 metrics profiles logs
dsd_uds_10mb_3k_contexts_throughput throughput ⚪ +0.03 metrics profiles logs
quality_gates_rss_dsd_low memory ⚪ -0.03 metrics profiles logs
otlp_ingest_metrics_5mb_throughput throughput ⚪ +0.03 metrics profiles logs
dsd_uds_100mb_3k_contexts_memory memory ⚪ -0.07 metrics profiles logs
otlp_ingest_traces_5mb_throughput throughput ⚪ +0.13 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_throughput throughput ⚪ +0.16 metrics profiles logs
otlp_ingest_traces_5mb_cpu (erratic) cpu ⚪ -0.18 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_throughput throughput ⚪ +0.19 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_memory memory ⚪ -0.20 metrics profiles logs
otlp_ingest_traces_5mb_memory memory ⚪ -0.28 metrics profiles logs
quality_gates_rss_idle memory ⚪ -0.34 metrics profiles logs
dsd_uds_500mb_3k_contexts_memory memory ⚪ -0.35 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_cpu (erratic) cpu ⚪ -1.38 metrics profiles logs
dsd_uds_500mb_3k_contexts_throughput throughput ⚪ +1.53 metrics profiles logs
otlp_ingest_metrics_5mb_memory memory ⚪ -3.86 metrics profiles logs
dsd_uds_1mb_3k_contexts_cpu (erratic) cpu 🟢 -7.53 metrics profiles logs
dsd_uds_512kb_3k_contexts_cpu (erratic) cpu 🟢 -8.82 metrics profiles logs
otlp_ingest_logs_5mb_memory (ignored) memory ⚪ -10.95 metrics profiles logs
Bounds Checks: ✅ Passed (5)
experiment check replicates observed links
quality_gates_rss_dsd_heavy memory_usage 10/10 ✅ 123 MiB ≤ 140 MiB metrics profiles logs
quality_gates_rss_dsd_low memory_usage 10/10 ✅ 39.6 MiB ≤ 50 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 10/10 ✅ 60.2 MiB ≤ 75 MiB metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory_usage 10/10 ✅ 179 MiB ≤ 200 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 26.7 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

@tobz tobz merged commit a7a8935 into main May 14, 2026
77 checks passed
@tobz tobz deleted the tobz/align-dsd-context-expiry branch May 14, 2026 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/components Sources, transforms, and destinations. area/docs Reference documentation. source/dogstatsd DogStatsD source. type/chore Updates to dependencies or general "administrative" tasks necessary to maintain the codebase/repo.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Align dogstatsd_context_expiry_seconds default and consider making it configurable.

2 participants