Skip to content

chore(app): ignore no-op updates to logging filter directives#1657

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
tobz/log-directive-ignore-duplicate
May 14, 2026
Merged

chore(app): ignore no-op updates to logging filter directives#1657
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
tobz/log-directive-ignore-duplicate

Conversation

@tobz
Copy link
Copy Markdown
Member

@tobz tobz commented May 14, 2026

Summary

As stated in the PR title.

When updating the "base filter" for the logging filter directives, we get a big long message about it that looks like this:

2026-05-14 18:01:41 UTC | DATAPLANE | INFO | (lib/saluki-app/src/logging/api.rs:287) | directives:"prometheus_exposition=info,datadog_agent_commons=info,saluki_components=info,memory_accounting=info,containerd_protos=info,agent_data_plane=info,saluki_metadata=info,saluki_metrics=info,saluki_context=info,process_memory=info,datadog_protos=info,saluki_config=info,saluki_common=info,stringtheory=info,saluki_error=info,saluki_core=info,otlp_protos=info,saluki_tls=info,saluki_env=info,saluki_app=info,saluki_api=info,saluki_io=info,ddsketch=info,ottl=info" | Updated base log filtering directives.

That's a lot of output... but it's even worse because this would be printed even if the update was a no-op and the "new" filter was identical to the current one. Oof.

This PR simply runs a check when an "update base filter" operation is received, making sure that the new base filter is different from the current base filter. If it's not, we ignore it.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

How did you test this PR?

Ran the adp-config-stream integration test before and after. Observed that before, we saw the above message, and after applying the change, we no longer saw it.

(This test is relevant here because we would only trigger base filter updates after bootstrap when getting the configuration from the Core Agent, since we do a single one-time update after getting the configuration to make sure our logging level matches the configured log level of the Core Agent.)

References

DADP-2

@tobz tobz added the type/chore Updates to dependencies or general "administrative" tasks necessary to maintain the codebase/repo. label May 14, 2026
@tobz tobz requested a review from a team as a code owner May 14, 2026 18:27
@dd-octo-sts dd-octo-sts Bot added the area/observability Internal observability of ADP and Saluki. label May 14, 2026
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented May 14, 2026

Binary Size Analysis (Agent Data Plane)

Target: c1313d0 (baseline) vs ca0f823 (comparison) diff
Analysis Type: Stripped binaries (debug symbols excluded)
Baseline Size: 37.31 MiB
Comparison Size: 37.32 MiB
Size Change: +1.82 KiB (+0.00%)
Pass/Fail Threshold: +5%
Result: PASSED ✅

Changes by Module

Module File Size Symbols
anon.8d8811c38c78989b07412358014ad95b.2.llvm.17737639912621458350 +2.91 KiB 1
anon.b1b224363bd8cc54b1f66cbf9d11eab3.59.llvm.1965883548458184779 -2.91 KiB 1
anon.41b0b8befc3118c3a2b0f17ec06872eb.9.llvm.17862187855904175761 -1.93 KiB 1
anon.bdadb00871588c874a55b8b73ce579a9.0.llvm.8997752761958168434 +1.93 KiB 1
anon.fd7b529fe56f46398ecdea2e5e277011.5.llvm.6425510092197803827 +1.69 KiB 1
anon.8d73643f7c50d1f3e6a42b28ce415d61.97.llvm.7686234851778971964 -1.69 KiB 1
[Unmapped] -1.62 KiB 1
anon.ff7950eb5cfb17ddfcd9e2b2a5594f26.82.llvm.9873202471916192885 +1.50 KiB 1
anon.b0aa5649cb3450213871fe59dcabc61d.15.llvm.299490569075470158 -1.41 KiB 1
anon.1f92070954cd764eaf703c4d91d9aa51.110.llvm.4697276323575673218 +1.34 KiB 1
anon.008f2eb7666b60adbf212aa1a1a9babf.1039.llvm.1675885107082596853 -1.34 KiB 1
anon.ff7950eb5cfb17ddfcd9e2b2a5594f26.97.llvm.9873202471916192885 +1.24 KiB 1
anon.ff7950eb5cfb17ddfcd9e2b2a5594f26.97.llvm.12774116853458931105 -1.24 KiB 1
tokio +1.19 KiB 26
anon.fd12fc45df31a30922855d17b6acc587.143.llvm.17639715875876597146 -1.13 KiB 1
anon.1655e58d66d1ef1230538706b7bd06b2.59.llvm.13790112841392064238 +1.13 KiB 1
anon.d6ea270b648dfac6661a1268469116b7.8.llvm.17970089049851553961 +1.11 KiB 1
anon.b1b224363bd8cc54b1f66cbf9d11eab3.73.llvm.1965883548458184779 -1.10 KiB 1
anon.fda1b3533837ce22d4b7e19c78568bdb.778.llvm.10227848518017916504 +1000 B 1
anon.6c6fb0a3124c685116a3355f068f52b9.4.llvm.3122426268597355725 -1000 B 1

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW] +2.91Ki  [NEW]      +2    anon.8d8811c38c78989b07412358014ad95b.2.llvm.17737639912621458350
  +0.1% +2.24Ki  +0.1% +1.57Ki    [1592 Others]
  [NEW] +1.93Ki  [NEW]     +96    anon.bdadb00871588c874a55b8b73ce579a9.0.llvm.8997752761958168434
  [NEW] +1.69Ki  [NEW]     +97    anon.fd7b529fe56f46398ecdea2e5e277011.5.llvm.6425510092197803827
  [NEW] +1.61Ki  [NEW]     +47    _<http_body_util::combinators::map_err::MapErr<B,F> as http_body::Body>::size_hint::hdada2fb9bb0c3934
  [NEW] +1.50Ki  [NEW]     +92    anon.ff7950eb5cfb17ddfcd9e2b2a5594f26.82.llvm.9873202471916192885
  [NEW] +1.42Ki  [NEW]     +33    core::ptr::drop_in_place<http_body_util::combinators::map_err::MapErr<tonic::codec::encode::EncodeBody<tonic_prost::codec::ProstEncoder<containerd_protos::containerd_include::containerd::services::namespaces::v1::ListNamespacesRequest>,tokio_stream::stream_ext::map::Map<tokio_stream::once::Once<containerd_protos::containerd_include::containerd::services::namespaces::v1::ListNamespacesRequest>,core::result::Result<containerd_protos::containerd_include::containerd::services::namespaces::v1::ListNamespacesRequest,tonic::status::Status>::Ok>>,tonic::status::Status::map_error<tonic::status::Status>>>::h18819913ef052bd7
  [NEW] +1.34Ki  [NEW]     +96    anon.1f92070954cd764eaf703c4d91d9aa51.110.llvm.4697276323575673218
  [NEW] +1.24Ki  [NEW]     +91    anon.ff7950eb5cfb17ddfcd9e2b2a5594f26.97.llvm.9873202471916192885
  [NEW] +1.13Ki  [NEW]     +74    anon.1655e58d66d1ef1230538706b7bd06b2.59.llvm.13790112841392064238
  [NEW] +1.11Ki  [NEW]     +42    anon.d6ea270b648dfac6661a1268469116b7.8.llvm.17970089049851553961
  [DEL] -1.13Ki  [DEL]     -74    anon.fd12fc45df31a30922855d17b6acc587.143.llvm.17639715875876597146
  [DEL] -1.24Ki  [DEL]     -91    anon.ff7950eb5cfb17ddfcd9e2b2a5594f26.97.llvm.12774116853458931105
  [DEL] -1.34Ki  [DEL]     -96    anon.008f2eb7666b60adbf212aa1a1a9babf.1039.llvm.1675885107082596853
  [DEL] -1.41Ki  [DEL]     -92    anon.b0aa5649cb3450213871fe59dcabc61d.15.llvm.299490569075470158
  [DEL] -1.42Ki  [DEL]     -33    core::ptr::drop_in_place<http_body_util::combinators::map_err::MapErr<tonic::codec::encode::EncodeBody<tonic_prost::codec::ProstEncoder<datadog_protos::agent_include::datadog::remoteagent::v1::RefreshRemoteAgentRequest>,tokio_stream::stream_ext::map::Map<tokio_stream::once::Once<datadog_protos::agent_include::datadog::remoteagent::v1::RefreshRemoteAgentRequest>,core::result::Result<datadog_protos::agent_include::datadog::remoteagent::v1::RefreshRemoteAgentRequest,tonic::status::Status>::Ok>>,tonic::status::Status::map_error<tonic::status::Status>>>::hbfc7b7eee5761dcd
  [DEL] -1.61Ki  [DEL]     -47    _<http_body_util::combinators::map_err::MapErr<B,F> as http_body::Body>::size_hint::hd3a85808fa15fdf4
 -23.9% -1.62Ki  [ = ]       0    [Unmapped]
  [DEL] -1.69Ki  [DEL]     -97    anon.8d73643f7c50d1f3e6a42b28ce415d61.97.llvm.7686234851778971964
  [DEL] -1.93Ki  [DEL]     -96    anon.41b0b8befc3118c3a2b0f17ec06872eb.9.llvm.17862187855904175761
  [DEL] -2.91Ki  [DEL]      -2    anon.b1b224363bd8cc54b1f66cbf9d11eab3.59.llvm.1965883548458184779
  +0.0% +1.82Ki  +0.0% +1.61Ki    TOTAL

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented May 14, 2026

Regression Detector (Agent Data Plane)

Run ID: a23db3c6-1822-4c83-9c16-9f65bb40c023
Baseline: c1313d06 · Comparison: ca0f8238 · Diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (35)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
dsd_uds_512kb_3k_contexts_cpu (erratic) cpu ⚪ +12.94 metrics profiles logs
dsd_uds_500mb_3k_contexts_throughput throughput ⚪ -1.96 metrics profiles logs
otlp_ingest_logs_5mb_memory (ignored) memory ⚪ +1.69 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_cpu (erratic) cpu ⚪ +1.58 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_cpu (erratic) cpu ⚪ +1.23 metrics profiles logs
dsd_uds_100mb_3k_contexts_cpu (erratic) cpu ⚪ +1.07 metrics profiles logs
dsd_uds_1mb_3k_contexts_cpu (erratic) cpu ⚪ +1.01 metrics profiles logs
otlp_ingest_traces_5mb_cpu (erratic) cpu ⚪ +0.96 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_memory memory ⚪ +0.17 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_throughput throughput ⚪ -0.16 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_memory memory ⚪ +0.14 metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory ⚪ +0.14 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_throughput throughput ⚪ -0.05 metrics profiles logs
dsd_uds_500mb_3k_contexts_memory memory ⚪ +0.02 metrics profiles logs
otlp_ingest_metrics_5mb_cpu (erratic) cpu ⚪ +0.02 metrics profiles logs
dsd_uds_512kb_3k_contexts_memory memory ⚪ +0.02 metrics profiles logs
otlp_ingest_traces_5mb_throughput throughput ⚪ -0.02 metrics profiles logs
dsd_uds_10mb_3k_contexts_memory memory ⚪ +0.01 metrics profiles logs
dsd_uds_1mb_3k_contexts_throughput throughput ⚪ -0.00 metrics profiles logs
dsd_uds_100mb_3k_contexts_throughput throughput ⚪ +0.00 metrics profiles logs
dsd_uds_512kb_3k_contexts_throughput throughput ⚪ +0.00 metrics profiles logs
otlp_ingest_logs_5mb_throughput (ignored) throughput ⚪ +0.01 metrics profiles logs
dsd_uds_100mb_3k_contexts_memory memory ⚪ -0.02 metrics profiles logs
dsd_uds_10mb_3k_contexts_throughput throughput ⚪ +0.02 metrics profiles logs
otlp_ingest_metrics_5mb_throughput throughput ⚪ +0.03 metrics profiles logs
quality_gates_rss_idle memory ⚪ -0.08 metrics profiles logs
quality_gates_rss_dsd_low memory ⚪ -0.19 metrics profiles logs
quality_gates_rss_dsd_medium memory ⚪ -0.21 metrics profiles logs
otlp_ingest_traces_5mb_memory memory ⚪ -0.26 metrics profiles logs
quality_gates_rss_dsd_heavy memory ⚪ -0.28 metrics profiles logs
dsd_uds_500mb_3k_contexts_cpu (erratic) cpu ⚪ -0.33 metrics profiles logs
dsd_uds_1mb_3k_contexts_memory memory ⚪ -0.38 metrics profiles logs
otlp_ingest_logs_5mb_cpu (ignored) cpu ⚪ -2.44 metrics profiles logs
otlp_ingest_metrics_5mb_memory memory ⚪ -3.10 metrics profiles logs
dsd_uds_10mb_3k_contexts_cpu (erratic) cpu ⚪ -3.84 metrics profiles logs
Bounds Checks: ✅ Passed (5)
experiment check replicates observed links
quality_gates_rss_dsd_heavy memory_usage 10/10 ✅ 123 MiB ≤ 140 MiB metrics profiles logs
quality_gates_rss_dsd_low memory_usage 10/10 ✅ 39.6 MiB ≤ 50 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 10/10 ✅ 60 MiB ≤ 75 MiB metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory_usage 10/10 ✅ 178 MiB ≤ 200 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 27 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit f35b1e8 into main May 14, 2026
79 checks passed
dd-octo-sts Bot pushed a commit that referenced this pull request May 14, 2026
## Summary

As stated in the PR title.

When updating the "base filter" for the logging filter directives, we get a big long message about it that looks like this:

> 2026-05-14 18:01:41 UTC | DATAPLANE | INFO | (lib/saluki-app/src/logging/api.rs:287) | directives:"prometheus_exposition=info,datadog_agent_commons=info,saluki_components=info,memory_accounting=info,containerd_protos=info,agent_data_plane=info,saluki_metadata=info,saluki_metrics=info,saluki_context=info,process_memory=info,datadog_protos=info,saluki_config=info,saluki_common=info,stringtheory=info,saluki_error=info,saluki_core=info,otlp_protos=info,saluki_tls=info,saluki_env=info,saluki_app=info,saluki_api=info,saluki_io=info,ddsketch=info,ottl=info" | Updated base log filtering directives.

That's a _lot_ of output... but it's even worse because this would be printed even if the update was a no-op and the "new" filter was identical to the current one. Oof.

This PR simply runs a check when an "update base filter" operation is received, making sure that the new base filter is different from the current base filter. If it's not, we ignore it.

## Change Type
- [ ] Bug fix
- [ ] New feature
- [x] Non-functional (chore, refactoring, docs)
- [ ] Performance

## How did you test this PR?

Ran the `adp-config-stream` integration test before and after. Observed that before, we saw the above message, and after applying the change, we no longer saw it.

(This test is relevant here because we would only trigger base filter updates after bootstrap when getting the configuration from the Core Agent, since we do a single one-time update after getting the configuration to make sure our logging level matches the configured log level of the Core Agent.)

## References

DADP-2

Co-authored-by: toby.lawrence <toby.lawrence@datadoghq.com> f35b1e8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/observability Internal observability of ADP and Saluki. mergequeue-status: done type/chore Updates to dependencies or general "administrative" tasks necessary to maintain the codebase/repo.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants