Skip to content

Exclude OTLP header and Datadog key configs from telemetry - #3961

Merged
bwoebi merged 12 commits into
masterfrom
brian.marks/omit-sensitive-config-telemetry
Sep 4, 2026
Merged

Exclude OTLP header and Datadog key configs from telemetry#3961
bwoebi merged 12 commits into
masterfrom
brian.marks/omit-sensitive-config-telemetry

Conversation

@bm1549

@bm1549 bm1549 commented Jun 6, 2026

Copy link
Copy Markdown
Collaborator

Description

Stops configuration telemetry from reporting DD_API_KEY or OTLP exporter header values.

DD_API_KEY now uses a sensitivity flag carried through ZAI config metadata. The OTLP header configs remain in supported-config metadata with sensitive: true, but are rejected at the PHP whitelist and native ingestion boundaries. DD_TRACE_ENABLED remains omitted separately because its shutdown-time value is not meaningful.

The metadata generator now reads sensitivity from all config headers and uses comment/string-aware parsing. Its CI self-test covers parser edge cases. Runtime coverage checks the native bypass path and a future OTLP traces header variant, with safe configs as positive controls.

Reviewer checklist

  • Test coverage seems ok.
  • Appropriate labels assigned.

@bm1549 bm1549 added the AI Generated Largely based on code generated by an AI or LLM. This label is the same across all dd-trace-* repos label Jun 6, 2026
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jun 6, 2026

Copy link
Copy Markdown

Pipelines  Tests

Unblock PR with BitsAI

⚠️ Warnings

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 7 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-php | Loader test on arm64 libc: [8.4, zts, arm64] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-php | merge-gate — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-php | test_extension_ci: [7.4] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

View all 7 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 60.65% (-0.01%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 1314586 | Docs | View more details | Give us feedback!

@pr-commenter

pr-commenter Bot commented Jun 6, 2026

Copy link
Copy Markdown

Benchmarks [ tracer ]

Benchmark execution time: 2026-09-02 16:12:53

Comparing candidate commit 1314586 in PR branch brian.marks/omit-sensitive-config-telemetry with baseline commit 2ad9386 in branch master.

Found 4 performance improvements and 3 performance regressions! Performance is the same for 185 metrics, 2 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:HookBench/benchHookOverheadTraceFunction-opcache

  • 🟩 execution_time [-10.162µs; -3.919µs] or [-5.892%; -2.272%]

scenario:MessagePackSerializationBench/benchMessagePackSerialization-opcache

  • 🟩 execution_time [-3.522µs; -2.458µs] or [-3.057%; -2.134%]

scenario:SamplingRuleMatchingBench/benchRegexMatching2

  • 🟥 execution_time [+91.459ns; +150.941ns] or [+6.277%; +10.360%]

scenario:SamplingRuleMatchingBench/benchRegexMatching3

  • 🟥 execution_time [+30.964ns; +108.836ns] or [+2.066%; +7.263%]

scenario:SamplingRuleMatchingBench/benchRegexMatching4

  • 🟥 execution_time [+43.977ns; +91.023ns] or [+2.977%; +6.161%]

scenario:SpanBench/benchOpenTelemetryInteroperability

  • 🟩 execution_time [-222.527µs; -218.960µs] or [-52.769%; -51.923%]

scenario:TraceAnnotationsBench/benchTraceAnnotationOverhead-opcache

  • 🟩 execution_time [-9.750µs; -4.692µs] or [-5.393%; -2.596%]

@bm1549
bm1549 force-pushed the brian.marks/omit-sensitive-config-telemetry branch 5 times, most recently from b468a83 to b5e01e5 Compare June 12, 2026 18:18
@pr-commenter

pr-commenter Bot commented Jun 12, 2026

Copy link
Copy Markdown

Benchmarks [ profiler ]

Benchmark execution time: 2026-09-02 15:23:55

Comparing candidate commit 1314586 in PR branch brian.marks/omit-sensitive-config-telemetry with baseline commit 2ad9386 in branch master.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 25 metrics, 10 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:php-profiler-timeline-memory-with-profiler

  • 🟩 cpu_system_time [-44.433ms; -15.741ms] or [-6.730%; -2.384%]

Comment thread ext/telemetry.c Outdated
@bm1549
bm1549 marked this pull request as ready for review July 29, 2026 22:08
@bm1549
bm1549 requested review from a team as code owners July 29, 2026 22:08
@bm1549
bm1549 requested review from LobeTia and removed request for a team July 29, 2026 22:08
@bm1549
bm1549 force-pushed the brian.marks/omit-sensitive-config-telemetry branch from 16486c5 to ea536a1 Compare July 31, 2026 19:50
bm1549 and others added 10 commits August 27, 2026 17:56
Add a `sensitive` flag to the configuration definition (zai_config_entry and its
memoized entry) so a DD_* configuration can be marked in its CONFIG(...)
declaration in ext/configuration.h. The configuration-telemetry enqueue loop
over the DD_* config table skips entries whose flag is set; DD_API_KEY and
DD_TRACE_ENABLED carry it.

Remove the OTLP header configurations (OTEL_EXPORTER_OTLP_HEADERS,
OTEL_EXPORTER_OTLP_METRICS_HEADERS, OTEL_EXPORTER_OTLP_LOGS_HEADERS) from the
OpenTelemetry SDK configuration whitelist so they are not tracked for telemetry.

Derive the "sensitive": true markers in metadata/supported-configurations.json
from the flag in ext/configuration.h, plus the OTLP header variants the
generator lists.

Add a .phpt test and extend the loader functional test to assert these
configuration values do not appear in the enqueued configuration telemetry
while non-sensitive configurations remain reported.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bm1549
bm1549 force-pushed the brian.marks/omit-sensitive-config-telemetry branch from ea536a1 to 00d6bf3 Compare August 28, 2026 13:19
@bm1549
bm1549 requested a review from a team as a code owner August 28, 2026 13:19
@bwoebi
bwoebi force-pushed the brian.marks/omit-sensitive-config-telemetry branch from 2b4fb63 to 54ad336 Compare September 2, 2026 12:39

@bwoebi bwoebi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've rewrote the code to be much simpler, I think we can now merge that.

@bwoebi
bwoebi merged commit ec268d4 into master Sep 4, 2026
2163 of 2172 checks passed
@bwoebi
bwoebi deleted the brian.marks/omit-sensitive-config-telemetry branch September 4, 2026 02:44
@github-actions github-actions Bot added this to the 1.25.0 milestone Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI Generated Largely based on code generated by an AI or LLM. This label is the same across all dd-trace-* repos

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants