feat(crashtracking): send debug log when no data is received at all - #2321
Conversation
gyuheon0h
changed the title
📚 Documentation Check Results📦
|
🔒 Cargo Deny Results📦
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a8e6a53de3
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
gh-worker-dd-devflow-36fce6
Bot
added
mergequeue-status: removed
and removed
mergequeue-status: waiting
labels
BenchmarksComparisonBenchmark execution time: 2026-08-05 19:19:32 Comparing candidate commit 6dc1fe2 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 1 metrics, 0 unstable metrics.
|
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 6dc1fe2 | 1785956872 | gyuheon0h/ct-no-data-debug |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| receiver_entry_point/report/2644 | execution_time | 3.586ms | 3.621ms ± 0.017ms | 3.618ms ± 0.010ms | 3.631ms | 3.650ms | 3.671ms | 3.692ms | 2.04% | 1.142 | 1.947 | 0.47% | 0.001ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| receiver_entry_point/report/2644 | execution_time | [3.619ms; 3.623ms] or [-0.065%; +0.065%] | None | None | None |
Baseline
Baseline benchmark details
Group 1
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 9ac89b9 | 1785954810 | main |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| receiver_entry_point/report/2644 | execution_time | 3.492ms | 3.561ms ± 0.060ms | 3.528ms ± 0.020ms | 3.635ms | 3.663ms | 3.680ms | 3.691ms | 4.62% | 0.763 | -1.122 | 1.68% | 0.004ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| receiver_entry_point/report/2644 | execution_time | [3.553ms; 3.569ms] or [-0.234%; +0.234%] | None | None | None |
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 6dc1fe2 | Docs | Datadog PR Page | Give us feedback! |
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
2238baa to
ba607a3
Compare
|
@codex r |
|
To use Codex here, create a Codex account and connect to github. |
|
@codex r |
|
To use Codex here, create a Codex account and connect to github. |
ba607a3 to
f9468bc
Compare
9439a82 to
0c64e83
Compare
0c64e83 to
6dc1fe2
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6dc1fe271a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| std::env::set_var( | ||
| "DD_TRACE_AGENT_URL", | ||
| format!("http://{}", listener.local_addr().unwrap()), | ||
| ); |
There was a problem hiding this comment.
When these tests are run by cargo test/libtest or any harness that does not isolate environment per test, this set_var changes a process-global endpoint and never restores it. Other receiver tests now create DebugLogger::new(None, None) and can emit via the env-derived endpoint on timeout/incomplete paths, so they may race with this mock listener or inherit its closed address after this test, causing order-dependent telemetry failures; please guard and restore the variable or serialize env users.
AGENTS.md reference: AGENTS.md:L70-L75
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
nextest isolates env per test
| debug_logger | ||
| .emit_and_wait( | ||
| ReceiverIssue::NoData, | ||
| &builder.uuid.to_string(), | ||
| "Receiver received no data".to_string(), | ||
| LogLevel::Warn, | ||
| ) | ||
| .await; |
There was a problem hiding this comment.
For a long-lived socket receiver (async_receiver_entry_point_unix_socket(..., one_shot=false)), the accept loop handles one connection at a time, so an empty connect/close now awaits this best-effort telemetry upload before accepting another report. If the agent endpoint is slow or unreachable, each empty connection can stall the receiver for the telemetry timeout and delay subsequent crash reports; the no-data signal needs to be spawned or otherwise bounded for that mode.
AGENTS.md reference: AGENTS.md:L70-L75
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
There can only be one report from one process? Or is this referring to sidecar process listening to many processes??
gleocadie
left a comment
gleocadie
left a comment
There was a problem hiding this comment.
LGTM
leave the AI comments and Rust thingy to experts :)
…mote_config/agentless_fetcher * 'main' of github.com:DataDog/libdatadog: feat(data-pipeline)!: add flush_and_close to the trace buffer (#2313) refactor(rc)!: make conversion from RemoteConfigProduct back and forth generally available (#2325) chore: release v40.0.0 (#2326) feat(datadog-ffe): server-side EVP flagevaluation payload + bincode-safe sidecar delivery (#2117) feat(data-pipeline): set structured span values (#2300) feat(telemetry)!: Add Installation signature and AppProduct changes payloads (#2213) fix(libdd-trace-utils): apply SpanLink flags masking when v0.5 json encoding (#2314) feat(data-pipeline): encode structured values (#2304) feat(crashtracking): send debug log when no data is received at all (#2321) chore: release v39.0.0 (#2324) feat(data-pipeline): add span links FFI (#2305) fix(crashtracking): check fields and exclude uuid for `has_data` (#2322) feat(data-pipeline): add span events FFI (#2301)
2 participants
What does this PR do?
Creates a
DebugLoggerfor use by the receiver. This debug logger has the ability to send telemetry logs always, whetherCrashtrackerConfig,Metadatais received or not. As new data arrives to the receiver, the debug logger self builds itself. This allows us to send debug logs from the receiver even if no data has been received by the receiver yet.Motivation
We have cases where customers crash and the receiver spawns, but there is not even a crash ping.
We should log this. Previously, we waited until
MetadataandCrashtrackerConfigwas received by the receiver before being able to send anything. However, theTelemetryCrashUploadertechnically needs nothing to be able to send logs -- it has fallback paths that derive the endpoint through environment variables. We should take advantage of thisAdditional Notes
Anything else we should know when reviewing?
How to test the change?
Describe here in detail how the change can be validated.