Skip to content

feat(sidecar)!: support appsec helper-rust integration with sidecar - #2310

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 13 commits into
mainfrom
glopes/sidecar-comm
Aug 25, 2026
Merged

feat(sidecar)!: support appsec helper-rust integration with sidecar#2310
gh-worker-dd-mergequeue-cf854d[bot] merged 13 commits into
mainfrom
glopes/sidecar-comm

Conversation

@cataphract

Copy link
Copy Markdown
Contributor

See DataDog/dd-trace-php#3725

Also fix zigbuild windows builds, for easier testing of windows when running Linux (zigbuild + wine).

Read Cargo's target OS and family in the crashtracker build script
instead of build-host cfg values. Build the CXX bridge for its target,
keep Unix-only C support and test libraries off Windows, and select the
dynamic CRT for Windows targets.

Teach the spawn worker trampoline build to distinguish MSVC and GNU
Windows environments, skip Unix libraries, and use compatible C++
flags without promoting MinGW warnings to errors. Use lowercase SDK
header names and enable the LibraryLoader API for Windows crashtracker
tests.
Add an AppSec backend factory that an embedding application can register
from a custom sidecar entry point. Start and stop that backend with the
sidecar listener, and replace the dynamically loaded helper library and
its private socket configuration with a sidecar request/response RPC.

Expose the RPC through the blocking client and C FFI. Associate helper
client IDs with sidecar connections, notify the backend when connections
or sessions close, and reject stale or conflicting IDs after a restart.
Send helper-targeted events to the configured AppSec log.
Teach the IPC service macro to generate a serialize-only client request
enum when parameters declare alternate #[ClientType] representations.
Add blocking channel calls that serialize borrowed request values.

Use byte slices for AppSec session IDs and payloads, and retain one
request across transport retries. The server still decodes the existing
owned request type, avoiding request allocations in FFI and clones on
retry.
Format embedded AppSec helper records with a UTC timestamp, level,
message, and module. Normalize records bridged from the log crate before
filtering helper targets and rendering their module paths.

Write helper records to their configured file and exclude other sidecar
records from it. Treat "<sidecar log>" as a request to use the main
sidecar log without creating a separate helper log layer.
Expose a client factory to the registered AppSec backend so embedded
components can enqueue actions into the sidecar telemetry receiver
without using FFI. Bind each client to an instance, service, and
environment while allowing its application metadata to be rebound.

Refresh cached telemetry clients on lookup so active clients are not
expired while the in-process path is in use.
Move AppSec lifecycle management into the sidecar server and add a
sidecar request that starts the registered backend on demand. Coordinate
concurrent initialization and ensure shutdown is owned by one caller.

This lets thread-mode listeners receive AppSec configuration after a
client connects, while process-mode sidecars still start from daemon
configuration.
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Clippy Allow Annotation Report

Tracked Clippy allow annotations changed vs main: ✅ -2 (20 → 18)

Rule Base PR Δ
unwrap_used 18 16 ✅ -2
By file and crate

By file

File Base PR Δ
datadog-sidecar/src/unix.rs 4 2 ✅ -2

By crate

Crate Base PR Δ
datadog-sidecar 45 43 ✅ -2

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. Panic-inducing macros in particular should be avoided. In the future, this report may become a PR-blocking quality gate.

@cataphract cataphract changed the title Support appsec helper-rust integration with sidecar feat(sidecar)!: support appsec helper-rust integration with sidecar Aug 3, 2026
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 56.45%
Overall Coverage: 76.66% (-0.04%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: af35296 | Docs | View more details | Give us feedback!

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7112284f0b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread datadog-sidecar/src/service/sidecar_server.rs Outdated
Comment thread datadog-sidecar-ffi/src/lib.rs Outdated
Comment thread datadog-sidecar/src/service/sidecar_server.rs
Comment thread datadog-sidecar/src/service/sidecar_server.rs
@dd-octo-sts

dd-octo-sts Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 89.69 MB 89.69 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.26 MB 8.26 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.07 MB 11.07 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 100.91 MB 100.91 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 26.65 MB 26.65 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 94.96 KB 94.96 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 180.95 MB 180.95 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 773.55 MB 773.55 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.74 MB 8.74 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 94.96 KB 94.96 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 25.68 MB 25.68 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 51.15 MB 51.15 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 23.23 MB 23.23 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 96.45 KB 96.45 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 185.86 MB 185.85 MB -0% (-8.00 KB) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 760.32 MB 760.32 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.75 MB 6.75 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 96.45 KB 96.45 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 27.62 MB 27.62 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 48.67 MB 48.67 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 79.96 MB 79.96 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 9.17 MB 9.17 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 95.67 MB 95.67 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.19 MB 11.19 MB 0% (0 B) 👌

@pr-commenter

pr-commenter Bot commented Aug 3, 2026

Copy link
Copy Markdown

Benchmarks

Comparison

Benchmark execution time: 2026-08-25 19:19:17

Comparing candidate commit af35296 in PR branch glopes/sidecar-comm with baseline commit 1766e7e in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 152 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:profile_add_sample2_frames_x1000

  • 🟥 execution_time [+31.329µs; +31.542µs] or [+4.514%; +4.545%]

Benchmark execution time: 2026-08-25 19:27:36

Comparing candidate commit af35296 in PR branch glopes/sidecar-comm with baseline commit 1766e7e in branch main.

Found 3 performance improvements and 0 performance regressions! Performance is the same for 165 metrics, 10 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:credit_card/is_card_number/378282246310005

  • 🟩 execution_time [-12.324µs; -12.119µs] or [-14.956%; -14.707%]
  • 🟩 throughput [+2095945.792op/s; +2130507.039op/s] or [+17.271%; +17.555%]

scenario:profiler_attached/fast_path_system/4096

  • 🟩 execution_time [-6.586ns; -6.432ns] or [-6.598%; -6.443%]

Candidate

Omitted due to size.

Baseline

Omitted due to size.

Comment thread datadog-sidecar/src/service/sidecar_server.rs Outdated
Comment thread datadog-sidecar/src/appsec.rs
Comment thread datadog-sidecar/src/log.rs
Comment thread datadog-sidecar/src/service/sidecar_interface.rs

@bwoebi bwoebi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good for now. I think we should eventually port the appsec backend code to libdatadog as well, and then we can clean this up even more.

@cataphract

Copy link
Copy Markdown
Contributor Author

Blocked by #2410

@cataphract

Copy link
Copy Markdown
Contributor Author

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Aug 25, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-08-25 20:09:45 UTC ℹ️ Start processing command /merge


2026-08-25 20:09:52 UTC ℹ️ MergeQueue: Pull request is not mergeable yet

It will be processed automatically as soon as GitHub reports it as mergeable. View in MergeQueue UI.

  • Run /code blockers to see what is blocking it.
  • Run /remove to cancel it.

2026-08-25 20:15:11 UTC ℹ️ MergeQueue: merge request added to the queue

The expected merge time in main is approximately 53m (p90).


2026-08-25 20:50:02 UTC ℹ️ MergeQueue: This merge request was merged

@cataphract

Copy link
Copy Markdown
Contributor Author

/code blockers

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Aug 25, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-08-25 20:13:23 UTC ℹ️ Start processing command /code blockers


2026-08-25 20:13:24 UTC ℹ️ Devflow:

Checking merge blockers for #2310...


2026-08-25 20:13:27 UTC ℹ️ Devflow: /code blockers

No merge blockers detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants