Skip to content

fix(ci): fix stale artifact caching in prebuild step#1838

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits intomainfrom
gyuheon0h/fix-stale-prebuild
Apr 1, 2026
Merged

fix(ci): fix stale artifact caching in prebuild step#1838
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits intomainfrom
gyuheon0h/fix-stale-prebuild

Conversation

@gyuheon0h
Copy link
Copy Markdown
Contributor

@gyuheon0h gyuheon0h commented Apr 1, 2026

What does this PR do?

inner_build_artifact had a simple if artifact_path.exists() { return } check that skipped cargo build entirely when the binary file was on disk, even if source code had changed. This caused bin_test tests to run against stale binaries after code edits.

Changes
rebuild_artifacts (used by the prebuild step): Always invokes cargo build, letting cargo's own dependency tracking handle staleness.
fetch_built_artifacts (used by tests): Only looks up the prebuilt artifact path. If it doesn't exist, the test fails with a message telling the developer to add the artifact to all_prebuild_artifacts(). No silent fallback building that would hide missing prebuild entries and slow down CI.

Motivation

Noticed bin_tests running against stale binaries here: https://gitlab.ddbuild.io/DataDog/apm-reliability/libddprof-build/-/jobs/1559430494

[PROF-14184]

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Run cargo nextest run locally with clean artifacts directory, then making changes to observe rebuild of stale artifacts, then again with no changes to observe cargo handling of non-stale files

[in progress]: doing the same combination of runs as the above local verification but in CI

Copy link
Copy Markdown
Contributor Author

gyuheon0h commented Apr 1, 2026

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 1, 2026

Clippy Allow Annotation Report

Comparing clippy allow annotations between branches:

  • Base Branch: origin/main
  • PR Branch: origin/gyuheon0h/fix-stale-prebuild

Summary by Rule

Rule Base Branch PR Branch Change

Annotation Counts by File

File Base Branch PR Branch Change

Annotation Stats by Crate

Crate Base Branch PR Branch Change
clippy-annotation-reporter 5 5 No change (0%)
datadog-ffe-ffi 1 1 No change (0%)
datadog-ipc 20 20 No change (0%)
datadog-live-debugger 6 6 No change (0%)
datadog-live-debugger-ffi 10 10 No change (0%)
datadog-profiling-replayer 4 4 No change (0%)
datadog-remote-config 3 3 No change (0%)
datadog-sidecar 55 55 No change (0%)
libdd-common 10 10 No change (0%)
libdd-common-ffi 12 12 No change (0%)
libdd-data-pipeline 5 5 No change (0%)
libdd-ddsketch 2 2 No change (0%)
libdd-dogstatsd-client 1 1 No change (0%)
libdd-profiling 13 13 No change (0%)
libdd-telemetry 19 19 No change (0%)
libdd-tinybytes 4 4 No change (0%)
libdd-trace-normalization 2 2 No change (0%)
libdd-trace-obfuscation 8 8 No change (0%)
libdd-trace-utils 15 15 No change (0%)
Total 195 195 No change (0%)

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality.

pub fn rebuild_artifacts<'b>(
crates: &[&'b ArtifactsBuild],
) -> anyhow::Result<HashMap<&'b ArtifactsBuild, PathBuf>> {
static ARTIFACTS: OnceCell<Mutex<HashMap<ArtifactsBuild, PathBuf>>> = OnceCell::new();
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this mutex is needed since this is only called once from the first prebuild step

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I don't think we need it anymore.

@gyuheon0h gyuheon0h changed the title Fix stale prebuilt artifacts fix(ci): fix stale artifact caching in prebuild step Apr 1, 2026
@gyuheon0h gyuheon0h marked this pull request as ready for review April 1, 2026 17:44
@gyuheon0h gyuheon0h requested review from a team as code owners April 1, 2026 17:44
Copy link
Copy Markdown
Contributor

@ekump ekump left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one question about a rustdoc example, but LGTM

@datadog-datadog-prod-us1-2
Copy link
Copy Markdown

datadog-datadog-prod-us1-2 bot commented Apr 1, 2026

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
Patch Coverage: 63.16%
Overall Coverage: 71.51% (-0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4ee2470 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 63.15789% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.50%. Comparing base (ce9506c) to head (4ee2470).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1838      +/-   ##
==========================================
- Coverage   71.50%   71.50%   -0.01%     
==========================================
  Files         426      426              
  Lines       67087    67096       +9     
==========================================
+ Hits        47973    47978       +5     
- Misses      19114    19118       +4     
Components Coverage Δ
libdd-crashtracker 66.01% <ø> (ø)
libdd-crashtracker-ffi 34.09% <ø> (ø)
libdd-alloc 98.77% <ø> (ø)
libdd-data-pipeline 86.20% <ø> (ø)
libdd-data-pipeline-ffi 72.22% <ø> (ø)
libdd-common 79.94% <ø> (ø)
libdd-common-ffi 73.87% <ø> (ø)
libdd-telemetry 62.78% <ø> (ø)
libdd-telemetry-ffi 16.75% <ø> (ø)
libdd-dogstatsd-client 82.64% <ø> (ø)
datadog-ipc 72.56% <ø> (+2.24%) ⬆️
libdd-profiling 82.04% <ø> (-0.02%) ⬇️
libdd-profiling-ffi 64.94% <ø> (ø)
datadog-sidecar 31.35% <ø> (ø)
datdog-sidecar-ffi 11.84% <ø> (ø)
spawn-worker 54.69% <ø> (ø)
libdd-tinybytes 93.16% <ø> (ø)
libdd-trace-normalization 81.71% <ø> (ø)
libdd-trace-obfuscation 87.24% <ø> (ø)
libdd-trace-protobuf 68.25% <ø> (ø)
libdd-trace-utils 88.73% <ø> (ø)
datadog-tracer-flare 86.88% <ø> (ø)
libdd-log 74.69% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts bot commented Apr 1, 2026

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 102.11 MB 102.11 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.76 MB 8.76 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 119.08 MB 119.08 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.36 MB 11.36 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 27.41 MB 27.41 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 80.69 KB 80.69 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 187.83 MB 187.82 MB -0% (-16.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 924.51 MB 924.51 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 9.06 MB 9.06 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 80.69 KB 80.69 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 27.01 MB 27.01 MB --.02% (-8.00 KB) 💪
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 61.33 MB 61.33 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 23.27 MB 23.27 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 81.94 KB 81.94 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 192.37 MB 192.40 MB +.01% (+24.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 909.65 MB 909.65 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.90 MB 6.90 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 81.94 KB 81.94 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 29.12 MB 29.12 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 57.70 MB 57.70 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 88.96 MB 88.96 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 10.34 MB 10.34 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 111.75 MB 111.75 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 12.09 MB 12.09 MB 0% (0 B) 👌

@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot merged commit f7427db into main Apr 1, 2026
94 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot deleted the gyuheon0h/fix-stale-prebuild branch April 1, 2026 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants