Skip to content

cc backend: cmake/meson stdenv units, drvPath-keyed#16

Merged
r-vdp merged 3 commits intomainfrom
cc-backend
Apr 22, 2026
Merged

cc backend: cmake/meson stdenv units, drvPath-keyed#16
r-vdp merged 3 commits intomainfrom
cc-backend

Conversation

@r-vdp
Copy link
Copy Markdown
Member

@r-vdp r-vdp commented Apr 22, 2026

Stacked on #15.

Adds a second Backend for C/C++ projects built with cmake or meson via
stdenv.mkDerivation. Project-grain (one drv = one project); per-TU
incrementality comes from a persistent out-of-tree build dir at
cache.incremental_dir(drv_path) so reconfigure is warm and ninja
rebuilds only the TUs whose .d depfiles changed.

Unit declaration — drvPath sidecar, no overrideAttrs

# bob.nix
let bobCc = import "${bob}/lib/cc.nix"; in
{
  workspaceMembers =;  # rust
  cc = bobCc.units {
    libfoo = { drv = pkgs.libfoo; src = "path/to/libfoo"; };
  };
}

bobCc.unit attaches bobCcSrc as a Nix-level attr (drv // { … }), so
drvPath is unchanged. The cc backend evaluates (import bob.nix {}).cc
once (cached on blake3(bob.nix)) to get a drvPath → src map and uses it
for is_unit and source-change tracking; nothing reaches the drv env. That
means if pkgs.libfoo also appears in some Rust crate's buildInputs,
bob build <rust-root> finds the same drv as a unit and a C edit cascades
through to the .so without any overlay plumbing or drv-hash ripple.

Supporting changes

  • Backend::is_unit gains drv_path and repo_root (the cc unit set is
    declared out-of-band; the drv env alone can't tell). Threaded through
    from_roots / scheduler.
  • Scheduler dispatches unit_name / build_script_hooks /
    output_populated / pipeline per-node so cc and rust units coexist in
    one graph.
  • Rust backend: bob build <profile>.<crate>
    <profile>.workspaceMembers.<crate>.build, so a repo's bob.nix can
    expose multiple cargoNix instances (e.g. prod-tuned vs dev-loop flags)
    under different prefixes.
  • cli: predicate_key folds in bob.nix content so the graph cache
    invalidates when the cc-unit set changes (the root drv path doesn't move).

Limitations

  • unpack/patch are skipped (the build runs against the live worktree), so
    patched derivations are not supported.
  • cc edges are done-gated (pipeline() = None). The early-signal path
    (header staging + a __cc-wrap link-gate + edge-aware policy) is sketched
    in crates/cc/src/lib.rs; the per-TU incrementality here is the
    order-of-magnitude win on its own.

Tested

On a real-world ~925-unit graph with one cc unit (ndl,
aws-neuron-kmdlib) declared:

edit bob build <rust-root>
ndl.c (cold) 3.7s configure+build+install
ndl.c (warm, 1 TU touched) 1.3s
no-op 0.02s (artifact cache hit)

The cc→rust cascade (edit ndl.c.so reflects it) over-invalidates
without the early-cutoff work in the follow-up PR; this PR establishes the
backend and the unit model.

Base automatically changed from worker-drop-warmup-source to main April 22, 2026 13:46
r-vdp added 3 commits April 22, 2026 13:47
Project-grain: a drv opts in via a single `bobCcSrc = "<rel src dir>"`
attr (helper in lib/cc.nix). That attr is the unit marker, the live
source for change detection, and the cmake/meson SOURCE_DIR — no
separate manifest.

Incrementality comes from a persistent out-of-tree build dir at
`cache.incremental_dir(drv_path)`: source edits keep the same dir so
ninja's .d-file tracking rebuilds only changed TUs; any non-source
input change moves drv_path → fresh dir → clean reconfigure.
unpack/patch are skipped so cmake/meson record the live worktree path
(stable across runs) instead of the wiped-per-run NIX_BUILD_TOP.

scheduler: per-node backend dispatch so cc and rust units coexist in
one graph. Edge classification stays dep-side; cc returns
pipeline()=None so cc edges are done-gated. The early-signal path
(header staging + cc-wrap link-gate + edge-aware policy for cc→rust)
is documented in crates/cc/src/lib.rs as the follow-up.
Maps to `<profile>.workspaceMembers.<crate>.build` so a repo's bob.nix can
expose multiple cargoNix instances (e.g. prod-tuned vs dev-loop rustc
flags) under different prefixes. The crate part is still gated on
Cargo.toml membership; the prefix is opaque to bob.
lib/cc.nix now attaches bobCcSrc as a Nix-level attr (drv // { … }), so
drvPath is unchanged and the same drv that appears in a Rust crate's
buildInputs is the one under cc.<name>. The cc backend evaluates
(import bob.nix {}).cc once to get the drvPath→src map and uses it for
is_unit and source-change tracking; nothing reaches the drv env.

This makes the C-edit cascade work without overlays: bob build
<rust-root> finds the cc unit in its closure, overrides::cascade
propagates the eff-hash, and only the cc unit + the linking cdylib
rebuild. Adding/removing a cc unit doesn't ripple through the Rust
drv graph.

core: Backend::is_unit gains drv_path and repo_root; threaded through
from_roots/scheduler. cli: predicate_key folds in bob.nix content so
the graph cache invalidates when the cc-unit set changes (the root drv
path doesn't move, so the cache key wouldn't otherwise).
@r-vdp r-vdp enabled auto-merge April 22, 2026 13:48
@r-vdp r-vdp merged commit b9a00a7 into main Apr 22, 2026
4 checks passed
@r-vdp r-vdp deleted the cc-backend branch April 22, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant