feat(cli): #238 two-stage compile-many primitive

[zackees]

Summary

Adds a new fbuild compile-many CLI subcommand that compiles many sketches
against the same board with framework + library archives built once
(stage 1) and per-sketch compile+link fanned out across a thread pool
(stage 2). Implements the primitive described in #238.

fbuild compile-many \
    --board <board> \
    --framework-jobs <N>   # stage 1 parallelism (default: min(cores, 2))
    --sketch-jobs <M>      # stage 2 parallelism (default: cores)
    <sketch1> [<sketch2> ...]

Design

• Stage 1 runs the orchestrator once for the first sketch with
  --framework-jobs workers driving intra-build parallelism. This is the
  memory-heavy stage (framework TU compile + library archives), so we
  keep the knob modest on constrained runners.
• Stage 2 fans out the remaining sketches across --sketch-jobs
  workers. Each worker invokes the orchestrator with jobs=1 since per-
  sketch work is single-TU and the outer thread pool already saturates
  cores. Framework artifacts written to disk by stage 1 are reused via
  the orchestrator's warm-build fast path + zccache.
• Routes through fbuild_build::get_orchestrator(platform) so all
  platform-specific behavior (toolchain resolution, framework
  installation, LDF, link flags, size reporting) lives in one place and
  compile-many automatically picks up future per-platform work without
  re-implementing it.
• Runs in-process from the CLI (no daemon round-trip per sketch),
  matching the issue's design goal: "one process, one toolchain load,
  one LDF run, one framework build".

The implementation is parameterized over a SketchBuilder trait so tests
can inject a mock and exercise the orchestration layer without dragging
in a real toolchain. The production builder (OrchestratorBuilder)
calls get_orchestrator directly.

Concurrent-safety review

• Per-sketch output dirs are unique by construction. The orchestrator
  derives the build root from <project_dir>/.fbuild/build/<env>/<profile>/,
  and each sketch in the input list has its own project_dir, so no
  two workers can race on the same firmware.elf. The test suite
  inserts every observed firmware path into a HashSet and asserts
  uniqueness on each insert.
• zccache is lock-free under concurrent stage-2 workers. fbuild adds
  zero in-process locks around zccache (crates/fbuild-build/src/zccache.rs
  has no Mutex/RwLock/file-lock). Stage-2 workers each invoke
  zccache wrap <gcc> ... as a child process; the upstream zccache
  daemon uses SQLite WAL + content-addressed object files, so concurrent
  reads of the same key never block and concurrent writes of distinct
  keys never block.

Test plan

• uv run soldr cargo build --workspace — clean
• uv run soldr cargo clippy --workspace --all-targets -- -D warnings — clean
• uv run soldr cargo fmt --all -- --check — clean
• uv run soldr cargo test -p fbuild-build — 508 lib tests + 5 new
  integration tests in tests/compile_many_two_stage.rs all pass
• uv run soldr cargo test -p fbuild-cli — 13 tests pass

The new integration tests assert:

• Stage 1 runs exactly once across N sketches; stage 2 produces N-1
  firmware artifacts.
• Results are returned in input order regardless of completion order.
• Stage 2 workers run truly concurrently (proven via a Barrier of
  size N — would deadlock under serial execution).
• Per-sketch firmware paths are unique (filesystem-level assertion).
• Stage-1 failure short-circuits stage 2.
• Single sketch falls through stage 1 only (no stage-2 invocation).

Acceptance gates

The numeric perf gates from #238 (warm <= 80 s, cold <= 110 s on
ubuntu-latest) cannot be measured locally. They will be validated by
the benchmark workflow after this PR lands and a release ships with the
new subcommand.

Local assertions delivered by this PR:

• Stage 1 runs exactly once across N sketches
• Stage 2 parallelism is honored (barrier-of-N concurrency test)
• Per-sketch output directories are unique

Deferred / out-of-scope

• The benchmark workflow update (.github/workflows/benchmark.yml) is
  Step 4 from feat(cli): two-stage compile-many primitive — framework once, sketches in parallel (#112) #238. That file lives on the orphan bench/fastled-examples
  branch, not on main, so it is a separate PR handled by another
  agent — explicitly out of scope here.
• Downstream FastLED ./compile wrapper switch — out of scope per feat(cli): two-stage compile-many primitive — framework once, sketches in parallel (#112) #238.

Closes #238

Summary by CodeRabbit

Release Notes

• New Features

  • Added compile-many command to the CLI for batch compiling multiple sketches with configurable parallelism
  • Two-stage build optimization: framework and libraries build once, then sketches compile in parallel using --framework-jobs and --sketch-jobs flags

• Documentation

  • Added documentation describing the batch compilation workflow

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR adds a new fbuild compile-many subcommand that builds firmware for multiple sketches efficiently by executing shared framework compilation once in stage 1, then compiling and linking individual sketches in parallel during stage 2. The feature includes request/response types, board environment resolution, concurrent worker management with order preservation, comprehensive tests, and CLI integration with configurable parallelism for each stage.

Changes

Two-Stage Sketch Compilation

Layer / File(s)	Summary
API contract and parallelism defaults crates/fbuild-build/src/compile_many.rs, crates/fbuild-build/src/lib.rs, crates/fbuild-build/src/README.md	CompileManyRequest accepts a board, sketch list, and optional framework/sketch job counts. SketchResult and CompileManyResult describe per-sketch and aggregate outcomes. Stage enum discriminates between framework (stage 1) and per-sketch (stage 2) work. Default job counts are clamped to 1–2 for framework (conservative) and 1–cores for sketches (generous).
Build infrastructure and board resolution crates/fbuild-build/src/compile_many.rs	Resolves each sketch's platformio.ini environment for the requested board, determines the Platform from board id, builds each sketch via the platform orchestrator, and persists build logs to per-sketch compile_many.log files.
Two-stage orchestration and concurrent execution crates/fbuild-build/src/compile_many.rs	compile_many and compile_many_with validate inputs, resolve platforms, run stage 1 sequentially on the first sketch to warm caches, and short-circuit on stage-1 failure. Stage 2 fans out remaining sketches across a scoped thread pool with an atomic work queue, writing each result into a fixed index slot to preserve input order.
Unit and integration tests crates/fbuild-build/src/compile_many.rs, crates/fbuild-build/tests/compile_many_two_stage.rs	Unit tests validate default job bounds, board/environment resolution, and error paths. Integration tests use a mock SketchBuilder to verify stage-1 runs exactly once, stage-2 runs for remaining sketches, workers execute concurrently, stage-1 failure skips stage-2, and result order matches input sketch order.
CLI subcommand and user-facing handler crates/fbuild-cli/src/main.rs, crates/fbuild-cli/Cargo.toml	New compile-many subcommand accepts board, framework/sketch job counts, build profile, verbosity, and sketch directories. Handler constructs a CompileManyRequest, applies defaults, runs orchestration on a blocking pool, and prints per-sketch and summary results.

Sequence Diagram

sequenceDiagram
  participant Client
  participant Orchestrator
  participant Stage1Worker
  participant Stage2WorkerPool
  participant FileSystem
  
  Client->>Orchestrator: compile_many(request)
  Orchestrator->>Orchestrator: validate sketches, resolve platforms
  
  Orchestrator->>Stage1Worker: build sketch[0] with framework_jobs
  Stage1Worker->>Stage1Worker: compile framework/libs once
  Stage1Worker->>FileSystem: write compile_many.log
  Stage1Worker-->>Orchestrator: SketchResult (stage 1)
  
  alt Stage 1 Success
    Orchestrator->>Stage2WorkerPool: spawn workers for sketches[1..]
    par Concurrent Workers
      Stage2WorkerPool->>Stage2WorkerPool: worker N: compile sketch[N]
      Stage2WorkerPool->>FileSystem: write per-sketch compile_many.log
      Stage2WorkerPool->>Stage2WorkerPool: preserve order via indexed slots
    end
    Stage2WorkerPool-->>Orchestrator: Vec~SketchResult~ (stage 2)
  else Stage 1 Failure
    Orchestrator-->>Client: CompileManyResult (only stage 1 result)
  end
  
  Orchestrator-->>Client: CompileManyResult (aggregate results + timing)

Loading

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly Related Issues

• feat(cli): two-stage compile-many primitive — framework once, sketches in parallel (#112) #238 — This PR implements the two-stage compile-many primitive with --framework-jobs and --sketch-jobs flags that directly address the design intent and acceptance criteria defined in that issue, including the CLI subcommand, parallel execution, and job configuration.

Poem

🐰 Two stages dance, one swift, one wide—
Framework warms the cache with pride,
Then sketches sprint in parallel streams,
Fulfilling FastLED's build-time dreams! ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat(cli): #238 two-stage compile-many primitive' clearly and specifically summarizes the main change: adding a new CLI feature for the compile-many two-stage build primitive referenced in issue #238.
Linked Issues check	✅ Passed	The PR fully implements the primary objective from #238: a two-stage compile-many primitive with independent --framework-jobs and --sketch-jobs knobs, proper orchestrator integration, concurrent-safety via unique sketch output dirs, result mapping, and comprehensive test coverage.
Out of Scope Changes check	✅ Passed	All changes are directly scoped to implementing the compile-many primitive: new lib/CLI subcommand, request/result types, two-stage execution logic, trait abstraction for builders, and integration tests. No unrelated changes detected.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/issue-238-compile-many

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (2)

crates/fbuild-build/src/compile_many.rs (2)

296-315: ⚡ Quick win

Consider logging when log file persistence fails.

The function silently returns None when directory creation or file write fails. Since these logs help with post-build debugging and benchmark summaries, failures should be logged so users know why log_path is absent.

Add warning logs for failures

 fn write_log_file(
     sketch: &Path,
     env_name: &str,
     profile: BuildProfile,
     build_log: fbuild_core::BuildLog,
 ) -> Option<PathBuf> {
     let build_dir = fbuild_packages::Cache::new(sketch).get_build_dir(env_name, profile);
-    if std::fs::create_dir_all(&build_dir).is_err() {
+    if let Err(e) = std::fs::create_dir_all(&build_dir) {
+        tracing::warn!("failed to create build dir for log: {}", e);
         return None;
     }
     let log_path = build_dir.join("compile_many.log");
     let body = build_log.into_lines().join("\n");
     match std::fs::write(&log_path, body) {
         Ok(()) => Some(log_path),
-        Err(_) => None,
+        Err(e) => {
+            tracing::warn!("failed to write log file {}: {}", log_path.display(), e);
+            None
+        }
     }
 }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/fbuild-build/src/compile_many.rs` around lines 296 - 315, The
write_log_file function currently swallows errors; update it to log failures
instead of silently returning None: when std::fs::create_dir_all(&build_dir)
fails, emit a warning including the build_dir and the error; likewise when
std::fs::write(&log_path, body) fails, emit a warning that includes the log_path
and the error before returning None. Locate these changes in write_log_file
(references: build_dir, log_path, build_log.into_lines()) and use the project's
logging facility (e.g., log::warn! or tracing::warn!) so the error context is
recorded.

---

80-85: 💤 Low value

Optional: Remove redundant .max(1) call.

The .unwrap_or(1) already guarantees the value is at least 1, so the subsequent .max(1) is redundant.

Minor cleanup

 pub fn default_sketch_jobs() -> usize {
     std::thread::available_parallelism()
         .map(|n| n.get())
         .unwrap_or(1)
-        .max(1)
 }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/fbuild-build/src/compile_many.rs` around lines 80 - 85, The function
default_sketch_jobs contains a redundant .max(1) because
std::thread::available_parallelism().map(|n| n.get()).unwrap_or(1) already
yields at least 1; remove the trailing .max(1) from default_sketch_jobs to
simplify the return expression while keeping behavior unchanged (refer to the
function default_sketch_jobs).

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/fbuild-build/src/compile_many.rs`:
- Around line 296-315: The write_log_file function currently swallows errors;
update it to log failures instead of silently returning None: when
std::fs::create_dir_all(&build_dir) fails, emit a warning including the
build_dir and the error; likewise when std::fs::write(&log_path, body) fails,
emit a warning that includes the log_path and the error before returning None.
Locate these changes in write_log_file (references: build_dir, log_path,
build_log.into_lines()) and use the project's logging facility (e.g., log::warn!
or tracing::warn!) so the error context is recorded.
- Around line 80-85: The function default_sketch_jobs contains a redundant
.max(1) because std::thread::available_parallelism().map(|n|
n.get()).unwrap_or(1) already yields at least 1; remove the trailing .max(1)
from default_sketch_jobs to simplify the return expression while keeping
behavior unchanged (refer to the function default_sketch_jobs).

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 5e5f7e3f-78c7-423c-8492-4507239fa5eb

📥 Commits

Reviewing files that changed from the base of the PR and between 6d4c0f7 and 900787d.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (6)

• crates/fbuild-build/src/README.md
• crates/fbuild-build/src/compile_many.rs
• crates/fbuild-build/src/lib.rs
• crates/fbuild-build/tests/compile_many_two_stage.rs
• crates/fbuild-cli/Cargo.toml
• crates/fbuild-cli/src/main.rs