Skip to content

#205 follow-up: Phase 7 FastLED-examples warm-cache bench harness #218

@zackees

Description

@zackees

Context

crates/fbuild-library-select/benches/resolve_warm.rs (#216) measures the cache-hit path against the synthetic MiniFramework fixture (~3× speedup, ~1.7 ms warm). That's a per-crate sanity guard, not the AC#5 measurement: the issue's perf criterion calls for the warm path to add ≤ 50 ms over current fbuild on the FastLED examples matrix.

bench/fastled-examples/ was scaffolded as a placeholder in #211 with a README that says it waits on Phase 4 + a follow-up to land the harness. Phase 4 is in production now (#212 + #217), so this is unblocked.

Scope

  1. Add a Rust binary under bench/fastled-examples/ (or cleaner: a workspace-member crate fbuild-bench-fastled-examples) that:

    • Discovers FastLED examples from a configurable path (env FASTLED_DIR default ~/dev/fastled).
    • Picks a representative subset (e.g. Blink, Pacifica, Animartrix, Audio, BlurBenchmark, Chromancer — 5–10 sketches that span the simple/complex spectrum).
    • For each example, builds a synthetic project: the .ino as src/main.cpp, FastLED's own src/ as a project library, plus a Teensyduino-class framework library set (start with MiniFramework-extended; can grow to a real Teensyduino tree if needed).
    • Runs fbuild_library_select::cache::resolve_cached cold and warm against each (example, lib-set) pair.
    • Prints a table: example | cold (ms) | warm (ms) | hit-confirmed.
    • Asserts every warm call returns from_cache = true (catches silent re-misses).
    • Optional: emit report.json for future PR-comment diffing.
  2. CI wiring in .github/workflows/bench-205.yml:

    • Add a workflow_dispatch-only lane (no automatic PR trigger — needs ~/dev/fastled checkout step).
    • The lane checks out FastLED at a pinned ref, runs the harness, uploads the report as an artifact.
  3. README updates:

    • bench/fastled-examples/README.md — replace the "waits on Phase 4" framing with actual usage instructions.
    • bench/README.md — list the new harness.

Out of scope

  • Per-board ELF-size acceptance gates — those live in acceptance-205.yml and tests/teensylc_acceptance.rs etc.
  • Driving the full fbuild build pipeline — this bench measures only the resolver, not the compile/link chain.
  • Threshold-based CI gating — first capture a baseline, then a follow-up adds the ≤ +50 ms gate.

Acceptance

  • uv run soldr cargo run -p fbuild-bench-fastled-examples (or equivalent) runs to completion locally with FASTLED_DIR pointing at ~/dev/fastled.
  • Report shows distinct cold vs. warm timings per example.
  • All warm calls confirm from_cache = true.
  • clippy --workspace --all-targets -D warnings clean.

Refs: #205 Phase 7 (AC#5).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions