Skip to content

[audit][infra] CI & gates hardening: release-mode job, SHA-pin actions, release.yml quality gates, no-stub-gate + color-guard blind spots, VAD test-note contradiction #176

Description

@realproject7

Found by the full-codebase audit of 2026-07-14 (PO manual §8.1: 15 scan agents over 100% of production files, adversarially verified, then PO-verified against source). Part of the v1.3 audit follow-up set under EPIC #134.

Goal

Five verified gaps in the automation that guards our invariants. One workflow-surface fix-unit.

Findings (verified on main f8eb9ea)

.github/workflows/ci.yml:59 (medium/certain)

  • Issue: CI never compiles or tests release mode, so the release-build privacy invariant (debug-only env escapes stripped) has zero automated coverage. cargo test and clippy both run the debug profile; every #[cfg(not(debug_assertions))] branch (dev_flags.rs release stubs, pipeline.rs without the ChannelDump) first compiles at tag time in release.yml, and no test anywhere asserts that a release binary ignores LIVECAP_BLEED_DUMP_DIR / LIVECAP_CAPTURE_VISIBLE / LIVECAP_AUTOSTART.
  • Impact: Exact recurrence of ticket [security] Gate LIVECAP_BLEED_DUMP_DIR behind cfg(debug_assertions) — release binary can covertly dump raw audio #161's HIGH privacy bug slips through green CI: a PR that drops the #[cfg(debug_assertions)] from the dump block in crates/livecap-core/src/pipeline.rs:345 (e.g. a bad merge-conflict resolution) changes nothing observable in debug builds, so cargo test, clippy, and no-stub-gate all pass — and the shipped release binary regains a covert raw mic+system audio WAV dump. Secondary: a compile error in release-only code is first discovered when the release tag build fails.
  • Fix direction: Add a cheap release-profile job to ci.yml (cargo check --workspace --release, ideally cargo test --workspace --release on the macos job), plus a dedicated gate: grep that every std::env::var("LIVECAP_...") read outside an explicit allowlist (suppression/floor tuning, LIVECAP_NODE, test dirs) is inside a #[cfg(debug_assertions)] item, failing CI otherwise.

.github/workflows/release.yml:42 (medium/certain)

  • Issue: Supply-chain hygiene: every third-party action in the release workflow is referenced by a mutable tag or branch (actions/checkout@v4, pnpm/action-setup@v4, actions/setup-node@v4, dtolnay/rust-toolchain@stable, Swatinem/rust-cache@v2) — none is pinned to a commit SHA — in the same job that materializes APPLE_CERTIFICATE, APPLE_CERTIFICATE_PASSWORD, APPLE_ID and APPLE_PASSWORD into env and produces the user-facing signed DMG.
  • Impact: If any of those upstream tags/branches is compromised (tag re-point or malicious release — the 2025 tj-actions/changed-files incident pattern), the attacker's code runs in the release job with contents:write and can exfiltrate the Apple signing identity or tamper with the DMG before checksumming, i.e. a signed, trojaned build of a privacy-first product. dtolnay/rust-toolchain@stable is a moving branch, the weakest of the five.
  • Fix direction: Pin all uses: references in release.yml (and ci.yml) to full commit SHAs with a version comment (e.g. uses: actions/checkout@08c6903 # v4), and add dependabot/renovate for github-actions to keep them fresh.

.github/workflows/release.yml:11 (low/certain)

  • Issue: The release workflow runs zero quality gates: no no-stub-gate, no color-guard, no clippy, no cargo test, no vitest — only the version-match check and pnpm build. Tag pushes are not covered by branch protection, so nothing enforces that the tagged commit ever passed CI.
  • Impact: An operator (or a compromised token with contents:write) tags v0.2.0 on a local branch commit that never went through a PR — e.g. one containing a failing privacy test or a stub — and the workflow happily builds, signs, and uploads the DMG to a draft release. The draft step is the only remaining human gate, and it displays no CI status for the tagged commit.
  • Fix direction: Add the cheap gates to the release job before building (./scripts/no-stub-gate.sh, ./scripts/color-guard.sh, cargo test --workspace, pnpm test), or gate the job on the tagged commit having a successful CI run (gh api commits/$GITHUB_SHA/check-runs) — codifying that a release is always cut from a CI-green commit.

scripts/no-stub-gate.sh:5 (low/certain)

  • Issue: The no-stub gate does not match the word 'stub' — the very marker it is named for — nor 'dummy', 'unimplemented', 'not implemented', or 'XXX'. The project's no-stub/no-mock policy (CLAUDE.md, EPIC [EPIC] LiveCap MVP — local-first live captions & translation overlay #1) is only enforced for 5 of the common deferred-functionality markers.
  • Impact: A developer ships function stubTranslateBatch() or a // stub implementation until #123 comment in src/host/session.ts; CI's no-stub-gate job passes green and the stub reaches a release build, directly violating the 'no stub/mock/placeholder code in production' invariant the gate exists to enforce. (Verified: no such markers exist in app code today, so extending the pattern is currently a no-op on the tree.)
  • Fix direction: Extend the pattern: PATTERN='TODO|FIXME|HACK|placeholder|\bmock|\bstub|\bdummy|unimplemented|not implemented|\bXXX\b' — and add a self-test line in a fixtures/ dir to prove the gate still fires.

scripts/color-guard.sh:22 (low/certain)

  • Issue: The color-token gate only scans src/**/*.css and only matches hex and rgb()/rgba() literals. hsl()/hsla()/hwb()/oklch()/color-mix() and CSS named colors pass, and colors assigned from TypeScript (el.style.background = ..., setProperty, inline style strings in template HTML) are entirely out of scope — a surface the codebase actively uses (src/dashboard.ts:203-215 sets style.border/background/color, currently with var() tokens).
  • Impact: The Design: tokenize sheet/on-accent/success colors + surface-opacity scale, add CI raw-color guard #116 token debt regrows through the unguarded channels: a new UI surface writes el.style.background = "#1e1e1e" in a .ts file, or a stylesheet uses color: hsl(0 0% 10%), and CI's color-guard job stays green — precisely the raw-literal drift the gate exists to block. Verified no such literals exist today, so tightening is a no-op on the current tree.
  • Fix direction: Extend COLOR_RE with hsla?(|hwb(|oklch(|color-mix( and add a second pass over src/**/.ts flagging hex/rgb()/hsl() string literals assigned to style properties (same value-side heuristic, same / color-guard-allow */ escape hatch).

crates/livecap-core/src/vad.rs:411 (low/certain)

  • Issue: vad_state_is_maintained_across_chunks asserts the synthetic harmonic generator IS detected as speech (and passes today — verified by running it), directly contradicting the NOTE at lines 467-472 in the same file claiming 'this Silero build never classifies the generator's output as speech'. longer_redemption_does_not_fragment_more is vacuously true (0 <= 0) whenever detection yields nothing. Tests relying on synthetic audio being classified as speech are exactly the vacuous-test class [bug] VAD 30s force-cut duplicates captions + grows silero buffer unbounded on long monologues (M3) #138/[bug] #138 force-cut fix causes a channel-killing panic (regression) — hotfix in progress #162 already burned on.
  • Impact: A Silero rev bump that stops (or starts differently) classifying the synthetic signal makes the first test fail spuriously or the second pass vacuously, and the contradictory in-file documentation misleads the next maintainer about which regime the tests actually exercise.
  • Fix direction: Port these two unit tests to the real-speech WAV fixtures used by tests/force_cut_wav.rs (or downgrade their assertions to the API-contract-only form used by take_current_speech_force_cuts_utterance), and correct the NOTE so it does not claim synthetic audio is never detected while a passing test proves otherwise.

Scope

  1. Add a release-invariants CI job (macos): cargo test --release -p livecap-app --no-run + release build of livecap-app + strings assert absence of LIVECAP_BLEED_DUMP_DIR and the dev-flag env names (turns the [security] Gate LIVECAP_BLEED_DUMP_DIR behind cfg(debug_assertions) — release binary can covertly dump raw audio #161-class invariant into a permanent gate).
  2. Pin all third-party actions in ci.yml + release.yml to commit SHAs.
  3. Make release.yml run the same gates as ci.yml before building a tag (or require the tagged commit to have a green CI run).
  4. no-stub-gate.sh: add stub/dummy/unimplemented/not implemented/XXX to the pattern set (case-insensitive where sane).
  5. color-guard.sh: extend to hsl()/hwb()/oklch()/color-mix()/named colors, and scan inline styles set from TS (best-effort grep; document what remains uncovered).
  6. Reconcile the vad.rs NOTE vs vad_state_is_maintained_across_chunks (the synthetic generator IS classified as speech today) — fix the note or the fixture so the test's premise is honest.

AC

  • All new/changed gates green on main; a seeded violation of each gate fails CI (prove one per gate in the PR description).

Security invariants

Never log or persist caption/audio content; no new dependencies; simplest fix that fully solves the ticket — no drive-by refactors.

Ticket-review amendments (Batch 36, 2026-07-14 — evidence in QW chat #873-#880)

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/quadworkDispatched to QuadWork batch (headless-testable)enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions