feat: ADR-W018 L2 — live add/edit of network sources (the DEV-stack spine)

[aperim-agent]

ADR-W018 Level 2 — live add/edit of network sources (the DEV-stack spine)

Runtime-add a network ingest source to a running multiview — spawn the ingest producer, normalize/jitter it, place it into a tile — without disturbing the output. Network/file kinds (rtsp/hls/ts/srt/rtmp/rist/file) now apply live on the running full-pipeline engine, through one uniform ingest path. This is the bottom of the DEV-stack and the API PR #179's deferred e2e chroma test needs.

What it lands

• Pipeline::live_ingest_spawner() + LiveSourceHub::start_with_ingest(...) — the production API: the hub gains an IngestSpawner seam; the full-pipeline run wires the pipeline's spawner, the software run wires None.
• LiveSourceCapability { synthetic, network } on the control AppState (+ with_live_sources, is_live), re-exported from multiview_control; the binary derives it from ingest.is_some() so the X-Multiview-Apply header never over-claims live.
• SourceKind::is_network_media() in the config schema — the level-2 classification point (disjoint from is_synthetic; ndi/youtube/aes67 in neither → restart). rist is included (it lowers to a rist:// AVIO URL and rides the identical ingest path).

Re-authored against current main (not cherry-picked)

The original PR #108 commits (23ca6955 RED, afe6ca68 impl) 9-way-conflict — pipeline.rs churned ~49× on main since. I read them as the intent, then re-implemented against current main's structures:

• Main had already absorbed much of the live-add foundation (the synthetic-only LiveSourceHub, command_drain_with_live_sources, shared_stores, the upsert_source drain arm that explicitly held the network path as "the next ADR-W018 slice"). This PR is that next slice.
• spawn_ingest_producer is extracted from IngestSupervisor::start so the startup supervisor and the live hub call the same function — a runtime add runs exactly the startup supervised ingest_loop (same reconnect bracket, jitter, PTS-normalize, rw-timeout), never a second-quality copy. Main's loop had also grown a #[cfg(youtube)] re-resolve sibling since feat(cli,control,web): network-source live add — runtime ingest spawn with placement consult (ADR-W018 level 2) #108 — preserved, re-ordered to spawn before the extracted producer.
• The original's standalone LiveSourceCapability coexists cleanly with main's later LiveApplyCaps (overlays, ADR-W022) — two parallel per-collection capability signals on AppState.

Invariant #1 (output-clock) — the live-added source is SAMPLED, never pacing

drive_streaming keeps emitting one frame/tick regardless of the new ingest. The drain only does cheap binding mutations at the frame boundary (store create/reuse + route key + config mirror); the heavy half (libav open + thread spawn + placement poll) is handed to the off-thread LiveSourceHub over a bounded channel. The hub worker never touches the output clock. Soak proof: live_added_file_source_goes_live_then_remove_slates_never_faltering (ffmpeg) drives add → LIVE → remove → NO_SIGNAL → rapid add/remove churn on a real pipeline and asserts !report.faltered.

Invariant #10 (isolation) — one actor, bounded drop-oldest, no back-pressure

The ingest is one hub-owned producer thread feeding its per-source lock-free TileStore (per-source isolation, no head-of-line blocking). The drain→hub channel is bounded try_send (full ⇒ shed + warn, tile rides the slate); the control/add path cannot back-pressure the engine. Admission goes through the cost model: select_live_decode_pick (ADR-W018 §7) re-polls NVML and consults the same select_device scorer with the candidate set restricted to the running island's device — a runtime add never fragments or migrates the GPU pipeline; an over-headroom/vanished island rejects to software decode, never overcommits. Chaos proof: the same churn test + the gpu placement unit tests (live_decode_pick_admits_onto_the_idle_island..., ..._rejects_an_over_headroom_island_to_software, ..._degrades_to_software_when_the_island_vanishes, live_spawner_consults_the_admission_path_on_every_decoded_spawn — exactly one fresh load poll per spawn).

Restores PR #179's deferred e2e test

crates/multiview-cli/tests/live_decode_chroma.rs (deferred from #179 because it referenced the not-yet-built live_ingest_spawner/start_with_ingest) now compiles + runs + passes on the live-add path: a live-added decode's U/V plane statistics match the startup decode of the same clip (the Defect-A U/V-inversion pin).

Gate evidence (lane-local target, env -u CARGO_TARGET_DIR)

• cargo fmt --all -- --check — clean
• cargo clippy --workspace --all-targets -- -D warnings — exit 0
• cargo clippy -p multiview-cli --features ffmpeg --all-targets -- -D warnings — exit 0 (the ffmpeg-gated leg that caught fix(ffmpeg): bypass libswscale for identity NV12 scale (hw Defect A) #179's CI failure)
• cargo clippy -p multiview-cli --features ffmpeg,overlay,gpu --all-targets -- -D warnings — exit 0 (compiles the gpu-gated placement code)
• cargo test --workspace — exit 0
• cargo test -p multiview-cli --features ffmpeg (e2e) — both realtime proofs pass; gpu placement unit tests 8/8; control typed_resources 23/23 (incl. 5 new capability-matrix tests)
• cargo deny — not required (no Cargo.toml/Cargo.lock changes)

HIGH-risk data-plane — not self-merging. Requesting the 3-lens cross-vendor panel (rule 21).

🤖 Generated with Claude Code

---

Pre-panel close-outs (orchestrator review items)

1. Inert cost-budget — verified + residual disclosed. select_live_decode_pick (and the pre-existing select_admission_pick) use CostBudget::new(100_000, 100_000, 100_000), an intentionally permissive per-engine Mpix/s budget. I verified the deferral claim: the budget being a near-inert gate with the VRAM-headroom hard gate + VRAM-dominant score as the real safety is documented in ADR-0018 ("hard gates are the real safety"; the cost model is "biased conservative"). NOTE: the pre-existing comment cited "ADR-0035 §5" for this — that citation is wrong (ADR-0035 has no §5 and does not scope a per-GPU perf-class budget table); commit 058b1fc corrects it to ADR-0018 and states the perf-class table is genuine future work (not yet built). Disclosed residual: a VRAM-roomy but decode-engine-saturated GPU passes both the permissive budget and the 0.85 headroom gate, so a live add CAN be admitted onto a GPU that cannot sustain another decode. Why acceptable for this ship: the never-off-air contract holds — an over-subscribed decode degrades the new tile only (the output clock samples last-good; the program is never touched — inv #1/#2), and invariant #9's closed-loop degradation sheds the cheapest-impact tile if the saturation bites. The reject→software-decode ladder (select_live_decode_pick returns None ⇒ software open) already covers VRAM/session/headroom exhaustion; only the decode-Mpix dimension is un-budgeted until the perf-class table lands.

2. Management-completeness (§H) — confirmed, no new operator control needed (one copy fix). W018-L2 (live add/edit of a network source) is fully reachable through the existing source-CRUD UI: web/src/pages/SourcesPage.tsx already offers every network/file kind (rtsp/hls/ts/srt/rtmp/file/youtube) with typed locator + RTSP-transport fields, and surfaces X-Multiview-Apply generically — so the server-side header flip needs zero new web wiring. The only web change required was honesty copy (rule 27): the apply-semantics callout + saved-description previously said network/file sources "go live via config export + restart", which my server change makes false. Commit 058b1fc refreshes that copy (synthetic + network/file apply live on a full-engine run; NDI/YouTube/AES67 and the software engine restart). No new API resource or UI control is missing.

3. Full fresh gate re-run (lane-local target, env -u CARGO_TARGET_DIR, -j1 for Rust):

• cargo fmt --all -- --check → exit 0
• cargo clippy --workspace --all-targets -- -D warnings → exit 0
• cargo clippy -p multiview-cli --features ffmpeg --all-targets -- -D warnings → exit 0
• cargo clippy -p multiview-cli --features ffmpeg,overlay,gpu --all-targets -- -D warnings → exit 0
• cargo test --workspace → exit 0 (a cold parallel build first hit the known spurious E0463 can't find crate multiview_input race in an untouched crate; a warm-cache retry passed clean)
• cargo test -p multiview-cli --features ffmpeg → the two e2e realtime proofs pass; gpu placement 8/8; control typed_resources 23/23
• web (since web/ changed): npm ci; eslint --max-warnings=0 exit 0; tsc --noEmit exit 0; lingui compile exit 0; vitest run 632 passed / 63 files exit 0
• cargo deny — not required (no Cargo.toml/Cargo.lock changes)

CI attribution: on the prior CI run both check + test jobs (macos + ubuntu) passed. The two red jobs were NOT attributable to this PR: web (eslint+tsc+vitest+build) fails on main at the same base SHA (pre-existing — locally the full web suite passes 632/632 here, so it is a CI-env flake in the WhepPlayer media tests), and feature-gated clippy (multiview-webrtc, native) is a flaky timing test (servicing_keeps_pace_with_the_receive_budget_under_flood) in an untouched crate that passed on re-run.

---

Correctness fix (commit 943706a) — DecodePlacement tri-state

Orchestrator review caught that my initial re-author flattened the 2a67e4e4 DecodePlacement amendment into Option<String> — a real overcommit/fragmentation bug, now fixed.

The bug: select_live_decode_pick returned Option<String>, so a placement reject returned None, leaving IngestPlan.cuda_ordinal = None. In open_and_stream that becomes new_preferring_hw(.., want_hw=true, None) → NVDEC opens on libav's default CUDA device. On a single-GPU over-headroom island the default device IS that island (overcommit); on a multi-GPU host it may be a different GPU (silent island fragmentation, forbidden by ADR-0018). "No ordinal" cannot distinguish no placement decision (default device fine) from placement rejected (hardware must NOT open).

The fix: an explicit, closed tri-state DecodePlacement::{Default, Pinned(ordinal), SoftwareOnly} on IngestPlan + a single decoder_open_args gate: SoftwareOnly → (want_hw=false, None) forces software even when NVDEC is compiled/present/not env-disabled; Pinned/Default keep the canonical want_hw_decode env reading (operator MULTIVIEW_DISABLE_NVDEC still wins over a pin). select_live_decode_pick → select_live_decode_placement: admit → Pinned (or Default when the island resolved no ordinal); reject AND island-vanished → SoftwareOnly. Restored tests/decode_placement.rs (the f0a6b7f/2a67e4e4 RED pin), which opens the real decoder via the gate's exact args and asserts a SoftwareOnly source comes back software (used_hw=false, no cuvid named); the gpu-lane unit tests now assert the placement value AND the decode-open behaviour per placement, not just the picked ordinal. ADR-W018 §7 amended for the as-built tri-state.

Re-gated after the fix (all green, lane-local target, env -u CARGO_TARGET_DIR, -j1 for Rust): cargo fmt --all -- --check exit 0; cargo clippy --workspace --all-targets -- -D warnings exit 0; cargo clippy -p multiview-cli --features ffmpeg,overlay,gpu --all-targets -- -D warnings exit 0; cargo test --workspace exit 0; cargo test -p multiview-cli --features ffmpeg,overlay,gpu: decode_placement 2/2, gpu placement units 8/8, live_network_apply + live_decode_chroma 1/1 each.

---

Panel-blocker resolutions (head 1abed23)

The Codex 3-lens panel reviewed head 058b1fcb (before the tri-state fix) and returned two findings; both are now resolved.

Blocker #1 (tri-state to the open path) — fixed in 943706a (the "DecodePlacement tri-state" section above). The panel saw the old Option<String> code where a reject's None opened NVDEC on the default device. Now DecodePlacement::{Default,Pinned,SoftwareOnly} threads through IngestPlan to the open path, SoftwareOnly → decoder_open_args → (want_hw=false, None), and tests/decode_placement.rs asserts SOFTWARE behaviour at the open path (the real new_preferring_hw call → used_hw=false, no cuvid named), not just that the pick returns None.

Major #2 (RIST capability gating) — fixed in 1abed23. RIST was advertised live-appliable unconditionally, so an ffmpeg-without-rist build over-claimed X-Multiview-Apply: live for a RIST source whose spawn would refuse (tile rides slate). Fixed: LiveSourceCapability gains a rist: bool the binary sets from cfg!(feature = "rist") (via with_rist); is_live returns self.rist for SourceKind::Rist, self.network for the other network kinds. is_network_media classification is unchanged (RIST is network media); only its live-apply capability is feature-gated — the honest level. Two new typed_resources tests pin both directions (with_rist(false) ⇒ RIST restart + no UpsertSource while a non-RIST network kind still flips live; with_rist(true) ⇒ RIST live + UpsertSource).

Re-gated after both fixes (all green, lane-local target, env -u CARGO_TARGET_DIR, -j1 for Rust): cargo fmt --all -- --check exit 0; cargo clippy --workspace --all-targets -- -D warnings exit 0; cargo clippy -p multiview-cli --features ffmpeg exit 0 AND --features ffmpeg,rist exit 0 AND --features ffmpeg,overlay,gpu exit 0 (all cfg branches compile); cargo test --workspace exit 0; control typed_resources 25/25; decode_placement 2/2; gpu placement units 8/8; live_network_apply + live_decode_chroma 1/1 each.