[#109] STT: per-model confidence-floor table + env override#120
Conversation
Replace the two hardcoded confidence-flow consts with a per-model-family floor table (tiny/base/small/medium/large-v3; quantized variants and the large-v3-turbo distillation map to their family). All families are seeded with the historical (0.5/0.6) values, so behavior is unchanged with no env set on any model. Add recompile-free live-tuning knobs LIVECAP_CONFIDENCE_FLOOR and LIVECAP_AUTO_DETECT_CONFIDENCE_FLOOR: parsed as f32, clamped to 0.0..=1.0, invalid/unset falls back to the table value. Tests: family mapping incl. quantized names, env override + clamp + invalid, auto >= forced invariant, and the existing five floor tests (updated to the model-aware signatures). No new runtime deps; no caption content logged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
project7-interns
left a comment
There was a problem hiding this comment.
Verdict: REQUEST CHANGES
Epic Alignment: NOT REVIEWED
Structural gate failed before code review: the live PR body is missing the required ## EPIC Alignment section.
Checked (evidence)
- Structural gate:
gh pr view 120 --json body,title,headRefOid,mergeStateStatusat headd9cdd0192fa3c55d08199b30d8d9f4c4bd2c84ceshowed## What,## Tests,## Verification, and## Safety / boundaries, but no required## EPIC Alignmentsection and no required## Self-Verificationsection. - Riskiest part of this diff: not assessed; the review procedure requires stopping before code review when the PR template gate fails.
- Kill-list: not scanned because the structural gate failed.
- CI: not used for verdict; structural gate fails independently of CI.
Findings
- [blocking] Required PR template sections are missing.
- File: PR #120 body
- Why it fails: Dev PRs must contain filled
## EPIC Alignmentand## Self-Verificationsections before review. This PR body does not include either section, so reviewers cannot verify the claimed epic alignment or self-check protocol. - Do instead: Update the PR body with filled
## EPIC Alignmentand## Self-Verificationsections, then re-request review from @re1 and @re2.
Decision
REQUEST CHANGES. I did not review the diff; this is the mandatory structural gate result.
|
RE2 review — PR #120 (Closes #109) @ d9cdd01 — APPROVE Reviewed against the actual
Non-blocking note (for #111, not gating): the two env knobs are independent, so an operator setting only No findings. APPROVE. |
project7-interns
left a comment
There was a problem hiding this comment.
Verdict: REQUEST CHANGES
Epic Alignment: FAIL
#109 / #107 require the auto-detect confidence floor invariant (auto >= forced) while adding live env overrides; the diff only preserves that invariant for the seeded table, not for resolved env values.
Checked (evidence)
- Structural gate:
gh pr view 120 --json body,headRefOid,mergeStateStatusatd9cdd0192fa3c55d08199b30d8d9f4c4bd2c84cenow includes filled## EPIC Alignmentand## Self-Verification. - Issue/epic context:
gh issue view 109requires env overrides plusauto floor >= forced floor invariant;gh api repos/realproject7/livecap/issues/107problem #1 names the hardcoded tiny-seeded floors as the epic gap. - Live diff:
gh pr diff 120 --patchchanges onlycrates/livecap-core/src/whisper/engine.rs. - Riskiest part of this diff: env override resolution, because it changes runtime gating without recompilation; it currently violates the auto-vs-forced strictness invariant.
- Kill-list: scanned changed runtime/test code; finding below.
- CI:
gh pr checks 120→ app-macos pass, packages-linux pass, no-stub-gate pass.
Findings
- [blocking] Env overrides can make auto-detect less strict than forced mode.
- File:
crates/livecap-core/src/whisper/engine.rs:342 - Why it fails:
confidence_floor()resolves the auto and forced env vars independently. IfLIVECAP_CONFIDENCE_FLOOR=0.9andLIVECAP_AUTO_DETECT_CONFIDENCE_FLOORis unset or lower, line 342 returns0.6for auto while line 344 returns0.9for forced. That violates #109'sauto floor >= forced floorinvariant and the existing #93 design where auto-detected language must require a higher margin than forced language. - Do instead: resolve both floors together, apply env overrides, then enforce
auto_detect >= forced(for example clamp/raise the resolved auto floor to the resolved forced floor), and add an env-combination regression test for forced override above auto/table.
- File:
Decision
REQUEST CHANGES. The mechanism is scoped correctly, but the live env knobs must preserve the same auto-detect strictness contract as the seeded table before approval.
RE1 review: the two env knobs (LIVECAP_CONFIDENCE_FLOOR / LIVECAP_AUTO_DETECT_CONFIDENCE_FLOOR) resolved independently, so a forced override above the auto floor could make auto-detect LESS strict than forced — inverting the #93 contract at runtime. Resolve both floors together in resolved_floors(), apply env overrides, then lift the resolved auto floor to at least the resolved forced floor. Add a regression test for forced-override-above-auto (table and explicit lower auto). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
project7-interns
left a comment
There was a problem hiding this comment.
Verdict: APPROVE
Epic Alignment: PASS
PR #120 now satisfies #109 under parent #107: it replaces tiny-seeded hardcoded STT floors with the per-model-family mechanism and env tuning knobs while preserving the #93 auto >= forced contract.
Checked (evidence)
- Re-review delta: reviewed fix commit
c9d94a2after prior RE1 finding atd9cdd01; PR head isc9d94a2dd382e7285b3a804cb1ea3569783ef389. - Prior finding resolved:
crates/livecap-core/src/whisper/engine.rs:343now resolves both floors together, andengine.rs:349lifts auto toauto_detect.max(forced). - Regression coverage:
crates/livecap-core/src/whisper/engine.rs:762addsenv_override_preserves_auto_ge_forced_invariant, covering forced override above table auto, forced above explicit lower auto, and higher auto preserved. - Riskiest part of this diff: runtime env override resolution; acceptable now because resolved floors enforce
auto_detect >= forcedafter all overrides. - Kill-list: scanned changed delta; clean.
- CI:
gh pr checks 120→ app-macos pass, packages-linux pass, no-stub-gate pass.
Findings
- None.
Decision
APPROVE. The blocking invariant issue is fixed, and the PR remains confined to crates/livecap-core/src/whisper/engine.rs with no new dependencies or caption-content logging.
|
RE2 re-review — PR #120 (Closes #109) @ c9d94a2 — APPROVE @re1's blocking finding is fully resolved. Reviewed the delta d9cdd01..c9d94a2.
No findings. APPROVE. |
|
RE2 verdict (current commit) — PR #120 (Closes #109) @ c9d94a2 — APPROVE Checked (evidence)
Findings
DecisionAPPROVE at c9d94a2. Mechanism-only change, no-env behavior byte-identical on every model, env knobs clamped with the |
Closes #109
EPIC Alignment
tinywhile production defaults tosmall, with no per-model table and no runtime override. This PR supplies the mechanism: a per-model-family floor table + recompile-free env knobs, so the calibration ticket (STT live verification & per-model floor calibration on real audio (closes the #92/#93/#94 verification gap) #111) can set real per-family values and tune live without recompiles.crate::model::MODEL_NAMESfor family mapping (model_family(),engine.rs). Exposes the internal floor resolverconfidence_floor(model_name, auto_detect)and two env knobsLIVECAP_CONFIDENCE_FLOOR/LIVECAP_AUTO_DETECT_CONFIDENCE_FLOOR(engine.rs,CONFIDENCE_FLOOR_ENV/AUTO_DETECT_CONFIDENCE_FLOOR_ENV). No public API change — all symbols are crate-private; the gate application (is_low_confidence_drop) keeps its behavior.Depends on:. Extends the existing STT: whisper emits phantom captions on silence/ambient noise #92/STT: wrong source-language detection (English transcribed as Chinese) #93 confidence-floor gate in place (no duplicate module/helper). Deliberately leaves the real floor VALUES to STT live verification & per-model floor calibration on real audio (closes the #92/#93/#94 verification gap) #111 (epic contract: calibration owns values with real-audio evidence) and the user-facing model choice to Settings: whisper model selection (small / medium / large-v3-turbo) with download-on-switch #110 — this table is exactly what Settings: whisper model selection (small / medium / large-v3-turbo) with download-on-switch #110/STT live verification & per-model floor calibration on real audio (closes the #92/#93/#94 verification gap) #111 build on, so neither has to rip it out.Self-Verification
cargo test -p livecap-core(compiles the crate + whisper.cpp) → built clean.cargo test -p livecap-core→ 52 passed; 0 failed (15 inwhisper::engine, incl. the preserved 5 floor tests updated to the model-aware signatures + new family/env tests + theauto >= forcedenv-invariant regression test).cargo clippy -p livecap-core --all-targets -- -D warnings→ clean (caught + fixed aclippy::assertions_on_constantson the invariant assert). Full-workspacecargo clippy --workspace --all-targets -- -D warnings+cargo test --workspace→ green in theapp-macosCI job (thelivecap-appcrate iscfg(target_os = "macos")-gated and does not build on Linux).scripts/no-stub-gate.sh→ passed.0.0..=1.0/ invalid→table ✓; (c) tests for family mapping incl. quantized, env override+clamp+invalid, auto≥forced invariant, existing 5 floor tests green ✓; (d) no behavior change with no env set on any model (all families seeded 0.5/0.6) ✓.env_override_replaces_table_value_and_is_clamped(env→table resolution end-to-end) andmodel_family_maps_names_and_quantized_variants(everyMODEL_NAMESentry resolves).What
Replaces the two hardcoded floor consts (
CONFIDENCE_FLOOR=0.5,AUTO_DETECT_CONFIDENCE_FLOOR=0.6) incrates/livecap-core/src/whisper/engine.rswith:ModelFamily{tiny/base/small/medium/large-v3};model_family()strips the quantization suffix sosmall-q5_1→small, and foldslarge-v3-turbo(-q5_0)into large-v3; unknown/future names fall back tosmall(production default).family_floors()seeds all families withSEED_FLOORS= (forced 0.5, auto-detect 0.6) viaconfidence_floor(model_name, auto_detect).LIVECAP_CONFIDENCE_FLOOR/LIVECAP_AUTO_DETECT_CONFIDENCE_FLOOR— parsed as f32, clamped to0.0..=1.0; unset / empty / unparseable / non-finite (inf/nan) → table value. Resolution order: env override → per-family table.resolved_floors()resolves both, then lifts the resolved auto floor to at least the resolved forced floor — a forced override above the auto floor can never make auto-detect less strict than forced (STT: wrong source-language detection (English transcribed as Chinese) #93 contract).Deviations
large-v3-turbo→large-v3family. The issue lists exactly five families andlarge-v3-turboisn't one, but it (and its quantlarge-v3-turbo-q5_0) exists inMODEL_NAMES. I fold the turbo distillation into thelarge-v3family and documented it inmodel_family. Behavior-equivalent today (all families seeded identically); flagged for STT live verification & per-model floor calibration on real audio (closes the #92/#93/#94 verification gap) #111 in case calibration wants turbo as its own family.crates/livecap-core.