fix(audio): stop near-silent renders becoming blank noise + guard archetype renders - #204
Conversation
…rchetypes Root cause of the blank/hiss voices: normalize_audio peak-normalized to -2 dBFS whenever max(|audio|) > 0. When the model emits a near-silent clip (peak at the noise floor, e.g. 1e-4), that applies thousands of × of gain and lifts the noise floor to full scale — silence turned into loud hiss. This affected every generation path (clone/dub/design/archetypes), which is why "some voices" came out as blank noise. - services/audio_dsp.py: normalize_audio gains a -50 dBFS silence floor. At or below it the audio is left untouched (stays inaudible) instead of being amplified. Real speech — even a whisper — peaks well above the floor, so normal output is unchanged. - api/routers/archetypes.py: after rendering, _is_blank_audio() detects a dead clip (empty / non-finite / peak < 0.02 — a real normalized clip peaks ~0.79). The render retries once with a different seed, then fails loudly (503 via the existing handlers) so a blank preview or voice profile is never cached/saved. Also extracts the script with a non-empty fallback. - core/archetypes.py: _build never falls back to an empty script (empty text synthesizes to silence). Tests (tests/, runs in CI): normalize_audio doesn't amplify silence but still normalizes real audio to target; _is_blank_audio flags dead renders and passes real audio; every archetype carries a non-empty sample script. Verified: full tests/ suite 601 passed incl. 8 new (the 2 test_supertonic3 failures are pre-existing on main — local .venv engine/license state, green in CI — and unrelated to this diff). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThis PR hardens archetype audio rendering against blank or silent outputs. It ensures scripts are non-empty at the source, adds a silence-floor safeguard to audio normalization, introduces blank-audio detection with retry logic in the renderer, and validates both safeguards with tests. ChangesBlank Audio Guard and Silence Protection
Sequence DiagramsequenceDiagram
participant API_Router as API Router (_render_archetype_wav)
participant TTS_Infer as TTS inference (_infer)
participant Blank_Check as _is_blank_audio
participant Cache as Cache/Save
API_Router->>TTS_Infer: call _infer(text, seed=seed0)
TTS_Infer->>Blank_Check: produced tensor
Blank_Check-->>API_Router: is_blank = True/False
alt is_blank == True
API_Router->>TTS_Infer: call _infer(text, seed=seed1)
TTS_Infer->>Blank_Check: produced tensor (retry)
Blank_Check-->>API_Router: is_blank = True/False
alt still blank
API_Router->>API_Router: raise RuntimeError (do not cache)
else not blank
API_Router->>Cache: save rendered wav
end
else not blank
API_Router->>Cache: save rendered wav
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/api/routers/archetypes.py`:
- Around line 54-57: Compute a single effective sample script from
(a["sample_script"], a["language"]) and reuse it for both rendering and DB
writes instead of using raw a["sample_script"] or the hardcoded _FALLBACK_SCRIPT
in multiple places: add a small helper (e.g.,
get_effective_sample_script(sample_script, language)) that returns sample_script
when present or a language-aware fallback otherwise (derive fallback from
_FALLBACK_SCRIPT or localize it based on language), then use that returned value
wherever you currently render audio (the render path that selects fallback) and
wherever you set ref_text / persist the sample in the DB (replace uses of
a["sample_script"] at the save path) so the transcript and saved audio always
match.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 3c47a07e-0f85-4eb6-a636-15af14eeb8b4
📒 Files selected for processing (5)
backend/api/routers/archetypes.pybackend/core/archetypes.pybackend/services/audio_dsp.pytests/test_archetype_blank_guard.pytests/test_normalize_audio_silence.py
| # A non-empty script is always required — synthesizing empty text yields | ||
| # silence. Every archetype carries a use-case script, but guard the render path | ||
| # too so a malformed archetype can never drive a blank render. | ||
| _FALLBACK_SCRIPT = "Here's a quick sample of this voice so you can hear how it sounds." |
There was a problem hiding this comment.
Centralize the effective sample script.
Line 102 can render fallback text, but Lines 246-247 still persist the raw a["sample_script"]. If this guard path ever fires, the saved profile will contain audio for one transcript and ref_text for another. The same duplication also hardcodes an English fallback for malformed Chinese archetypes. Please derive the final script once from (sample_script, language) and reuse it for both rendering and DB writes.
Suggested shape
+def _effective_sample_script(a: dict) -> str:
+ text = (a.get("sample_script") or "").strip()
+ if text:
+ return text
+ if a.get("language") == "Chinese":
+ return "大家好，欢迎来到这个声音示范，希望你会喜欢这一段简单的朗读。"
+ return "Here's a quick sample of this voice so you can hear how it sounds."
+
async def _render_archetype_wav(a: dict, out_path: Path) -> None:
@@
- text = (a.get("sample_script") or "").strip() or _FALLBACK_SCRIPT
+ text = _effective_sample_script(a)
@@
- profile_id, profile_name, audio_filename, a["sample_script"],
+ profile_id, profile_name, audio_filename, _effective_sample_script(a),Also applies to: 102-102, 246-247
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/api/routers/archetypes.py` around lines 54 - 57, Compute a single
effective sample script from (a["sample_script"], a["language"]) and reuse it
for both rendering and DB writes instead of using raw a["sample_script"] or the
hardcoded _FALLBACK_SCRIPT in multiple places: add a small helper (e.g.,
get_effective_sample_script(sample_script, language)) that returns sample_script
when present or a language-aware fallback otherwise (derive fallback from
_FALLBACK_SCRIPT or localize it based on language), then use that returned value
wherever you currently render audio (the render path that selects fallback) and
wherever you set ref_text / persist the sample in the DB (replace uses of
a["sample_script"] at the save path) so the transcript and saved audio always
match.
|
| Filename | Overview |
|---|---|
| backend/api/routers/archetypes.py | Adds _is_blank_audio guard and retry logic for archetype rendering; seed used for the retry render is not propagated back to the use_archetype DB insert, leaving the stored seed stale when fallback fires. |
| backend/services/audio_dsp.py | Adds -50 dBFS silence floor to normalize_audio; near-silent signals are no longer amplified to full-scale hiss. Clean fix with correct math and well-scoped impact. |
| backend/core/archetypes.py | Replaces empty-string fallback in _build with _SCRIPTS["narration"] so no archetype ever carries an empty sample_script; straightforward one-liner fix. |
| tests/test_archetype_blank_guard.py | New test suite covering _is_blank_audio for zeros/noise-floor/empty/NaN and verifying every archetype carries a non-empty sample_script. |
| tests/test_normalize_audio_silence.py | New regression tests for normalize_audio: near-silent input stays inaudible, all-zeros preserved, real audio hits target dBFS, just-above-floor still normalizes, empty passthrough. |
Comments Outside Diff (1)
-
backend/api/routers/archetypes.py, line 245-248 (link)When the blank-audio retry fires in
use_archetype, the actual audio file is produced with seed_PREVIEW_SEED + 1(43) but the DB row always records_PREVIEW_SEED(42). Any downstream code that reads the stored seed to re-derive the voice — or that logs it for reproducibility — will see the wrong value. The seed column should reflect the seed that actually produced the saved WAV.
Reviews (2): Last reviewed commit: "fix(gallery): static log message in blan..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| try: | ||
| import torch | ||
|
|
||
| t = audio_tensor if isinstance(audio_tensor, torch.Tensor) else torch.as_tensor(audio_tensor) | ||
| if t.numel() == 0: | ||
| return True | ||
| t = t.detach().to("cpu", dtype=torch.float32) | ||
| if not torch.isfinite(t).all(): | ||
| return True | ||
| return t.abs().max().item() < 0.02 | ||
| except Exception: # never let the checker itself block a render | ||
| return False |
There was a problem hiding this comment.
The bare except Exception: return False means any unexpected failure inside the checker — including an ImportError if torch somehow isn't importable at that point, a CUDA OOM during .to("cpu"), or a future tensor-dtype mismatch — causes the blank guard to pass silently, letting a potentially bad render proceed to be cached and served. Since the checker only runs in a post-render context where torch is already loaded, the comment's "never let the checker itself block a render" goal could be achieved more safely by narrowing the catch to except (RuntimeError, ValueError) and letting truly unexpected errors propagate.
2 participants
Fix "blank noise" voices + guard archetype renders
Root cause (
services/audio_dsp.py):normalize_audiopeak-normalized to −2 dBFS whenevermax(|audio|) > 0. When the model emits a near-silent clip (peak at the noise floor, e.g.1e-4), that applies ~8000× gain and lifts the noise floor to full scale — silence becomes loud hiss. It had no silence floor, so it hit every generation path (clone / dub / design / archetypes). That's why some voices came out as blank noise.Two-layer fix
normalize_audiosilence floor (−50 dBFS). At/below it the audio is left untouched (stays inaudible) instead of being amplified. Real speech — even a whisper — peaks well above the floor, so normal output is unchanged. Benefits the whole app, not just archetypes.api/routers/archetypes.py). After rendering,_is_blank_audio()flags a dead clip (empty / non-finite / peak < 0.02 — a real normalized clip peaks ~0.79). The render retries once with a different seed, then fails loudly (503 via the existing handlers) so a blank preview or voice profile is never cached or saved. Script is extracted with a non-empty fallback.core/archetypes.py:_buildnever falls back to an empty script (empty text → silence).Tests (
tests/, run in CI)normalize_audiodoes not amplify silence (1e-4 stays < 0.01) but does normalize real audio to target (~0.79); all-zeros stays zero; just-above-floor still normalizes; empty passthrough._is_blank_audioflags zeros / noise-floor / empty / NaN; passes real audio.sample_script.Verified locally: full
tests/suite 601 passed incl. 8 new. The 2test_supertonic3failures are pre-existing onmain(local.venvengine/license state; green in CI) and unrelated to this diff.🤖 Generated with Claude Code
Summary by CodeRabbit
Bug Fixes
Tests