Skip to content

audio: fix chunk splices, output delay and level policy in the enhancers - #360

Closed
CryptVenture wants to merge 1 commit into
0xShug0:mainfrom
CryptVenture:pr/enhancement-alignment
Closed

audio: fix chunk splices, output delay and level policy in the enhancers#360
CryptVenture wants to merge 1 commit into
0xShug0:mainfrom
CryptVenture:pr/enhancement-alignment

Conversation

@CryptVenture

Copy link
Copy Markdown
Contributor

What this changes

Three independent defects in the arithmetic around the enhancement models, not in the models themselves. Each is reachable in its previous form through a new options struct, so existing call sites are unchanged unless they ask otherwise.

ZipEnhancer: hard splices and a 250 ms tail hole

It spliced its segmented output with std::copy every 1.5 s, and left up to 250 ms of digital silence on the tail for roughly one input length in six — the lengths where (n - 32000) % 24000 == 0.

Replaced with the accumulate-and-normalise overlap-add that deepfilternet2.cpp already uses. Accumulated weight is now exactly 1.0 on every returned sample and the output length matches the input.

Measured on Metal with a stride-aligned 59 s input:

trailing silence
before 4000 samples (250 ms)
after 0

Envelope-step attenuation against the hard splice, whose step is 1.0: 38.17 dB at a 5 ms fade, 50.13 dB at 20 ms, 78.06 dB at the shipped 500 ms default.

A linear, amplitude-complementary fade is correct here and equal-power is not: ZipEnhancer's overlapping chunks see the same input samples through a deterministic model, so the two sides of a join are strongly correlated and equal-power would put a +3 dB bump at every seam.

RNNoise: 20 ms of uncompensated output delay

Every processed file was time-shifted 960 samples against its input — which matters for any A/B, any mix against the original, and any downstream alignment. deepfilternet2.cpp already crops its own 480-sample delay; RNNoise did not.

It now pads the tail and crops the front the same way. An impulse in the final 20 ms of input previously vanished from the output entirely; the test asserts it now survives at its input index with zero stray energy.

FlashSR: peak normalisation destroyed the input level

It normalised every output to 0.999, so one loud sample rewrote the level of the whole file — a −40 dBFS whisper and a −3 dBFS shout both came back at −0.009 dBFS.

The ceiling is now a limit rather than a target and the input peak is restored: a 37 dB input swing survives as a 37 dB gain swing, where the old policy flattened five test cases onto 0.999.

Note on the reference fixtures

The fixtures under tests/unittests/assets were captured from the upstream implementations, which carry the delay and the normalisation. test_rnnoise_utility and test_flashsr_utility now ask for legacy behaviour explicitly, so they remain parity tests. No fixture was regenerated.

Validation

Build

cmake -S . -B build -DENGINE_BUILD_TESTS=ON
cmake --build build

Test

ctest -R enhancement_alignment_test

tests/unittests/test_enhancement_alignment.cpp — 8 cases, no model weights needed. Sweeps 44 input lengths around the tail case and asserts overlap-add weight is exactly 1.0 with no trailing zero run, carries the old give-up coverage as a regression witness so the bug is proven to have existed, checks the fade envelope is monotonic and complementary to 1e-6, and pins the RNNoise crop arithmetic and the FlashSR gain policy.

Run command used for the Metal A/B

audiocpp_enhance --backend metal --denoise zipenhancer --in in.wav --out out.wav

Backend tested: Metal and CPU. Full suite 39/39 on macOS/Metal, Apple M4 Max.

Affects

Output changes for ZipEnhancer's segmented path (>6 s input) — that is the fix. The un-segmented path (2–6 s) returns up to 99 more samples; the samples the old path returned are bit-identical and the new ones are the genuine tail. RNNoise output shifts 20 ms earlier. FlashSR output level changes for every file, so anything downstream that assumed near-full-scale FlashSR output will now be quieter by however much headroom the input had.

Known limitations

  • ZipEnhancer's per-block RMS normalisation is still per-block, so the two sides of a join are normalised independently. The crossfade now smears the resulting level step over the overlap instead of concentrating it in one sample, but making the gain continuous across chunks is a redesign of the normalisation and wants a listening test first.
  • FlashSR still measures its peak over the whole file, so it cannot stream.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ATa5YkLUPMDPRL7w1gCo9p

Three independent defects in the arithmetic around the enhancement
models, not in the models themselves. Each is reachable in its previous
form through a new options struct, so existing call sites are unchanged
unless they ask otherwise.

ZipEnhancer spliced its segmented output with std::copy every 1.5 s and
left up to 250 ms of digital silence on the tail for roughly one input
length in six -- the lengths where (n - 32000) % 24000 == 0. Replaced
with the accumulate-and-normalise overlap-add that deepfilternet2.cpp
already uses. Accumulated weight is now exactly 1.0 on every returned
sample and the output length matches the input.

Measured on Metal with a stride-aligned 59 s input:

  before   trailing silence 4000 samples (250 ms)
  after    trailing silence 0

Envelope-step attenuation against the hard splice, whose step is 1.0:
38.17 dB at a 5 ms fade, 50.13 dB at 20 ms, 78.06 dB at the shipped
500 ms default.

A linear, amplitude-complementary fade is correct here and equal-power is
not: ZipEnhancer's overlapping chunks see the same input samples through
a deterministic model, so the two sides of a join are strongly
correlated and equal-power would put a +3 dB bump at every seam.

RNNoise left its 20 ms synthesis delay in the output, so every processed
file was time-shifted 960 samples against its input -- which matters for
any A/B, any mix against the original, and any downstream alignment.
deepfilternet2.cpp already crops its own 480-sample delay; RNNoise now
pads the tail and crops the front the same way. An impulse in the final
20 ms of input previously vanished from the output entirely; the test
asserts it now survives at its input index with zero stray energy.

FlashSR peak-normalised every output to 0.999, so one loud sample
rewrote the level of the whole file and the input's own level was
discarded -- a -40 dBFS whisper and a -3 dBFS shout both came back at
-0.009 dBFS. The ceiling is now a limit rather than a target and the
input peak is restored: a 37 dB input swing survives as a 37 dB gain
swing, where the old policy flattened five test cases onto 0.999.

The reference fixtures under tests/unittests/assets were captured from
the upstream implementations, which carry the delay and the
normalisation, so test_rnnoise_utility and test_flashsr_utility now ask
for legacy behaviour explicitly and remain parity tests.

Tests: tests/unittests/test_enhancement_alignment.cpp -- 8 cases, no
model weights needed. Sweeps 44 input lengths around the tail case and
asserts overlap-add weight is exactly 1.0 with no trailing zero run,
carries the old give-up coverage as a regression witness, checks the fade
envelope is monotonic and complementary to 1e-6, and pins the RNNoise
crop arithmetic and the FlashSR gain policy.

Build: cmake -S . -B build -DENGINE_BUILD_TESTS=ON && cmake --build build
Test:  ctest -R enhancement_alignment_test
Backend tested: Metal (audiocpp_enhance --backend metal --denoise
zipenhancer) and CPU; full suite 39/39.

Known limitation: ZipEnhancer's per-block RMS normalisation is still
per-block, so the two sides of a join are normalised independently. The
crossfade now smears the resulting level step over the overlap instead of
concentrating it in one sample, but making the gain continuous across
chunks is a redesign of the normalisation and wants a listening test
first. FlashSR still measures its peak over the whole file and so cannot
stream.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ATa5YkLUPMDPRL7w1gCo9p
@0xShug0
0xShug0 marked this pull request as draft August 31, 2026 00:37
@0xShug0
0xShug0 marked this pull request as ready for review September 1, 2026 16:48
@CryptVenture

Copy link
Copy Markdown
Contributor Author

Closing for now to stay inside the 3-concurrent-PR policy (see the discussion on #422). This one has also gone stale against main; I will rebase it and resubmit once a review slot is free.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant