Proposal: opt-in low-level background audio filter before streaming ASR

[aaldrich]

Problem

During a long push-to-talk dictation, extremely faint background audio can reach the streaming recognizer and produce unwanted live previews and final text even when the user never speaks.

I reproduced this with a wireless microphone while a television played faintly in the background:

1. Select a voice engine that supports streaming previews.
2. Begin push-to-talk and remain silent for more than four seconds.
3. Continue holding until a live preview appears.
4. Release push-to-talk.

In one 27.5-second reproduction, the captured audio averaged approximately -76.9 dB and satisfied FluidVoice's existing clear-silence thresholds, but the streaming recognizer still interpreted the faint television audio as speech and inserted text.

The existing Skip Silent Recordings option does not address this path. It is a post-capture check for recordings up to four seconds, while streaming ASR receives the raw PCM during recording.

Proposed direction

Add a separate opt-in setting:

Filter Low-Level Background Audio

Prevent faint background audio from reaching live or final transcription. May suppress unusually quiet speech.

When enabled, a lightweight activity gate would process the 16 kHz mono PCM before it enters streaming or final ASR:

• Evaluate audio in 20 ms frames.
• Require 40 ms of sustained activity before opening.
• Preserve approximately 160 ms of pre-roll so word beginnings are not clipped.
• Preserve approximately 360 ms of hangover so word endings and brief pauses remain intact.
• Close again during sustained low-level sections.
• Return no transcription when no activity is accepted.
• Leave level metering, microphone readiness, and dictionary-training capture unchanged.

This would be separate from Skip Silent Recordings, which would retain its current behavior. The new filter would be disabled by default because unusually quiet speech could otherwise be suppressed.

Validation completed

I have a candidate implementation ready, but I am starting with this Discussion before opening a pull request.

It has been tested against:

• sustained faint background audio;
• speech immediately above the activity threshold;
• speech onset and pre-roll;
• isolated transients;
• long pauses between speech bursts;
• arbitrary audio callback boundaries;
• state reset between sessions; and
• independence from Skip Silent Recordings.

Strict SwiftLint and the full macOS Apple Silicon Xcode test suite pass. The real-world silent-TV reproduction is rejected while normal speech and speech separated by pauses remain intact.

Questions

1. Would maintainers accept this as a separate opt-in setting?
2. Is Filter Low-Level Background Audio an appropriate name?
3. Should the initial thresholds remain fixed and conservative, or should the design use an adaptive noise floor?

AI assistance disclosure

The candidate implementation was developed with AI assistance. The resulting patch has been validated through synthetic regression tests, strict linting, the complete Apple Silicon test suite, and real-world reproduction testing. I will remain responsible for responding to review feedback and maintaining the contribution.

[altic-dev]

Seems like a useful one for sure if we can make it without increasing latency in any way and prove the approach works well :))

Happy to collab on a PR if you're down!!

[aaldrich]

@altic-dev I've been using it for a day and while watching Football with a lot of talking via the announcers and it is working very well. I can have long pauses and it rarely picks up the announcer talking.

[aaldrich]

Thanks @altic-dev — absolutely, and thank you for being open to collaborating on this.

I benchmarked the exact candidate gate on Apple Silicon before opening the PR:

• FluidVoice’s normal 4,096-frame / 48 kHz callback: 0 ms additional audio-delivery latency
• Gate processing: 102 µs median / 138 µs p95
• Processing uses 0.162% of the normal callback budget
• Synthetic smaller packets: bounded activation effect of +30 ms at 10 ms, +20 ms at 20 ms, and 0 ms at 40 ms or larger
• The 160 ms pre-roll preserves the speech onset
• Strict SwiftLint passed
• Full macOS Apple Silicon suite passed: 540 tests, 0 failures

Benchmark report and method: https://github.com/aaldrich/FluidVoice/blob/pr-assets/low-level-background-audio-filter/latency-benchmark.md

I am preparing the PR now and will link it back here. The filter remains a separate opt-in setting, disabled by default, and the existing Skip Silent Recordings behavior stays unchanged.

[aaldrich]

Thanks again, @altic-dev. The PR is now ready for review: #993

It links back to this Discussion and includes the opt-in setting, regression coverage, Settings screenshot, benchmark report, and AI-assistance disclosure. I am happy to adjust the naming, thresholds, or architecture based on your review.