Feature Contribution: Advanced Audio Processing — ML Noise Suppression, Noise Gate, Compressor

[cproudlock]

Feature Contribution

I've implemented three new audio processing features for the voice system and would like to offer them upstream. The implementation is complete and tested on my self-hosted instance.

Branch: cproudlock:echowire-refactor (diff)

I wasn't able to create a PR directly — if fork PRs are enabled, I'm happy to resubmit as a proper PR.

---

Summary

Adds three new audio processing features using @sapphi-red/web-noise-suppressor and the Web Audio API:

• ML Noise Suppression (RNNoise) — WASM-based neural network that filters background noise (fans, typing, traffic). Uses AudioWorklet for off-main-thread processing at 48kHz with ~13ms latency.
• Noise Gate — Mutes the microphone when audio falls below a configurable dB threshold (-100 to 0). Uses hysteresis (separate open/close thresholds) to prevent rapid toggling.
• Compressor — DynamicsCompressorNode that evens out loud and quiet audio for more consistent voice levels.

Processing Chain

Raw mic → RNNoise (if enabled) → Noise Gate (if enabled) → Compressor (if enabled) → LiveKit

When all three are disabled, no processor is attached — zero overhead.

Why not @livekit/krisp-noise-filter?

Krisp requires LiveKit Cloud and doesn't work with self-hosted LiveKit. @sapphi-red/web-noise-suppressor provides ready-made AudioWorklet nodes for RNNoise with WASM SIMD auto-detection, making it a drop-in solution for any deployment.

Files Changed

File	Description
pnpm-workspace.yaml	Add @sapphi-red/web-noise-suppressor to catalog
fluxer_app/package.json	Add dependency
fluxer_app/rspack.config.mjs	Copy worklet/WASM files to dist/audio-worklets/
fluxer_app/src/utils/AudioProcessor.tsx	New — TrackProcessor<Track.Kind.Audio> implementation
fluxer_app/src/stores/VoiceSettingsStore.tsx	4 new persisted settings
fluxer_app/src/stores/voice/VoiceMediaManager.tsx	applyAudioProcessor() method
fluxer_app/src/stores/voice/MediaEngineFacade.tsx	Expose applyAudioProcessor()
fluxer_app/src/actions/VoiceSettingsActionCreators.tsx	Trigger processor rebuild on setting change
fluxer_app/src/components/modals/tabs/VoiceTab.tsx	"Advanced Audio Processing" UI section

Settings Added

Setting	Type	Default	Description
advancedNoiseSuppression	boolean	false	ML noise suppression (RNNoise)
noiseGateEnabled	boolean	false	Noise gate toggle
noiseGateThreshold	number	-50	Gate threshold in dB (-100 to 0)
compressorEnabled	boolean	false	Dynamic compressor toggle

Implementation Details

• Uses LiveKit's TrackProcessor<Track.Kind.Audio> interface for clean integration
• AudioWorklet files + WASM binaries copied to dist/audio-worklets/ via rspack CopyPlugin
• Settings persisted via MobX persistence (IndexedDB), changes apply immediately mid-call
• Processor rebuilds graph on settings change; fully tears down when all processing disabled
• RNNoise uses SIMD auto-detection for best performance where available
• Noise gate uses 90ms hold time and -10dB hysteresis gap to prevent chatter

[nstgeorge]

Double check your branch. You may have leaked root passwords for some of your servers in your docs (even though the IPs look private).

[cproudlock]

Thanks for the heads up! Removed the password and scrubbed it from git history. Changing the passwords on the affected servers now.

[hoptional]

Thanks a lot for your work on this! The lack of a noise gate is what is keeping me from switching over from Teamspeak. I hope it will be merged.

[Kamalaja]

This will be available in v2 (and potentially earlier on canary).