Repository navigation
This is the first published release of the 0.2 series.
Upgrading from 0.1
| 0.1 | 0.2 |
|---|---|
AudioPipelineCoordinator::new_with_defaults() |
Pipeline::new()? |
AudioPipelineCoordinator::new(Arc::new(vad), chunker, dc, nr) |
Pipeline::with_components(vad, chunker, dc, nr)? |
coordinator.process_frame(&bytes) |
pipeline.process_bytes(&bytes) (&mut self; .chunks holds the cleaned chunks) |
VadDetector::new(config, metrics) |
VadDetector::new(config) or VadDetector::with_metrics(config, metrics) |
VADStats |
vad::VadStats |
VadConfig { activation_margin, release_margin, energy_weight, flux_weight, .. } |
VadConfig { activation_ratio, release_ratio, .. } |
chunk.overlap_next |
the next chunk's overlap_prev, or chunk.samples_with_context() |
AudioFormat::WebM |
AudioFormat::Vorbis or AudioFormat::Opus |
AudioBuffer, AudioChunk, the fixtures feature |
removed |
Error and AudioFormat are #[non_exhaustive], so a match on them
needs a wildcard arm. Opus needs features = ["opus"] (Rust 1.95).
Added
- Decoding of FLAC, ALAC, MP3, AAC (MP4/M4A), Ogg/WebM Vorbis, AIFF, CAF,
and ADPCM through Symphonia 0.6, plus 8-bit, 32-bit, float, mu-law, and
A-law WAV. Any channel count is downmixed to mono. AudioFormatConverter::convert_filefor reading from a path.Pipeline::process_samplesfor already-decoded 16 kHz audio, and
Pipeline::reset.VadConfig::min_speech_energyto tune the absolute level below which a
frame is never speech.VadDetector::with_metricsfor attaching a metrics collector.Pipeline::with_max_speech_hold(default 2 s): long uninterrupted speech
is emitted in chunks while it is still going instead of after it ends.ProcessedChunk::samples_with_context: the previous chunk's tail followed
by this chunk's samples. In the pipeline, the context also carries across
process_samplescalls.VadDetector::flushto analyze the last partial frame and close a segment
that runs to the end of the stream.Error::Unsupportedfor recognized formats that can't be decoded, such
as Opus without theopusfeature (the message names the feature).- Opus decoding (Ogg and WebM) behind the optional
opusfeature, using the
pure-Rustruopusdecoder. Its output matches libopus to within float
rounding (121 dB SNR on speech). It is off by default because it adds
about 20 crates and needs Rust 1.95. cargo bench --bench throughput, aprocess_fileexample, and
cargo fuzztargets for decoding and the pipeline.
Changed
- Breaking:
AudioPipelineCoordinatoris nowPipeline. Its methods
take&mut selfand return the cleanedProcessedChunks instead of only
a count.process_frameis nowprocess_bytes. - Breaking:
VadDetector::newtakes only aVadConfig. - Breaking:
VADStatsis nowvad::VadStats. - Breaking: the VAD decides on frame energy relative to a tracked noise
floor and nothing else. The spectral-flux score, combined threshold and
their settings never changed a decision once the relative energy gate was
in place, so they are gone:VadConfiglosesenergy_smoothing
(nownoise_smoothing),flux_smoothing,flux_floor,
threshold_smoothing,activation_margin,release_margin,
base_threshold,energy_weight,flux_weightandfft_size(), and
gainsactivation_ratio(default 2.5) andrelease_ratio(default 1.3).
The hysteresis between them raises F1 on real speech from 0.946 to 0.958
and splits speech in heavy noise into fewer pieces.
VadMetricsSnapshot::adaptiveandAdaptiveThresholdSnapshotare
replaced byVadMetricsSnapshot::noise_floor. SpeechChunk::confidenceis now how far the segment's speech rises above
the noise floor (0 at the floor, 1 at 20 dB above). It used to be 1.0 for
almost every segment.- Breaking:
ConversionMetadatareports a singleconversion_timeand
original_channelsis au16. - Breaking:
AudioFormatnames the codec, not the container, and is
#[non_exhaustive]:WebMis gone (a WebM file reportsVorbisor
Opus),AlacandAdpcmare new.FormatDetector::detectalways
probes the stream and fills in all metadata;
FormatDetector::detect_with_metadata,AudioMetadata::format_only, and
AudioMetadata::with_properties
are removed. - Decoding no longer copies the input buffer.
- Breaking:
DcHighPassFilter::processtakesOption<VadContext>by
value, likeNoiseReducer::reduce. - Breaking: configuration validation errors are
Error::Configuration
everywhere;Erroris#[non_exhaustive]. - The VAD noise floor is seeded from the quietest frame of the first
300 ms and keeps adapting during speech. Before, a recording that started
with noise above the energy gate was classified as one long speech
segment. - The default
hangover_framesis 20 (200 ms) instead of 3 (30 ms; the
docs said 60 ms), and hangover frames no longer extend a segment's end.
Segments get 50 ms of padding on each side (VadConfig::speech_padding,
new). On real speech in three languages, scored against Silero VAD and a
human annotation, a sentence is no longer split at every gap between
words (e.g. 6 segments instead of 56 on a 30 s meeting clip) and F1 goes
from 0.89 to 0.95. VadStats::speech_framescounts frames that measured as speech; hangover
frames now count as silence.- Because the noise floor now adapts, the default VAD energy gate is 40x
lower (0.0005 instead of 0.02). The old value missed 60% of speech
peaking at -6 dBFS and all speech at -20 dBFS in a synthesized test set. - Channels are mixed to mono before resampling instead of after.
- Resampling uses a Kaiser-windowed sinc filter instead of linear
interpolation. A 12 kHz tone at 48 kHz used to alias to 4 kHz at full
level; it is now attenuated by about 100 dB. - Licensing: the crate is now MIT only (0.1.x was MIT OR Apache-2.0).
- Edition 2024. Cargo now resolves dependencies against
rust-version, so
1.87 builds without pinning anything. - Library code no longer logs at
info/warnon every call or warns when a
stage takes longer than a fixed wall-clock budget.
Fixed
- Resampler response documentation now distinguishes downsampling, lower-rate
input filtering, and unfiltered 16 kHz passthrough; the codec support table
no longer claims mu-law or A-law decoding in CAF. - The noise floor's slow rise during speech now has the same time constant
(5 s) at any frame hop. It was a fixed per-frame factor, so with
frame_overlap: 0.75steady noise under speech was absorbed in half the
time, and with no overlap in twice the time. Defaults are unchanged. VadConfig::validaterejects an infinitemin_speech_energy, which passed
the sign check and silently turned detection off.PreprocessingConfig::validaterejects a non-finite high-pass cutoff or
dc_bias_alpha, which previously passed and turned the filter output into
NaN.- A
ChunkerConfigbuilt as a struct literal with a sub-millisecond
target_durationis nowError::ConfigurationfromChunker::chunk
instead of a divide-by-zero panic. NoiseReducernow carries the last frame of the stream as real left-hand
context and keeps its STFT frames on one hop grid across calls, so a
continuous signal split across calls (at any sample, in blocks of any size)
no longer has a discontinuity at the leading edge of each chunk once a noise
profile is learned (previously each call zero-padded its own left edge and
restarted its frame grid). The trailing
edge of a call still lacks future context and is re-covered on the next
call; eliminating that residual entirely would require an output-lagging
(buffered) API, which is a separate change.- A truncated stream (cut mid-packet) is now rejected with
InvalidInput
instead of returning the partial audio decoded so far asOk. - Symphonia 0.6.1 or newer is required so a malformed metadata length cannot
trigger an allocation for a payload that is absent from the input. - The decoder caps how many samples one decode may buffer (64 Mi interleaved,
256 MiB), and how many 16 kHz samples it will resample to, so a
decompression bomb — a tiny compressed file declaring hours of audio — is
refused instead of allocating gigabytes. - The pipeline no longer emits the start of a speech segment that is still
in progress as a silence chunk. - Pipeline speech chunks emitted from a window with no silence of its own
(long speech held pastmax_speech_hold, or the end of a stream) had no
snr_db. They now use the level of the last silence in the stream. NoiseReducerno longer fades each chunk in from near zero at its first
samples (about -23 dB on a steady tone with 500 ms chunks).- The fourth-order high-pass filter now has Butterworth stage Q values; the
cutoff is at -3 dB instead of -6 dB. VadStats::speech_framescounted segments instead of frames, and
silence_frameswas always zero.QualityMetrics::spectral_centroidis now computed from the spectrum. It
used to be a weighted time index, not a frequency.VadDetector::resetnow also resets the noise floor.VadDetector::detectkeeps samples that don't fill a frame for the next
call. Before, each call started a new frame grid, so blocks under about 64
samples never detected speech.- With
frame_overlapabove 0.5, consecutive speech segments could overlap
and the chunker emitted the overlapping audio twice. Segments are now
disjoint, andChunkermerges overlapping input segments. - Decoding no longer returns empty or partial audio without an error for
chained Ogg streams, or for files where many packets fail to decode.
Same-layout chained streams decode fully; a sample rate or channel count
change isError::Unsupported. - A panic inside Symphonia on malformed input (found by fuzzing: a
Matroska element with a zero length byte) is returned as
Error::InvalidInputinstead of unwinding into the caller. StandardAudiosamples are now guaranteed finite and within[-1, 1]:
NaN or infinite input is an error, and louder-than-full-scale float input
or resampler overshoot is clamped.- Inputs declaring a sample rate outside 4-768 kHz are rejected with
Error::Unsupported. Fuzzing found a 10 KB file claiming 1.04 GHz that
made the resampler build a 1.7 GB filter and run for 40 s. NoiseReductionConfig::validaterejects a hop shorter than one sample,
which madereduceloop forever.- Symphonia 0.6 fixes an untrusted allocation in the MP4 reader: a 5 KB
M4A with a corrupt sample table could make 0.5 try to allocate 32 GB and
abort the process (found by fuzzing).
Removed
- Breaking:
buffer::AudioBuffer,types::AudioChunk, thefixtures
feature andfixturesmodule,Chunker::default()as an inherent method
(useDefault),ProcessedChunk::samples_without_overlap,
ProcessedChunk::overlap_next(it duplicated the next chunk's
overlap_prev),ProcessedChunk::total_samples_with_overlap,
ConversionMetadata::has_performance_issue,
AudioFormat::is_container_format,ProcessingResult::backpressure_active,
andStageLatencies::broadcasting_avg. glob,serde_json,parking_lot, andranddependencies;houndis
now dev-only.