feat(audio): add advanced audio processing kernels (v0.2.12) #99
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add GPU-accelerated audio processing operations for ASR/Whisper preprocessing: - from_pcm: Convert int16/float32 PCM to GPUArray - stereo_to_mono: Convert stereo to mono - normalize_peak/rms: Audio level normalization - resample: 48kHz -> 16kHz polyphase resampling Also adds Int16 DataType support and updates build.sh default to SM 120 / CUDA 12.9. Tests: 11 new audio tests, all pass Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add GPU ring buffer kernels (write/read with wrap-around) - Add Hann windowing for overlap-add processing - Add overlap_add kernel for stream output reconstruction - Create AudioRingBuffer class for real-time audio buffering - Create AudioStream class for chunked processing with windowing - Support configurable chunk size and hop size (default 50% overlap) - Add 9 new streaming tests (20 total audio tests pass) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add GPU-accelerated VAD with energy and zero-crossing rate features - Implement frame-level feature computation kernels - Add threshold-based decision with adaptive noise floor estimation - Support hangover smoothing to extend speech regions - Create VAD class with configurable parameters - Return SpeechSegment objects with sample/time boundaries - Add 7 new VAD tests (27 total audio tests pass) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add comprehensive audio DSP kernels for Whisper/ASR support: Spectral processing (high priority): - Custom Radix-2 FFT (Cooley-Tukey + Stockham) - no cuFFT dependency - STFT with configurable window/hop size - Power/magnitude spectrum computation - Mel filterbank creation and application - Log-mel spectrogram and dB conversion - MFCC extraction with DCT-II - Delta/delta-delta features Audio preprocessing (medium priority): - Pre-emphasis and de-emphasis filters - DC offset removal - High-pass filter - Noise gate (amplitude-based) - Spectral gate (frequency-domain) - Short-term energy computation High-level API: - mel_spectrogram() - one-call mel spectrogram - log_mel_spectrogram() - one-call log-mel extraction All kernels maintain Driver-Only mode (CUDA::cuda_driver only). 24 new tests added, all 229 tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add comprehensive audio processing operations (Driver-Only, no cuFFT): - ISTFT: Inverse STFT with overlap-add and window normalization - Griffin-Lim: Iterative phase reconstruction from magnitude - Pitch detection: Autocorrelation and YIN algorithm - Spectral features: centroid, bandwidth, rolloff, flatness, contrast - Zero-crossing rate: Frame-based ZCR computation - CQT: Constant-Q Transform via STFT bin interpolation - Chromagram: Both STFT-based and CQT-based - HPSS: Harmonic-Percussive Source Separation with median filtering - Time stretch / Pitch shift: Phase vocoder implementation All kernels use custom Radix-2 FFT (no CUDA Toolkit runtime dependency). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add demo_v0212.py showcasing advanced audio processing - Update README with v0.2.12 audio features - Bump version to 0.2.12 New audio operations: - ISTFT, Griffin-Lim phase reconstruction - Spectral features (centroid, bandwidth, rolloff, flatness, contrast) - Pitch detection (YIN), autocorrelation - CQT, chromagram - HPSS (harmonic-percussive separation) - Time stretch, pitch shift 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Add comprehensive GPU audio processing operations with custom Radix-2 FFT (no cuFFT dependency).
New Audio Operations
istft,griffin_limspectral_centroid,spectral_bandwidth,spectral_rolloff,spectral_flatness,spectral_contrastdetect_pitch_yin,detect_pitch_yin_frames,autocorrelationcqt,chroma_stft,chroma_cqt,zero_crossing_ratehpss,harmonic,percussivetime_stretch,pitch_shiftChanges
audio_kernels.cuh)audio.cu,audio.hpp)ops_bindings.cpp)audio.py)demo_v0212.py)Key Features
Test plan
🤖 Generated with Claude Code