feat(audio): analysis pass and audio-reactive components#59
Merged
Conversation
- AudioAnalysis (per-video-frame RMS amplitude + 16 log-spaced bands 20Hz-16kHz, 0..1 normalized globally per track) cached in core via the video_frame_cache pattern; computed in rustmotion (Symphonia decode, Hann-2048 rustfft) at the same call sites as preextract_video_frames (both encode paths, incremental, studio) - new audio_spectrum component (bars/radial, resampled bar count, graceful min_height degradation) and waveform (line/filled, sliding window; draw_style field name since style is the universal CssStyle field) - generic audio-reactive style binding: amplitude or band source mapped onto opacity/scale/translate_y/rotation with lerp(min,max) and optional moving-average smoothing; deterministic min when no analysis is cached - audio-reactive skill rule; rustfft added to the rustmotion crate only (pure Rust)
10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #47 — the music-visualization family.
rustmotioncrate only), cached in core via the establishedvideo_frame_cachepattern, filled at the same call sites aspreextract_video_frames(both encode paths, incremental, studio preview).audio_spectrum(bars/radial, resampled bar count, bottom-anchored / center-radial) andwaveform(line/filled, sliding window centered on now; field nameddraw_style—styleis the universal CssStyle field). Missing analysis degrades gracefully (min-height bars / flat line), never panics.audio-reactivestyle binding:{track, source: "amplitude"|{"band": n}, property: opacity|scale|translate_y|rotation, min, max, smoothing_frames}— lerped and composed after animations inbuild_child; deterministicminwith an empty cache.Tests: 6 new — a hand-written WAV fixture (440Hz sine + silence) through the production load path asserting amplitude envelope and 3× band dominance; synthetic-cache pixel tests for spectrum/waveform/binding (hermetic, unique keys per test); corpus entries enforced by the completeness test. 296 total, all green.
Verify:
cargo test --workspace→ 296 passed ✓ ·cargo fmt --check✓ · clippy 0 warnings (--all-targets) ✓