The idea: audio does not need to understand a video to help an agent navigate it. Squish now uses audio activity as a timestamped attention signal, then asks vision to zoom and confirm the evidence.
In 0.3.1 (over 0.2.3):
- Audio-activity map for local CLI + MCP — every contact sheet can carry a normalized energy band aligned to the same absolute timeline as its frames. JSON results add
audio.samples[]without changing the frozensquish-cli-v0andsquish-mcp-v0contract names. - Full-clip normalization across zooms — windowed runs remain comparable to their overview because activity levels are normalized against the source clip peak, not only the selected window.
- Agent navigation discipline — the
video-navigationskill now treats audio only as a candidate address: overview → candidate → absolute-time zoom → visual confirmation → timestamped answer. - High-frequency preservation — extraction now preserves continuous activity above the original envelope Nyquist limit; the regression is covered by a dedicated test.
- Public reproduction —
examples/audio-navigationincludes a redistributable 24-second fixture where a brief audio transient proposes a range and a short pink marker becomes visible only after zooming.
Important scope: audio activity is energy only. It is not transcription, speech recognition, sound classification, speaker identification, emotion inference, or semantic understanding. This release adds audio only to the local CLI and local STDIO MCP. The PWA, hosted API, remote MCP, and submitted ChatGPT App remain visual-only.
Install: npx -y @getsquish/squish@0.3.1 <video> --json
Local MCP: npx -y @getsquish/squish@0.3.1 mcp
Agent skill: npx skills add getsquish/squish
Requires Node 20 or newer plus ffmpeg and ffprobe on PATH. Clean package and MCP-handshake CI runs on macOS and Ubuntu Linux.
Reproduce the audio-navigation loop · Made at getsquish.app 🦑