2.5.0 — the downstream-tools API

A surface for tools built on top of the harness, plus two recorder fixes and a memory fix.

New, for downstream tools

• transcribeFile(bytes, opts) and soundcheck stt <file> — transcribe a whole file and get the full result: transcript, confidence, the word timeline (start, end, confidence, punctuated form), optional utterance segments, and the media duration. Repeatable --keyterm, --utterances, content type inferred from the extension. Containerized audio declares its own encoding and sample rate, so neither is sent.
• judgeText(transcript, rubric, backend?) and soundcheck judge --transcript <file> — run a rubric against text from anywhere, with no scenario and no trace rendering.
• similarity(expected, heard, { ignoreFillers }) — a 0..1 closeness score on the same canonicalization the gate uses, folding contractions and optionally dropping disfluencies first.
• findPhrase(words, expectedText, { from, to }) — locate an expected line in a word timeline within a time window, for offset and boundary checks.
• --offline on every command — a hard refusal at every network entry point, REST and WebSocket alike. The key resolves from four locations, so a dry run could otherwise reach the API; with this it cannot, and no key is required.

Fixed

• Audio dropouts in the whole-call recording. Agent audio arriving while the recorder was paused between turns was discarded, and the recorder zero-padded whatever had not arrived within each 100 ms tick — silent holes inside words. Audio is now always captured, the paused window is flushed contiguously on resume, and a jitter buffer with short seam ramps replaces the padding. A third instance of the same class was then found in the fix itself: a per-turn done-stamp went stale when one turn carried several speech segments, so the second segment was released as zero-padded slivers.
• The comparison diff no longer allocates a quadratic table. Small inputs keep the same recurrence and tie-break in a flat Int32Array; above four million cells it switches to Hirschberg's linear-space algorithm. A 10,000-token diff now takes 577 ms and 4 MB of heap.

Also

• Red-team scenarios (redTeam) and stopWhen: { toolCalled }, so a scripted call ends when the behavior under test occurs instead of reading lines that plead for something already done.
• New gallery demos: a realistic agent that refuses several times under a bereavement pretext and then waives verification, beside the well-built agent on the identical script with every gate green.

256 offline checks, no key and no network. Published to npm via trusted publishing with provenance.

2.4.0 — audio-loop verification

Round-trip audio-loop verification lands as first-class capability: the normalization-aware comparison (soundcheck compare), the 16-fixture canonical audio corpus (soundcheck fixtures check|roundtrip|generate), a content-comparing validate --tts, the new spoken_matches_text gate (11 gates total), an opt-in smart-format transcription option, a nightly recognition-drift check, and a key-resolution fix (an explicitly empty DEEPGRAM_API_KEY no longer falls back to .env files). 239 offline tests. See CHANGELOG.md for the full record. Published to npm via trusted publishing with provenance.

v2.3.0 — run --json failure contract (ears for coding agents)

soundcheck-cli 2.3.0

npm install -g soundcheck-cli     # the command is `soundcheck`

The agent-first release. A coding agent can't hear — so it converges on green text tests while the spoken agent is still wrong. run --json gives it the failure as machine-readable evidence it can patch from.

Added

• run --json — a machine-readable failure contract for coding agents and CI. soundcheck run … --json emits a stable, schema-versioned JSON document instead of making an agent scrape the HTML report or stdout. Per scenario: the gate results, the trace-driven diagnosis (the same evidence + remediation hint the tune fixer reads — patch from evidence, not gate names), oracleHeard (what the STT oracle heard), the optional judge verdict, and a reproduce command to re-run just that scenario; the run summary adds a deduped failingGates taxonomy. Bare --json writes JSON to stdout and routes all human output to stderr (stdout stays parseable); --json <file> writes a file and keeps the human output. Exit code unchanged (0 iff every gate passed).

  soundcheck run scenarios --aut ./agent.ts --replay --json | jq .summary

Changed

• Docs lead with the published install. The guide, command reference, tutorial, and coding-agent skill present npm install -g soundcheck-cli (the command stays soundcheck) as the primary install, with the source checkout kept for the bundled examples and contributing.

Full notes: CHANGELOG.

🤖 Generated with Claude Code

v2.2.0 — first public npm release (soundcheck-cli)

soundcheck-cli 2.2.0 — first public npm release

npm install -g soundcheck-cli     # the command is `soundcheck`

First release on the npm registry. Soundcheck is the open-source, repo-local CI harness for speech-to-speech voice agents: a synthetic caller is driven over real Deepgram audio, and an oracle (STT over the recording) plus deterministic gates decide pass/fail — Playwright-style regression tests over what your agent actually says.

Added

• spoken_consistent_with_tool gate — catches an agent verbally caving to a caller's wrong date (it says the right thing, then confirms a date it never booked) or an internally incoherent "weekday, month day" (e.g. "Thursday, June 2nd" when June 2 is a Tuesday). Wired into every date-bearing example.
• Sample gallery: real, unplanted catches — a well-built healthcare agent talked off its grounded date, and a support agent socially-engineered into the forbidden, irreversible deleteAccount. Surfaced by re-running scenarios under an adversarial caller; each report carries a "not planted" banner.
• Alphanumeric read-back verification in spoken_matches_tool — for an identifier an agent must read back (flight number, confirmation code), it checks the digit runs were spoken intelligibly, tolerating STT mishearing the letters.
• User-global Deepgram key fallback — getKey() now also checks ~/.config/soundcheck/.env, so a globally-installed soundcheck works from any directory without a local .env.
• Codex reference fixer for tune — examples/tune-demo/codex-fixer.sh drives the Codex CLI (read-only) as a --fixer; a real run went 0/1 → 1/1 on both the trained and an unseen held-out date.
• Coding-agent skill + soundcheck install-skill — a progressive-disclosure skill that teaches a coding agent to drive Soundcheck, with a smart multi-agent install default (Claude Code always; Codex/Gemini if present).
• Complete docs — GUIDE, COMMANDS, GATES, TUTORIAL, plus a structured README Getting Started.

Fixed

• bin/soundcheck.mjs chooses dist vs src by location, not file presence — a dev checkout always runs live src, so a leftover dist/ can't silently shadow your source; an installed copy still loads built dist.

Full notes: CHANGELOG.

🤖 Generated with Claude Code