Bug-hunt round 46: missing ffprobe dependency, curl -f omission, stale CHANGELOG claim - #65
Merged
Conversation
install_ffmpeg_local's Linux branch already downloads and verifies the johnvansickle static tarball, which bundles ffprobe next to ffmpeg, but only ever installed ffmpeg — ffprobe was extracted and then discarded. transcribe -audio's offset derivation shells out to ffprobe to read a recording's creation_time tag; without it on PATH, every external recording silently falls back to a 0 offset, indistinguishable from a recording whose timestamp tag is genuinely missing or unreadable. docs/explanation/how-alignment-works.md and docs/reference/cli.md only documented the tag-based causes of that fallback, so a reader hitting this (most likely via getting-started.md's documented no-Homebrew "local" install path) had nothing pointing at the actual cause. Both pages now name a missing ffprobe as a cause alongside the tag being missing or unreadable. The macOS local-install branch still does not install ffprobe (evermeet publishes it as a separate signed download, requiring a second fetch/verify/install sequence this round leaves unattempted rather than add an unverified network path) — untouched, and now accurately undocumented as no longer the only doc gap. Assisted-by: Claude:claude-sonnet-5
resolveModel's not-found guidance and its mirror in docs/how-to/transcribe-a-recording.md both printed `curl -L --create-dirs -o ... <url>` for the ggml model download, omitting -f/--fail. Without it, curl treats an HTTP error response (a moved or withdrawn Hugging Face asset, a transient CDN error page) as a successful transfer and writes the error body into the destination .bin file at exit 0 — the reader gets a "successful" download that whisper-cli only rejects later, confusingly, at model-load time. install.sh's own equivalent guidance already uses `curl -fL` for the identical recipe; both sites now match it. New TestResolveModelGuidanceFailsClosedOnHTTPError pins the printed guidance to `curl -fL`, confirmed to fail before this change and pass after. Assisted-by: Claude:claude-sonnet-5
The testimony analyze bullet under the released [0.2.0] CHANGELOG section closed with an unscoped "The CLI holds no API keys and makes no network calls" — the sole remaining unscoped instance of this claim; every other occurrence in the repo (README.md, AGENTS.md, docs/reference/cli.md, docs/how-to/analyse-a-session.md, and others) uses the deliberately scoped "analyze never calls a model, holds no keys, and adds no network dependency" form, precisely because the CLI-wide reading is false: demo's rrweb recorder loads from a public CDN, and transcribe's ASR engines fetch model files over the network. That CDN load already existed at v0.2.0's release, so the unscoped reading was inaccurate on the day it was written, not just since. Rescoped the bullet's closing sentence to analyze specifically, which the surrounding bullet is already about and which the claim is true of, without altering the substance of the release's historical record. Assisted-by: Claude:claude-sonnet-5
Assisted-by: Claude:claude-sonnet-5
Two issues raised by this round's adversarial docs-accuracy review: - docs/reference/cli.md's new ffprobe provenance bullet used "whose ffprobe was not found on PATH", which grammatically attaches ffprobe to the recording rather than to the environment running transcribe. Reworded to "for which ffprobe was not found on PATH". - .abcd/work/DECISIONS.md's round 46 entry claimed "every other occurrence [of the network-calls claim] uses the deliberately scoped analyze-only form" to justify the CHANGELOG.md fix — false: AGENTS.md, internal/analyze/analyze.go, and .abcd/work/CONTEXT.md all keep "The CLI" as the subject. The actual distinguishing feature is the predicate, not the subject: those three qualify it to "adds no network dependency" (a narrower, still-true claim about the analysis layer's own design), where CHANGELOG.md's [0.2.0] entry paired a CLI-wide subject with the stronger, falsifiable "makes no network calls". Corrected the entry to name the actual distinguishing qualifier; the CHANGELOG.md fix itself (rescoping to `analyze`, which the stronger predicate is genuinely true of) is unaffected. Assisted-by: Claude:claude-sonnet-5
This was referenced Aug 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Round 46 of the autonomous bug-hunt loop (state tracked on #24).
Four parallel dimension sweeps (code bugs in
cmd//internal/; docs vs functionality; infrastructure; internal doc consistency) surfaced three candidates. All three survived two independent adversarial refuters each and are fixed here; all are nitpick-severity (no substantive/behaviour-breaking defect this round).Confirmed findings (3 nitpicks)
1. Local Linux install never installs
ffprobe, and docs don't disclose it as a fallback causeinstall.sh'sinstall_ffmpeg_local(Linux branch, was around line 343) downloads and verifies the johnvansickle static ffmpeg tarball, which bundlesffprobealongsideffmpeg, but only ever installedffmpeg—ffprobewas extracted and discarded.internal/transcribe/ffmpeg.go'sderiveOffset(~line 207) doesexec.LookPath("ffprobe")and silently returnsoffset=0, falsewhen it's missing — a third, undocumented cause of thedefault 0: audio creation time unavailablefallback, alongside a missing/unreadablecreation_timetag.docs/explanation/how-alignment-works.md:19anddocs/reference/cli.md:67named only the tag-based causes. A reader who followsdocs/tutorials/getting-started.md:19-20's documented no-Homebrew "local" install path on Linux getsffmpegwith noffprobe, and every subsequenttranscribe -audiosilently falls back to offset 0, permanently, with nothing pointing at the actual cause.Fixed:
install.shnow installsffprobefrom the same already-downloaded, already-verified tarball. Both docs pages now name a missingffprobeas a cause. The macOS local-install branch (evermeet publishesffprobeas a separate signed download) is left as-is — adding a new, untested network-fetch-and-verify path was judged out of proportion for this round — and is now accurately reflected as the residual gap.2.
whisper.cppmodel-download guidance and its doc mirror omitcurl -finternal/transcribe/whispercpp.go:141(resolveModel's not-found error) anddocs/how-to/transcribe-a-recording.md:54both printed acurl -L ... -o ... <url>recipe for the ggml model download, omitting-f/--fail.-f, an HTTP error response (a moved or withdrawn Hugging Face asset) is written into the destination.binfile and curl exits 0 — confirmed by direct local test against a 404. The reader gets a "successful" download that only fails, confusingly, atwhisper-cliload time.install.sh's own equivalent guidance for the same URL and destination already usedcurl -fL(it differs from the other two only by--create-dirsvsmkdir -p, an intentional equivalent).Fixed: both sites now use
curl -fL, matchinginstall.sh. NewTestResolveModelGuidanceFailsClosedOnHTTPError(internal/transcribe/transcribe_test.go) pins the guidance text, confirmed to fail before the change and pass after.3. Released
CHANGELOG.md[0.2.0]entry pairs a CLI-wide subject with an unqualified network-calls claimCHANGELOG.md'stestimony analyzebullet under the released## [0.2.0]section closed with "The CLI holds no API keys and makes no network calls." This is the only occurrence of the phrase family in the repo that is both CLI-wide and unqualified: three occurrences scope the subject toanalyze(README.md,docs/reference/cli.md,docs/how-to/analyse-a-session.md), and three keep "The CLI" as subject but qualify the predicate to "adds no network dependency" (AGENTS.md,internal/analyze/analyze.go,.abcd/work/CONTEXT.md) — a narrower, still-true claim about the analysis layer's own design. CHANGELOG's blunter "makes no network calls" is a claim of zero runtime network activity anywhere in the CLI, falsified bydemo's rrweb CDN load (already present at v0.2.0's release) andtranscribe's ASR model fetch.Fixed: rescoped the bullet's closing sentence to
analyze— the bullet's actual subject, and true of it at the stronger predicate — without altering the substance of the release's historical record.Dimensions with zero findings
Code bugs (
cmd/,internal/) and infrastructure (CI/release workflows vsAGENTS.md,install.shvs its own docs,examples/sample-sessionvs the schemas the code reads/writes) both came back clean after full sweeps against the current code, with no re-litigation of anything already fixed or rejected in.abcd/work/DECISIONS.md's prior 45 rounds.Verification
Two independent adversarial refuters per finding, each explicitly tasked with proving it wrong, unreproducible, or intended behaviour. All three findings survived both refuters (no split verdicts, no discards this round); the refuters' scrutiny narrowed and corrected each finding's framing before the fix (e.g. identifying that
whispercpp.go's guidance string, not the docs, was the un-reviewed baseline for finding 2).This round's own merge-gate review (docs-accuracy lens) additionally caught an earlier draft of finding 3's writeup overclaiming that "every other occurrence uses the analyze-scoped form" and a grammatically mis-attached "whose ffprobe" in the finding-1 doc edit; both are corrected in this PR (see the
docs: fix merge-gate review findingscommit) rather than left standing.go build,gofmt -l .,go vet ./...,go test ./...,go test -race ./...all clean; pipeline smoke (merge+reportagainst a scratch copy ofexamples/sample-session) succeeded;sh -n install.sh && bash -n install.shand the CI installer flag-handling checks (--help,--dir,--bogus) all pass. Nogo.modchange..abcd/work/DECISIONS.mdcarries the round's entry.Assisted-by: Claude:claude-sonnet-5