Approve stdout and stderr separately in every approval test - #59
Merged
Conversation
The approval suite only captured stdout (the image list), so a change to what a manifest sends to stderr — an error appearing or disappearing — was invisible. Later PRs change exactly that (treating an unsupported kind as skippable; the CLI's exit/stderr behavior), and those changes should land as reviewable golden-file diffs. Add a shared verify() helper that approves stdout and stderr as two separate golden files, and route every test (TestKind, TestError, TestMultiple) through it. Expected behavior now lives entirely in the goldens: success cases carry an empty stderr golden, and the Service case carries the error text. This needs go-approval-tests v1.5.0 for WithAdditionalInformation; received files are gitignored. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pc6NAURAqjU4LYJx93tgSC
This was referenced Jul 23, 2026
MPV
force-pushed
the
claude/approve-stdout-stderr
branch
from
July 23, 2026 08:38
a0611cd to
abb45b2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Give the approval suite a shared
verify(t, file)helper that approves what the tool prints to stdout (the images) and to stderr (the error message, if any) as two separate golden files, and route every test through it.Requires bumping
go-approval-testsv1.2.0 → v1.5.0 forOptions().WithAdditionalInformation(...)..received.txtfiles are gitignored.Why
The suite only captured stdout, so a change to what a manifest sends to stderr — an error appearing or disappearing — was invisible. Later PRs change exactly that (treating an unsupported kind as skippable; the CLI's exit/stderr behavior), and those changes should land as reviewable golden-file diffs rather than silent shifts.
What's in the diff
Nine
verify()call sites, each gaining a.stdout+.stderrgolden (the old single-stream.approved.txtis renamed to.stdout.approved.txt, and a.stderr.approved.txtis added):TestKindkinds andTestMultiple— carry an empty.stderrgolden. That empty golden is the point: a regression that leaked to stderr on a passing manifest would now fail the suite.TestError.Service— carries a non-empty.stderrgolden holding the current text,error processing document: unsupported kind Service. This is the baseline that Skip non-workload documents instead of aborting the stream #54 later flips to empty when a Service becomes skippable.Behavior change
None — test-infrastructure only (test code +
go.mod/go.sum+.gitignore). Supersedes the draft #46.Note
Replaces #58, which I inadvertently closed — GitHub refuses to reopen a PR after its branch was force-pushed. Same branch (
claude/approve-stdout-stderr), same commit, correct diff.Stack
Base of both stacks: parsing (#49 → #54 → #56) and CLI (#50 → #55 → #57).
🤖 Generated with Claude Code
https://claude.ai/code/session_01Pc6NAURAqjU4LYJx93tgSC
Generated by Claude Code