fix(boatstack): key visual-evidence trust to product identity - #254
Merged
Conversation
Trusting a manifest only at an identical head commit had no fixpoint: publication requires committing the reviewed pr.md, which moves HEAD while the product diff (which excludes pr.md) is unchanged, so PASS evidence always degraded to NOT_VERIFIED before publish-pr could see it. Trust is now keyed to ProductDiffSHA256 alone; SourceCommit stays recorded for provenance and the preview template names the capture commit instead of assuming HEAD. Invariants pinned by tests: - committing pr.md never invalidates PASS evidence (status and fingerprint stable across the preview commit) - any product-diff change stales evidence immediately, and require coerces it to BLOCKED Disclosure-Reviewed: reviewed — public-safe only, private facet kept out of this commit
This was referenced Jul 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
resolvePRVisualEvidencetrusted a manifest only whenSourceCommit == headCommit && ProductDiffSHA256 == diffHash. That predicate has no fixpoint through publication:publish-prrequires committing the exact reviewedpr.md(clean-tree check), which moves HEAD — whileproductDiffexplicitly excludespr.md. So PASS evidence always degraded toNOT_VERIFIEDon the mandatory preview commit, re-capturing changed the manifest fingerprint, the regenerated preview needed a new commit, and the loop never converged. Underrequire, publication with current screenshots was unreachable. Latent today because no test publishes with PASS evidence.What changed
ProductDiffSHA256 == diffHash. Pixels may only depend on product content; committing the reviewed preview (excluded from the product diff) can never invalidate them, and any product change stales them immediately.SourceCommitstays recorded for provenance; the PR preview template's Visual-evidence Commit column now names the capture commit (falling back to HEAD when no manifest exists) instead of assuming HEAD.resolvePRVisualEvidencedrops the now-unusedheadCommitparameter;captureProductDiffdocs updated to state the trust rule.Invariants pinned
TestCommittingPreviewNeverInvalidatesPassVisualEvidence: PASS status and visual fingerprint are stable across the preview commit; the committed preview still checks; provenance keeps the capture commit and the template renders it.TestProductDiffChangeInvalidatesPassVisualEvidence: a different product diff →NOT_VERIFIED;requirecoerces toBLOCKED.Context
Slice 1 of 5 of the visual-evidence-as-deterministic-requirement program (auto-capture during ship, suggest→require escalation for plan-approved scenarios, attach-retry prescription, per-surface harnesses). This staleness fix is the enabler: auto-capture cannot be built on a trust rule that self-invalidates.
Verification
go build ./... && go vet ./... && go test ./...green inproduct-engineering-loop/release_notes.py validate+preflightPASS (new note:2026-07-30-visual-evidence-survives-preview-commit.md)