test(examples): hold the published catalogue against a fresh render - #526
Merged
Conversation
A third of the catalogue rendered on every run with nothing comparing the result. Thirty-four documents — the cover-letter presets, most of the CV gallery, a few feature demos — were published to the showcase site fresh at each cut and never held against anything, so a change to the engine moved them and no test said so. They are committed as previews now, which is what puts them under CommittedAssetDriftTest: a PDF is compared by its bytes, and a difference names the file that moved. The unpublished list drops from thirty-five entries to one. The emoji gallery stays out on weight: its embedded glyph set renders to nearly 4 MB against 1.4 MB for the other thirty-four together, and committing it would put another copy of that in history on every deliberate re-render. Its render is unguarded as a result, which the note says plainly rather than leaving it to look like an oversight. The previews are rendered at the asset version, the one the drift guard compares at, and the examples README no longer calls the committed set a curated subset.
The runner disagreed with emoji-svg-vs-png.pdf and engine-deck.pptx. Both carry pixels this repository rasterises at render time, and the same glyphs come out with different antialiasing on Windows and on Linux — which AssetContent already records: of 104 documents, five failed to match across the two machines, and the ones its part-level exemptions could not absorb were simply never committed. These were two of them. They cannot use that exemption. A PDF is compared whole and has no part to name, and the deck's difference is in content it renders rather than in a part it embeds. Committing them fails the build on the runner for a document nobody touched, so they go back on the list — with the reason recorded per file, so the next reader does not repeat the experiment to learn it. Thirty-two documents remain newly guarded.
The entry and the examples README still said thirty-four documents and one exception, which was true of the commit before the two machine-rendered ones went back on the list.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
CommittedAssetDriftTestholds a committed preview against a fresh render and fails on thedifference. It is the only thing in the build that notices when the engine starts rendering a
document differently — the showcase site regenerates its own copies at every cut, so nothing
compares those.
A third of the catalogue was outside it. Thirty-five documents sat on the guard's
UNPUBLISHED_PREVIEWSlist — the cover-letter presets, most of the CV gallery, the emoji andclip demos, two decks — rendered on every run with nothing looking at the result. For those,
the whole test suite proved one thing: that they rendered without throwing.
That is not hypothetical. Twice in the last week the guard caught a real change to DOCX
output, and the second time it was two cells of text disappearing from a shipped example. Any
of the same in the thirty-five would have reached the site in silence.
What changed
compared by its bytes with only the
/IDdropped, so any movement in the content streamnames the file that moved.
default is published and the list is what has to earn an exception.
set and renders to 3.97 MB against 1.4 MB for the thirty-two together, so committing it
would put another copy of that in history on every re-render. The other two because their
pixels are rasterised at render time and the runner antialiases them differently; see below.
examples/README.mdno longer describes the committed set as a curated subset, and explainswhat committing buys.
Committed previews go from 2.24 MB to 3.6 MB.
Verification
./mvnw -B -ntp clean verify -pl :graph-compose-core,…,:graph-compose-coverage -amand./mvnw -B -ntp test -f examples/pom.xml→ BUILD SUCCESS on both, examples 73 tests, 0failures.
That the files merely exist proves nothing, so the guard was made to bite: flipping one byte
in the middle of
cover-letter-nordic-clean-v2.pdfturns the suite red naming that file, andrestoring it turns it green. (The first attempt at this probe was a no-op — a PDF's content
streams are compressed, so patching a literal operator found nothing to patch. The byte flip
is the version that proves anything.)
What the runner found
Two of the thirty-four disagreed on Linux:
emoji-svg-vs-png.pdfandengine-deck.pptx.Both are back on the list, with the reason recorded per file.
The reason is not flakiness. Both carry pixels this repository rasterises at render time, and
the same glyphs come out with different antialiasing on Windows and on the runner. That is
measured, not guessed, and it was already measured here:
AssetContent's own note recordsthat of 104 documents five failed to match across the two machines, and says the ones its
part-level exemptions could not absorb were never committed. These were two of those five —
the information was in the repository before this branch was pushed, and connecting it to the
list would have saved the red build.
They cannot use that part-level exemption: a PDF is compared whole and has no part to name,
and the deck's difference is in content it renders rather than in a part it embeds. Widening
the comparison to absorb them would cost every other document its exactness, so they stay
unguarded and say why.
Thirty-two documents are newly guarded, taking the catalogue from 67 held to 99 of 102.
Lane: test — examples module and committed assets. No production code.