Hand the screenshot run its locales in something gradle keeps - #652
Merged
Conversation
AGP 9.4.0 cuts a `-Pandroid.testInstrumentationRunnerArguments.<key>` value at
the first comma. The lane joined the fifteen locales with one, so the runner was
handed `-e locales cs-CZ` and each device photographed a single language:
9.3.2 args_map { key: "locales" value: "de-DE,en-US" }
9.4.0 am instrument ... -e locales de-DE
It is AGP's bug rather than ours - the same truncation drops every class but the
first from the documented `class=TestA,TestB` - so the lane joins with a plus,
which arrives whole, and the test splits on either. A run driving `am instrument`
by hand is unaffected and can still spell it with a comma.
What made this cost a release is that it passed. `ScreenshotTests` photographs
the locales it is handed, so one language is a full run of a short list, and
`screenshots_narrowed?` skips the full-set check on a release runner holding half
a set. The lane now checks that every language it asked for came out, which holds
for half a set as well.
Found on the 4.19.0 release: both devices reported success with cs-CZ alone, and
`screenshot-set` refused the other fourteen. The store kept 4.18.0's pictures,
which is what that job is split out to make survivable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017zSYiEKUuTFhMy3jpee99C
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017zSYiEKUuTFhMy3jpee99C
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017zSYiEKUuTFhMy3jpee99C
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The 4.19.0 release photographed one locale of fifteen, on both devices, and both
screenshotsjobs reported success.screenshot-setcaught it and the store kept 4.18.0's pictures.What happens
AGP 9.4.0 cuts a
-Pandroid.testInstrumentationRunnerArguments.<key>value at the first comma. Measured on this tree against one emulator, same command, same everything but the plugin:locales=de-DE,en-USunder 9.3.2args_map { key: "locales" value: "de-DE,en-US" }locales=de-DE,en-USunder 9.4.0-e locales de-DElocales=de-DE+en-USunder 9.4.0-e locales de-DE+en-USclass=A#doesNotExist,Bunder 9.4.0-e class A#doesNotExistsizes=a b,cunder 9.4.0-e sizes a bA space survives and a comma does not, and the last row is the documented multi-class form, so this is AGP's rather than ours. Nothing in the 9.4.0 release notes mentions it.
What changes
The lane joins with a plus and the test splits on either, so
am instrumentdriven by hand still takes a comma.The lane checks that every language it asked for came out. That is the part that let this reach the store:
ScreenshotTestsphotographs the locales it is handed and passes, so a list mangled on the way in is indistinguishable from a run asked for one language - andscreenshots_narrowed?skips the full-set check on a release runner, which is holding half a set by design. The new check is per language, so it holds for that half too.Checked
Against the API 36 AVD with the fifteen-locale list:
-e locales cs-CZ+de-DE+en-US+es-ES+et+fr-FR+hi-IN+it-IT+ja-JP+pl-PL+pt-BR+ru-RU+sv-SE+tr-TR+zh-CNarrives whole. A full local capture is not part of this - the next release run is what exercises it end to end.4.19.0's own pictures are not restored by this:
build/v4.19.0is written, so the release run refuses that version. The store shows 4.18.0's until the next release.🤖 Generated with Claude Code
https://claude.ai/code/session_017zSYiEKUuTFhMy3jpee99C