ci: pre-populate synthtool cache to avoid race condition#17230
Merged
Conversation
Temporarily add pull_request trigger to verify the fix. TAG=agy CONV=acb0ee9b-3d87-4b9e-b3c6-0d4aa3efbd35
Contributor
|
Note Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported. |
Member
Author
Member
Author
|
It failed. |
… conflict This avoids parallel git checkout/pull race conditions by cloning the templates once and pointing SYNTHTOOL_TEMPLATES to it. TAG=agy CONV=acb0ee9b-3d87-4b9e-b3c6-0d4aa3efbd35
The templates now default to Python 3.10, so this workaround is no longer needed and causes AssertionError. TAG=agy CONV=acb0ee9b-3d87-4b9e-b3c6-0d4aa3efbd35
We have verified the fix works. Reverting the temporary trigger before merge. TAG=agy CONV=acb0ee9b-3d87-4b9e-b3c6-0d4aa3efbd35
Member
Author
|
Regenerate succeeded in https://github.com/googleapis/google-cloud-python/actions/runs/26264596009/job/77305110304. |
ohmayr
approved these changes
May 22, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR resolves a blocking parallel generation issue on GitHub Actions and updates Spanner's post-processing config to align with recent template updates.
1. Fix GHA parallel generation race condition
When running
librarian generate --allin GHA, multiple libraries are generated in parallel. They concurrently invokesynthtool, which races to clone/update the templates repository (googleapis/synthtool) in the shared cache directory~/.cache/synthtool/synthtool.This results in either:
fatal: failed to resolve HEAD as a valid refwhen runninggit branch.git checkout/git pullconcurrently, failing withfatal: Unable to create ... index.lock: File exists.Fix:
We now clone
googleapis/synthtoolonce sequentially at the beginning of the workflow (Clone Synthtool Templates), and configureSYNTHTOOL_TEMPLATESenvironment variable globally to point to this local directory. This completely bypasses all concurrent git clone/pull operations during parallel generation, resolving the race conditions and speeding up the build.2. Fix Spanner post-processing AssertionError
We removed the obsolete
Supported Python Versionsreplacement block from.librarian/generator-input/client-post-processing/spanner-integration.yaml.Why it was needed:
2408166fec), Python 3.7-3.9 support was dropped specifically for Spanner. Because thesynthtooltemplates still generatedPython >= 3.9inREADME.rstby default, a post-processing workaround was added to force it to3.10.e643ce8ingoogleapis/synthtool),synthtooltemplates were updated globally to drop Python 3.9 support for all mono-repo libraries.README.rstfor Spanner now already defaults toPython >= 3.10. The May 19 workaround became obsolete and failed withAssertionError: Replaced 0 rather than 1 instancesbecause the "before" pattern (expecting3.9) no longer exists.TAG=agy
CONV=acb0ee9b-3d87-4b9e-b3c6-0d4aa3efbd35