bugfix: Reduce the batch writer's batch size.#1512
Merged
Conversation
The web features job is currently failing Currently, I see this error: ``` ERROR invalid status while batch writing status="rpc error: code = InvalidArgument desc = The transaction contains too many mutations. Insert and update operations count with the multiplicity of the number of columns they affect. For example, inserting values into one key column and four non-key columns count as five mutations total for the insert. Delete and delete range operations count as one mutation regardless of the number of columns affected. The total mutation count includes any changes to indexes that the transaction generates. Please reduce the number of writes, or use fewer indexes. (Maximum number: 80000)" ``` You'll see that a transaction can actually count multiple times if there things like an index. So I just reduced the batchSize down to something that works. (Tested in on [staging](https://pantheon.corp.google.com/run/jobs/details/europe-west1/staging-europe-west1-web-features/executions?inv=1&invt=Abx65w&project=webstatus-dev-internal-staging)) Other changes: - My IDE highlighted an optimization on the looping for the concurrent batch size writer. So I just used the auto-fix for that.
jrobbins
approved these changes
May 20, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The web features job is currently failing
Currently, I see this error:
You'll see that a transaction can actually count multiple times if there things like an index. So I just reduced the batchSize down to something that works. (Tested in on staging)
Other changes: