Batch commit causes INVALID_ARGUMENT Row.. was already created in table bsos #882
Description
Batch commit can trigger an INSERT of the same row twice in the same transaction:
https://sentry.prod.mozaws.net/operations/syncstorage-prod/issues/9910134/
A database error occurred: RpcFailure: 3-INVALID_ARGUMENT Row [<fxa_uid>,<fxa_kid>,<collection_id>,<bso_id>] in table bsos was already created in this transaction.
The cause seems to be the older post_bsos optimization: when pending bsos are included in a batch commit request, we write them to the bsos table immediately via post_bsos, bypassing the batch table.
The problem being post_bsos writes in bulk mutations whereas batch commit uses plain sql. Mutations results are buffered and thus not visible in the transaction.
E.g. post_bsos issues an insert mutation for a bso_id that doesn't exist, then batch commit inserts its own version of the same bso_id (a duplicate). The buffered insert mutation is finally applied at the end of the request with the batch commit version of the bso having already been created, invalidating the insert mutation.
The given test case (which should be added to the e2e tests with a fix for this, as the unit tests don't use mutations), we would fail with this error:
- Create a batch with bso_id 'b0'
- Issue a batch commit request including a duplicate bso_id 'b0' in the same payload
A good solution here might just be to issue the post_bsos after the batch commit instead of where the batch append would have been issued.