Cosmos: flow transactional batch failures through the execution strategy so they can be retried#38597
Cosmos: flow transactional batch failures through the execution strategy so they can be retried#38597AndriySvyryd with Copilot wants to merge 5 commits into
Conversation
…angesAsync Co-authored-by: AndriySvyryd <6539701+AndriySvyryd@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR fixes Cosmos transactional batch failures bypassing the execution strategy retry pipeline by throwing on batch failure (instead of returning a “failed result”) and moving the execution-strategy scope to wrap the full SaveChanges batch sequence, allowing transient failures to reach ShouldRetryOn and be retried.
Changes:
- Replace the transactional batch “result” return pattern with a
CosmosTransactionalBatchExceptionthrown fromProcessBatchResponse. - Change
ICosmosClientWrapper.ExecuteTransactionalBatchAsyncto returnTaskand execute once (no per-batch execution strategy wrapping in the client). - Wrap the full set of transactional batches in a single execution strategy invocation inside
CosmosDatabaseWrapper, with retry skipping of already-committed operations, and add a functional test verifying the exception reaches the execution strategy.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| test/EFCore.Cosmos.FunctionalTests/CosmosTransactionalBatchTest.cs | Adds a functional test asserting transactional batch failures flow through the execution strategy and nothing is committed. |
| src/EFCore.Cosmos/Storage/Internal/ICosmosClientWrapper.cs | Updates the transactional batch API to return Task rather than a result object. |
| src/EFCore.Cosmos/Storage/Internal/CosmosTransactionalBatchException.cs | Introduces an internal exception type carrying errored entries and the underlying CosmosException. |
| src/EFCore.Cosmos/Storage/Internal/CosmosDatabaseWrapper.cs | Moves execution strategy retry scope to encompass the entire batch set and adds committed-operation skipping across retries. |
| src/EFCore.Cosmos/Storage/Internal/CosmosClientWrapper.cs | Executes transactional batches once and throws CosmosTransactionalBatchException on failure (instead of returning a failure result). |
…e transactional batch Co-authored-by: AndriySvyryd <6539701+AndriySvyryd@users.noreply.github.com>
…fected; add test Co-authored-by: AndriySvyryd <6539701+AndriySvyryd@users.noreply.github.com>
|
|
||
| var exception = new CosmosException(response.ErrorMessage, errorCode, 0, response.ActivityId, response.RequestCharge); | ||
| return new CosmosTransactionalBatchResult(errorEntries, exception); | ||
| throw new CosmosException( |
There was a problem hiding this comment.
@copilot Call WrapUpdateException here, supplying errorEntries like they were computed before. Make sure that the execution strategy can unwrap the exception
There was a problem hiding this comment.
ProcessBatchResponse now restores the original errorBatchEntries computation (entries whose per-item status code matches the batch-level error code) and throws a DbUpdateConcurrencyException (PreconditionFailed) or DbUpdateException (Conflict / other) wrapping the CosmosException, using the entry Id already stored on each CosmosTransactionalBatchEntry.
The execution strategy can unwrap DbUpdateException via the existing ExecutionStrategy.CallOnWrappedException before calling ShouldRetryOn, so transient failures (e.g. TooManyRequests) are still correctly identified and retried.
ExecuteBatchesAsync is updated to catch (DbUpdateConcurrencyException) for the suppression path; other DbUpdateException variants pass through to the execution strategy unwrapped.
…currencyException for suppression Co-authored-by: AndriySvyryd <6539701+AndriySvyryd@users.noreply.github.com>
Cosmos transactional batch failures returned a result object from
ProcessBatchResponserather than throwing, so the failure surfaced as aDbUpdateExceptionoutside any execution-strategy scope and could never be retried by anExecutionStrategy(ShouldRetryOnwas never called).Changes
CosmosClientWrapper.ProcessBatchResponsenow computes the errored entries (those whose per-item status code matches the batch-level error code), wraps theCosmosExceptioninto aDbUpdateConcurrencyException(PreconditionFailed) orDbUpdateException(Conflict / other), and throws it directly.CosmosTransactionalBatchResultandCosmosTransactionalBatchExceptionare removed.ExecuteTransactionalBatchAsyncno longer wraps itself in the execution strategy; it runs once and returnsTask(interface updated).SaveChangesAsync—CosmosDatabaseWrappertakes an injectedIExecutionStrategyand wraps the whole set of batches in a singleExecuteAsync. ABatchExecutionState.CommittedOperationscounter lets a retry skip already-committed operations while preserving the original ordering of single-entry saves interleaved with transactional batches. The execution strategy unwrapsDbUpdateExceptionviaExecutionStrategy.CallOnWrappedExceptionto reach the innerCosmosExceptionbefore callingShouldRetryOn, so transient failures are correctly identified and retried.DbUpdateConcurrencyExceptionfrom a batch is suppressed viaOptimisticConcurrencyExceptionAsync,RowsAffectedis not incremented (matchingSaveAsyncsingle-entry behavior), whileCommittedOperationsstill advances so a subsequent retry skips the already-processed operation.Because a transactional batch is atomic, a failed batch commits nothing and is retried in place; deterministic transaction serialization across attempts makes index-based skipping of committed operations valid.
Result
The repro from the issue now behaves as expected — a transient batch failure reaches the strategy and is retried:
Two functional tests verify the behavior:
SaveChanges_transactional_batch_failure_flows_through_execution_strategy— a conflicting batch reachesShouldRetryOnand yieldsRetryLimitExceededException, with nothing committed.SaveChanges_suppressed_concurrency_exception_in_transactional_batch_reports_zero_rows_affected— a stale-ETag batch update suppressed by an interceptor returns 0 rows affected and leaves the store unchanged.