Return HTTP 409 (Conflict) for Sequential Bundle Race Condition

[mikaelweave]

Description

This PR changes SQL Server transaction-bundle behavior for the confirmed WI 188321 root cause. When MergeAsync sees SQL conflict 50409 while enlisted in a C# transaction, it now stops retrying inside that already-invalidated transaction and throws ResourceConflictException, which maps to HTTP 409 Conflict. This prevents the request from continuing to SqlTransaction.Commit() and surfacing the later zombie transaction symptom as a generic HTTP 500.

The completed SqlTransaction fallback in BundleHandler remains a 500 for non-conflict root causes. The change is intentionally scoped to enlisted SQL concurrency conflicts; the existing non-enlisted retry path still retries and preserves its current exhausted-retry behavior.

This PR also adds targeted coverage for:

• Enlisted SQL conflict fail-fast behavior at the SQL data store layer.
• Transaction bundle conflict wiring to HTTP 409.
• The existing completed/zombie transaction fallback behavior.

The initial PowerShell repro harness was removed from the branch.

AB#188321

Testing

• dotnet test .\src\Microsoft.Health.Fhir.Api.UnitTests\Microsoft.Health.Fhir.R4.Api.UnitTests.csproj --no-restore --filter "FullyQualifiedName~BundleHandlerTests" --logger "console;verbosity=minimal"
• dotnet test .\src\Microsoft.Health.Fhir.Api.UnitTests\Microsoft.Health.Fhir.R4.Api.UnitTests.csproj --no-restore --filter "FullyQualifiedName~GivenATransaction_WhenTransactionIsZombiedAtCommit|FullyQualifiedName~GivenATransaction_WhenInnerRequestReturnsConflictOperationOutcome" --logger "console;verbosity=minimal"
• dotnet test .\src\Microsoft.Health.Fhir.SqlServer.UnitTests\Microsoft.Health.Fhir.SqlServer.UnitTests.csproj --no-restore --framework net8.0 --filter "Name~MergeAsync_OnSqlConflict_WithEnlistedTransaction" --logger "console;verbosity=minimal"
• dotnet test .\src\Microsoft.Health.Fhir.SqlServer.UnitTests\Microsoft.Health.Fhir.SqlServer.UnitTests.csproj --no-restore --framework net9.0 --filter "Name~MergeAsync_OnSqlConflict_WithEnlistedTransaction" --logger "console;verbosity=minimal"

FHIR Team Checklist

• Update the title of the PR to be succinct and less than 65 characters
• Add a milestone to the PR for the sprint that it is merged (i.e. add S47)
• Tag the PR with the type of update: Bug, Build, Dependencies, Enhancement, New-Feature or Documentation
• Tag the PR with Open source, Azure API for FHIR (CosmosDB or common code) or Azure Healthcare APIs (SQL or common code) to specify where this change is intended to be released.
• Tag the PR with Schema Version backward compatible or Schema Version backward incompatible or Schema Version unchanged if this adds or updates Sql script which is/is not backward compatible with the code.
• When changing or adding behavior, if your code modifies the system design or changes design assumptions, please create and include an ADR.
• CI is green before merge
• Review squash-merge requirements

Semver Change (docs)

Patch

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.34%. Comparing base (ee554e9) to head (bb50e22).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5592   +/-   ##
=======================================
  Coverage   77.33%   77.34%           
=======================================
  Files         996      996           
  Lines       36534    36550   +16     
  Branches     5538     5541    +3     
=======================================
+ Hits        28255    28268   +13     
+ Misses       6912     6910    -2     
- Partials     1367     1372    +5     

see 11 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.