mergeMigration

[xinlian12]

Issue

This change is part of the feature work #32131. In order to support merge (which requires the epk version leases) and align with changeFeedProcessor long term strategy, we introduced a new API handleLatestVersionChanges.

When customer switch from handleChanges to handleLatestChanges, due to the internal lease store switch, if there is no any support, then the changeFeed processing will start from beginning, which could cause high amount duplicate processing, which is not ideal.

Solution

In order to minimize the impact when customer switch API, we added automatic migration support in this PR.

• If a pkRangeId version lease store is detected, then SDK is going to bootstrap based on the existed pkRangeId version lease store. At the end of the bootstrap, it will also delete all previous pkRangeId version leases, which will be used as a signal to coordinate with changeFeedProcessorInstance still use handleChanges API.
• If a pkRangeId version lease store does not exists, then SDK is going to bootstrap from scratch, but it will also initialize an empty pkRangeId version lease store at the same time to prevent it is being initialized again in the future.

##Previous Main Flow

  graph TD
  A[Initialize] --> B{Is lease store initialized?}
  B -- Yes --> C[Finish]
  B -- No -->  D[Acquire Lock]
  D --> E[Create epk version leases from pkRanges]
  E --> F[Mark epk version lease store initialized]
  F --> G[Release epk version lease store lock]
  G --> C

Loading

##Current Main Flow

 graph TD
A[Initialize] --> B{Is epk version lease store initialized?}
B -- Yes --> C[Finish]
B -- No -->  D[Acquire Lock]
D --> E{Is PkRangeId Version lease store initialized?}
E -- YES --> F[Get all pkRangeId version leases]
F --> G[Create epk version leases from pkRangeId version leases]
G --> H[Mark epk version lease store initialized]
H --> I[Delete all pkRangeId version leases]
I --> J[Release epk version lease store lock]
J --> C
E -- NO --> K[Acquire pkRangeId version lease store lock]
K --> L[Create epk version leases from pkRanges]
L --> M[Mark epk version lease store initialized]
M --> N[Mark pkRangeId version lease store initialized]
N --> O[Release epk version lease store lock]
O --> P[Release pkRangeId version lease store lock]
P --> C

Loading

[azure-sdk]

API change check

API changes are not detected in this pull request.

[xinlian12]

/azp run java - cosmos - tests

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[FabianMeiswinkel]

Thanks - looks great!

[xinlian12]

/azp run java - cosmos - tests

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[xinlian12]

Tested the following failed tests locally
readMyWrites
before_OrderbyDocumentQueryTest

[xinlian12]

/check-enforcer override