refactor: implement cooperative state machine for range/list operations #18204
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/
Summary
refactor: implement cooperative state machine for range/list operations
Fix blocking issue during initialization data transmission.
Problem:
When establishing a watch stream, the meta-service sends large amounts of
initialization data to the client. During this transmission, other events
are blocked until completion, including add-watcher commands.
This creates a deadlock: if initialization data is large, it blocks all
subsequent Dispatcher operations. When a second watch request arrives,
it must wait for the first one to complete sending all initialization data.
Since adding a new watcher requires holding the state machine lock,
multiple concurrent watch requests will block the state machine entirely,
causing timeouts for other requests.
Solution:
Make the process cooperative by not waiting for watch stream transmission
to complete. Instead, queue the add-watcher command and return immediately.
This allows subsequent watch requests to proceed without waiting for previous
initialization data transmissions to finish.
Tests
Type of change
Related Issues
This change is