Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CXXCBC-377: Implement ExtParallelUnstaging in transactions #457

Merged
merged 6 commits into from Oct 24, 2023

Conversation

DemetrisChr
Copy link
Contributor

@DemetrisChr DemetrisChr commented Oct 4, 2023

Motivation

Implement the ExtParallelUnstaging extension. Unstaging in parallel will be a major performance improvement, especially for larger transactions.

Changes

  • Each mutation unstaging is posted to the cluster's IO context to achieve parallelism - modified commit_doc, remove_doc, rollback_insert and rollback_remove_or_replace to achieve this, making use of the KV operations' handlers
  • Added helpers for error handling which either set the exception on the promise or trigger a delayed retry
  • Add unstaging_state that is used to limit the number of mutations being unstaged in parallel and prevents any further unstaging if an error has occurred
  • Add async_exp_delay and async_constant_delay which use an asio timer to perform an asynchronous wait and then invoke a callback (delay calculated in the same way as exp_delay and constant_delay) - used for retries during unstaging

Results

  • SDK tests pass
  • ExtParallelUnstagingTest in FIT passes

core/transactions/staged_mutation.cxx Outdated Show resolved Hide resolved
core/transactions/staged_mutation.cxx Outdated Show resolved Hide resolved
core/transactions/staged_mutation.cxx Outdated Show resolved Hide resolved
core/transactions/staged_mutation.cxx Show resolved Hide resolved
@DemetrisChr DemetrisChr force-pushed the CXXCBC-377-parallel-unstaging branch 2 times, most recently from ab6670a to 57e05f0 Compare October 13, 2023 14:04
@DemetrisChr DemetrisChr changed the title CXXCBC-377: Support ExtParallelUnstaging CXXCBC-377: Implement ExtParallelUnstaging in transactions Oct 13, 2023
@DemetrisChr DemetrisChr force-pushed the CXXCBC-377-parallel-unstaging branch 2 times, most recently from ab6670a to de48409 Compare October 17, 2023 11:56
core/transactions/staged_mutation.cxx Show resolved Hide resolved
core/transactions/staged_mutation.cxx Outdated Show resolved Hide resolved
core/transactions/staged_mutation.cxx Show resolved Hide resolved
core/transactions/staged_mutation.cxx Outdated Show resolved Hide resolved
core/transactions/staged_mutation.cxx Outdated Show resolved Hide resolved
@DemetrisChr DemetrisChr force-pushed the CXXCBC-377-parallel-unstaging branch 2 times, most recently from dbc7cf4 to bdbe1a3 Compare October 18, 2023 22:31
@DemetrisChr DemetrisChr marked this pull request as ready for review October 19, 2023 07:59
@avsej avsej force-pushed the CXXCBC-377-parallel-unstaging branch from 0ee2b9c to aee1504 Compare October 24, 2023 20:48
@avsej avsej merged commit 1658962 into couchbaselabs:main Oct 24, 2023
13 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants