Skip to content

Fixes race condition that caused unneeded user compactions to run#4554

Merged
keith-turner merged 1 commit into
apache:2.1from
keith-turner:compaction_race
May 13, 2024
Merged

Fixes race condition that caused unneeded user compactions to run#4554
keith-turner merged 1 commit into
apache:2.1from
keith-turner:compaction_race

Conversation

@keith-turner

Copy link
Copy Markdown
Contributor

The following is an example of the problem this change fixes

  1. Thread 1: A user compaction is currently running for a tablet
  2. Thread 2: Tablet server receives a compaction request RPC from manager and it checks to see if the compaction is needed for the same tablet. If finds it is needed.
  3. Thread 1: completes user compaction, so a compaction is no longer needed for the tablet
  4. Thread 2: Initiates a user compaction of the tablet because its check in step 2 passed.

This change adds a second check in step 4 above that should prevent this race condition because the check is done at a point when its known no concurrent user compaction is running. The original check was left as a fail fast check, but a comment was added explaining its not sufficient for correctness.

The following is an example of the problem this change fixes

 1. Thread 1: A user compaction is currently running for a tablet
 2. Thread 2: Tablet server receives a compaction request RPC from manager and it
    checks to see if the compaction is needed for the same tablet.  If finds it is needed.
 3. Thread 1: completes user compaction, so a compaction is no longer
    needed for the tablet
 4. Thread 2: Initiates a user compaction of the tablet because its
    check in step 2 passed.

This change adds a second check in step 4 above that should prevent this race
condition because the check is done at a point when its known no
concurrent user compaction is running.  The original check was left as a
fail fast check, but a comment was added explaining its not sufficient
for correctness.

@dlmarion dlmarion left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I looked at the existing code and this seems ok.

@keith-turner keith-turner merged commit c88d368 into apache:2.1 May 13, 2024
@keith-turner keith-turner deleted the compaction_race branch May 13, 2024 21:04
@ctubbsii ctubbsii modified the milestones: 3.1.0, 2.1.3 Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants