New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: avoid merge deadlock when concurrent split wins #31294

Merged
merged 1 commit into from Oct 15, 2018

Conversation

Projects
None yet
4 participants
@benesch
Member

benesch commented Oct 12, 2018

@bdarnell, in a stroke of luck, the existing conditional put API supports deleting the existing value when you pass nil.


When merging two adjacent ranges P and Q, we need to be careful to
delete Q's range descriptor with a conditional put, or we can deadlock
with a concurrent split. See the comments and test case within for
details. This commit fixes the last issue that is blocking merges from
being turned on by default.

Release note: None

@benesch benesch requested review from bdarnell and tschottdorf Oct 12, 2018

@benesch benesch requested a review from cockroachdb/core-prs as a code owner Oct 12, 2018

@cockroach-teamcity

This comment has been minimized.

Show comment
Hide comment
@cockroach-teamcity

cockroach-teamcity Oct 12, 2018

Member

This change is Reviewable

Member

cockroach-teamcity commented Oct 12, 2018

This change is Reviewable

@bdarnell

:lgtm_strong:

That's a surprisingly simple test for such a complex issue.

Reviewed 3 of 3 files at r1.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained


pkg/storage/client_merge_test.go, line 1133 at r1 (raw file):

			if bt := req.GetBeginTransaction(); bt != nil &&
				bt.Key.Equal(keys.RangeDescriptorKey(roachpb.RKeyMin)) &&
				atomic.AddInt64(&launchSplit, -1) == 0 {

Nit: Why use a negative value here?

@tschottdorf

:lgtm: good job.

Reviewed 3 of 3 files at r1.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (and 1 stale)


pkg/storage/client_merge_test.go, line 1133 at r1 (raw file):

Previously, bdarnell (Ben Darnell) wrote…

Nit: Why use a negative value here?

I'm confused by the comment, Ben. He could use a CAS, is that what you mean? Or subtract 1 below and add +1 here?

@bdarnell

Reviewable status: :shipit: complete! 2 of 0 LGTMs obtained


pkg/storage/client_merge_test.go, line 1133 at r1 (raw file):

Previously, tschottdorf (Tobias Schottdorf) wrote…

I'm confused by the comment, Ben. He could use a CAS, is that what you mean? Or subtract 1 below and add +1 here?

I missed the Store below and was just confused. Nevermind (although I think a CAS might capture the intent a little more cleanly).

storage: avoid merge deadlock when concurrent split wins
When merging two adjacent ranges P and Q, we need to be careful to
delete Q's range descriptor with a conditional put, or we can deadlock
with a concurrent split. See the comments and test case within for
details. This commit fixes the last issue that is blocking merges from
being turned on by default.

Release note: None
@benesch

This comment has been minimized.

Show comment
Hide comment
@benesch

benesch Oct 15, 2018

Member

Ack, switched to a CAS. Thanks for the reviews!

bors r=tschottdorf,bdarnell

Member

benesch commented Oct 15, 2018

Ack, switched to a CAS. Thanks for the reviews!

bors r=tschottdorf,bdarnell

craig bot pushed a commit that referenced this pull request Oct 15, 2018

Merge #31294
31294: storage: avoid merge deadlock when concurrent split wins r=tschottdorf,bdarnell a=benesch

@bdarnell, in a stroke of luck, the existing conditional put API supports deleting the existing value when you pass nil.

---

When merging two adjacent ranges P and Q, we need to be careful to
delete Q's range descriptor with a conditional put, or we can deadlock
with a concurrent split. See the comments and test case within for
details. This commit fixes the last issue that is blocking merges from
being turned on by default.

Release note: None

Co-authored-by: Nikhil Benesch <nikhil.benesch@gmail.com>
@craig

This comment has been minimized.

Show comment
Hide comment
@craig

craig bot commented Oct 15, 2018

Build succeeded

@craig craig bot merged commit 3471a00 into cockroachdb:master Oct 15, 2018

3 checks passed

GitHub CI (Cockroach) TeamCity build finished
Details
bors Build succeeded
Details
license/cla Contributor License Agreement is signed.
Details

@benesch benesch deleted the benesch:merge-cond-delete branch Oct 15, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment