Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kvcoord,kvclient,gossip: gossip range lease acquisition #52572

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

knz
Copy link
Contributor

@knz knz commented Aug 10, 2020

Fixes #50199.

Before this patch, it was possible for a range cache to contain an
outdated lease if the node with the lease was restarted or went
AWOL.

This patch introduces a mechanism by which the new owner of a range
lease announces this ownership to other nodes via gossip.
Any cached lease for that range gets updated from the gossip update
if the gossiped lease is more recent than the one known.

Release note (general change): CockroachDB nodes now learn more
actively of range leadership transfers from other nodes. This makes
query performance generally more resilient to routine node restarts,
as fewer queries now get routed to an outdated or unavailable node.

@knz knz requested review from andreimatei and tbg August 10, 2020 15:56
@knz knz requested a review from a team as a code owner August 10, 2020 15:56
@knz knz added this to In progress in DB Server & Security via automation Aug 10, 2020
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@knz
Copy link
Contributor Author

knz commented Aug 10, 2020

This still needs some testing, but I wanted to air the prototype out first to see what CI thinks of it.


// Gossip that we have the new lease, to
// update any cached entry on other nodes.
// FIXME(andrei): do we want to move this call under the condition
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andreimatei @nvanbenschoten I'll take suggestions about how much sense it makes to gossip leases that are not expiration-based.

Also, is there a utility to call IsLeaseValid here? I'd say yes, because of the (unlikely) scenario where the lease was just transferred already to another node by the time we reach this point here, and we don't want this code here to race with the new leaseholder on the gossip update. Thoughts?

Before this patch, it was possible for a range cache to contain an
outdated lease if the node with the lease was restarted or went
AWOL.

This patch introduces a mechanism by which the new owner of a range
lease announces this ownership to other nodes via gossip.
Any cached lease for that range gets updated from the gossip update
if the gossiped lease is more recent than the one known.

Release note (general change): CockroachDB nodes now learn more
actively of range leadership transfers from other nodes. This makes
query performance generally more resilient to routine node restarts,
as fewer queries now get routed to an outdated or unavailable node.
@tbg tbg added the X-noremind Bots won't notify about PRs with X-noremind label May 6, 2021
@knz knz marked this pull request as draft May 6, 2021 10:47
@knz knz removed the X-noremind Bots won't notify about PRs with X-noremind label May 6, 2021
@tbg tbg removed their request for review June 21, 2021 11:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
DB Server & Security
  
In progress
Development

Successfully merging this pull request may close these issues.

kv: gossip leaseholder changes as they occur
3 participants