KEP-2835: Add Prioritized Leader Election KEP

[howardjohn]

• One-line PR description: Add initial data for KEP-2835

• Issue link: Prioritized leader election #2835

[k8s-ci-robot]

Hi @howardjohn. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: howardjohn
To complete the pull request process, please assign fedebongio after the PR has been reviewed.
You can assign the PR to them by writing /assign @fedebongio in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• keps/sig-api-machinery/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[howardjohn]

/assign @jiahuif

[sftim]

/ok-to-test

[k8s-ci-robot]

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Rerun command
pull-enhancements-verify	6a1e25b	link	/test pull-enhancements-verify

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[deads2k]

in the sig call, I think we decided that this should probably be a goal and it would benefit cases of upgrades with a weight for latest version.

[deads2k]

In the API, I think we want to indicate what the weighting function is and an integer weight. That allows for different versions of a controller-manager to change its weighting function and for other leaders to be able to make an intelligent decision about what to do when they don't understand that weighting function.

For instance, when the lease is expired, old clients that do not understand the weighting function could still take the lease and simply indicate a weight of 0.

[howardjohn]

Ah, I didn't understand this in the call but I get it now. I am not sure an integer will support all cases.

Some examples

---

Newest version wins (string):
example key: v1.8.1
example key comparison function: return semver(myVersion).NewerThan(semver(existingKey)

Newest version wins (int):
example key: 8 (attempt to look at only the minor version
example key comparison function: return myVersion > existingKey

The issue with int here is we are trying to encode a SemVer into a single integer. There are ways around this, but not a good one. For example, if we later need to distinguish beyond minor version we cannot.

---

Selected controller wins (our primary use case

string:
example key: my-controller-identifier
example key comparison function: return existingKey != selectedIdentifier && myIdentifier == selectedIdentifier (where selectedIdentifier is determined out-of-band)

int: not really possible

---

Select geographically closest controller

string:
example key: us-west
example key comparison function: return myKey is closer to TargetRegion than existingKey

int: not really possible unless we encode each region as an integer

---

[deads2k]

Since lease stealing will no longer be rare, I think avoiding a condition of having two leaders should be a goal. I think it's just a matter of waiting for a period longer than the LeaderElectionConfig.RetryPeriod, which would mean exposing that duration somewhere.

[p0lyn0mial]

What happens in a scenario where the old client understands the key, its key is lower in priority but there is no current leader? Should the old client be able to acquire the lock? Maybe having a leader with lower priority is better than no leader?

[howardjohn]

If there is no current leader all clients will attempt to become the leader. Consider a case with two clients, priority=low and priority=high.

Case 1: p=low wins the election, and immediately is evicted by p=high when it takes over
Case 2: p=high wins the election

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.