Promote taint based evictions to GA #1450

damemi · 2020-01-14T17:06:09Z

Based on this comment in the TBE issue (#166 (comment)) we are creating an enhancement to officially track the promotion of Taint Based Evictions to GA

damemi · 2020-01-14T17:06:21Z

/cc @ahg-g @Huang-Wei

Huang-Wei · 2020-01-14T18:15:41Z

keps/sig-scheduling/20200114-taint-based-evictions.md

+## Summary
+
+Taint Based Evictions was introduced as an alpha feature in Kubernetes 1.6 and was promoted to
+beta in 1.13. Taint Based Evictions evicts pods from a node based on taints applied to the node


It's the tricky part...

Long time ago, when the feature was proposed, the feature was supposed to evict pods based on node conditions. But along with the development, it's finalized with a much smaller scope:

Taint nodes with a NoExecute effect automatically, when the nodes gets not ready or unreachable. While TaintNodeByCondition feature apply taints with a NoSchedule effect. Sometime people confuse with them.

Other functionalities such as evicting pods and setting tolerationSeconds are out of TaintBasedEviction's scope (by taint manager and default toleration seconds admission controller).

So the name is very confusing, it's worth highlighting the tricky parts.

If we disable TaintBasedEviction, the eviction behavior and setting default toleration seconds still work. Which also prove that those are not the scope of TaintBasedEviction.

And maybe we should emphasize the word "automatically", as NoExecute comes with taint long time ago, but system-managed NoExecute taints are exactly managed by this feature.

Huang-Wei · 2020-01-14T18:20:42Z

keps/sig-scheduling/20200114-taint-based-evictions.md

+
+### Goals
+
+Promote Taint Based Evictions to GA.


We should say "Ensure nodes are tainted properly with a NoExecute effect when it's not ready or unreachable, so that scheduler can use taints to make scheduling decisions consistently."

Huang-Wei · 2020-01-14T18:24:46Z

keps/sig-scheduling/20200114-taint-based-evictions.md

+
+## Motivation
+
+Taint Based Evictions has been in beta since 1.13 and has functioned well since then.


The Motivation section applies generally to the feature, not particularly for promoting it to GA.

How about: TaintNodesByCondition has ensured the nodes to be tainted well with NoSchedule effect, upon different node conditions. However, it's also required to taint nodes with NoExecute effect automatically upon some node conditions such as node gets not ready or unreachable.

Huang-Wei · 2020-01-14T18:27:24Z

keps/sig-scheduling/20200114-taint-based-evictions.md

+
+### Non-Goals
+
+Make any change of functionality to the feature.


Ditto. Non-goals applies to the feature.

damemi · 2020-01-14T18:49:51Z

@Huang-Wei updated with your feedback

Huang-Wei · 2020-01-14T19:29:03Z

/lgtm

@k82cn @ahg-g do you want to take another look?

ahg-g · 2020-01-15T14:00:04Z

keps/sig-scheduling/20200114-taint-based-evictions.md

+title: Promote Taint Based Evictions to GA
+authors:
+  - "@damemi"
+owning-sig: sig-scheduling


sig-scheduling is not the owning sig of this feature, the scheduler doesn't include any code specifically for this feature.

@liggitt which sig does the node lifecycle controller falls under?

would that be sig-node?

probably sig-node

or sig-cloud-provider for the "not reachable nodes" aspect

ahg-g · 2020-01-15T14:19:43Z

keps/sig-scheduling/20200114-taint-based-evictions.md

+* [Node lifecycle controller eviction tests](https://github.com/kubernetes/kubernetes/blob/47d5c3ef8d/pkg/controller/nodelifecycle/node_lifecycle_controller_test.go#L196)
+
+### Integration tests
+* [Taint based evictions integration test](https://github.com/kubernetes/kubernetes/blob/47d5c3ef8df2b1b26da739aec0ada15d41f20cf3/test/integration/scheduler/taint_test.go#L580) (note that prior to 1.17, this test existed as an [end-to-end test](https://github.com/kubernetes/kubernetes/blob/001f2cd2b553d06028c8542c8817820ee05d657f/test/e2e/scheduling/taint_based_evictions.go)


before graduating to GA, we should make sure that the tests are moved under the proper sig.

liggitt · 2020-01-15T17:10:33Z

keps/sig-scheduling/20200114-taint-based-evictions.md

+reviewers:
+  - "@Huang-Wei"
+approvers:
+  - "@liggitt"


I would recommend sig-node/sig-cloud-provider for approvers

damemi · 2020-01-16T18:10:24Z

@liggitt @ahg-g I've updated to list sig-node as the owning sig and both node and cloud-provider as approvers. I also made a note under the graduation criteria that tests should be moved to the appropriate sig

ahg-g · 2020-01-17T00:52:13Z

/lgtm

ahg-g · 2020-01-17T00:52:28Z

/hold

damemi · 2020-01-20T14:49:22Z

Just checking, is there anything else this needs? Waiting on this to merge to start breaking out the issues in kubernetes/kubernetes#87161

ahg-g · 2020-01-20T16:19:39Z

/lgtm

k8s-ci-robot · 2020-01-20T16:20:17Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahg-g, damemi, Huang-Wei

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~keps/sig-scheduling/OWNERS~~ [ahg-g]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

damemi · 2020-01-20T18:26:23Z

/hold cancel

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jan 14, 2020

k8s-ci-robot requested review from ahg-g and Huang-Wei January 14, 2020 17:06

k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jan 14, 2020

k8s-ci-robot requested a review from k82cn January 14, 2020 17:06

k8s-ci-robot added kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. labels Jan 14, 2020

damemi force-pushed the taint-based-evictions branch from 2cef2aa to b12fab8 Compare January 14, 2020 17:43

Huang-Wei reviewed Jan 14, 2020

View reviewed changes

damemi force-pushed the taint-based-evictions branch from b12fab8 to 68bcf4b Compare January 14, 2020 18:49

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 14, 2020

k8s-ci-robot assigned Huang-Wei Jan 14, 2020

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 14, 2020

ahg-g reviewed Jan 15, 2020

View reviewed changes

liggitt reviewed Jan 15, 2020

View reviewed changes

Promote taint based evictions to GA

fa0f1ab

damemi force-pushed the taint-based-evictions branch from 68bcf4b to fa0f1ab Compare January 16, 2020 18:09

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 16, 2020

k8s-ci-robot assigned ahg-g Jan 17, 2020

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 17, 2020

k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jan 17, 2020

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 20, 2020

k8s-ci-robot merged commit 490d72c into kubernetes:master Jan 20, 2020

k8s-ci-robot added this to the v1.18 milestone Jan 20, 2020

This was referenced Jan 21, 2020

[TaintBasedEvictions] Ensure test coverage is complete and up-to-date kubernetes/kubernetes#87427

Closed

Promote TaintBasedEvictions to GA in 1.18 kubernetes/kubernetes#87161

Closed

ahg-g mentioned this pull request Jan 24, 2020

Move TaintBasedEvictions feature gates to GA kubernetes/kubernetes#87487

Merged

Ye-Tian-Zero mentioned this pull request Jan 27, 2020

Taint Based Eviction #166

Closed

chenchun mentioned this pull request Jan 25, 2024

Clean up useless daemonSetStore field of node life controller struct kubernetes/kubernetes#122864

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Promote taint based evictions to GA #1450

Promote taint based evictions to GA #1450

damemi commented Jan 14, 2020

damemi commented Jan 14, 2020

Huang-Wei Jan 14, 2020 •

edited

Huang-Wei Jan 14, 2020 •

edited

Huang-Wei Jan 14, 2020

Huang-Wei Jan 14, 2020

Huang-Wei Jan 14, 2020

damemi commented Jan 14, 2020

Huang-Wei commented Jan 14, 2020

ahg-g Jan 15, 2020

ahg-g Jan 15, 2020

damemi Jan 15, 2020

liggitt Jan 15, 2020

liggitt Jan 15, 2020

ahg-g Jan 15, 2020

liggitt Jan 15, 2020

damemi commented Jan 16, 2020

ahg-g commented Jan 17, 2020

ahg-g commented Jan 17, 2020

damemi commented Jan 20, 2020

ahg-g commented Jan 20, 2020

k8s-ci-robot commented Jan 20, 2020

damemi commented Jan 20, 2020


		## Motivation

		Taint Based Evictions has been in beta since 1.13 and has functioned well since then.


		### Non-Goals

		Make any change of functionality to the feature.

Promote taint based evictions to GA #1450

Promote taint based evictions to GA #1450

Conversation

damemi commented Jan 14, 2020

damemi commented Jan 14, 2020

Huang-Wei Jan 14, 2020 • edited

Choose a reason for hiding this comment

Huang-Wei Jan 14, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

damemi commented Jan 14, 2020

Huang-Wei commented Jan 14, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

damemi commented Jan 16, 2020

ahg-g commented Jan 17, 2020

ahg-g commented Jan 17, 2020

damemi commented Jan 20, 2020

ahg-g commented Jan 20, 2020

k8s-ci-robot commented Jan 20, 2020

damemi commented Jan 20, 2020

Huang-Wei Jan 14, 2020 •

edited

Huang-Wei Jan 14, 2020 •

edited