Add DisableAcceleratorUsageMetrics Feature Gate #91930
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
k8s-ci-robot
added
release-note
|
/sig-node |
k8s-ci-robot
added
area/kubelet
sig/node
|
This PR may require API review. If so, when the changes are ready, complete the pre-review checklist and request an API review. Status of requested reviews is tracked in the API Review project. |
RenaudWasTaken
force-pushed
the
DisableAcceleratorUsageMetrics
branch
2 times, most recently
from
93ecdea
to
fe462d5
Compare
RenaudWasTaken
force-pushed
the
DisableAcceleratorUsageMetrics
branch
3 times, most recently
from
0441f3b
to
b12cd14
Compare
|
/test pull-kubernetes-e2e-kind-ipv6 |
|
/remove-kind api-change |
k8s-ci-robot
removed
the
kind/api-change
|
/retest |
|
/lgtm |
k8s-ci-robot
added
the
lgtm
|
/kind api-change |
k8s-ci-robot
added
the
kind/api-change
lavalamp
reviewed
Jun 18, 2020
|
/lgtm will move back in milestone if sig-release approves. |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: dchen1107, derekwaynecarr, RenaudWasTaken The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
derekwaynecarr
added
the
priority/important-longterm
k8s-ci-robot
removed
the
needs-priority
Signed-off-by: Renaud Gaubert <rgaubert@nvidia.com>
RenaudWasTaken
force-pushed
the
DisableAcceleratorUsageMetrics
branch
from
e2018ca
to
34dc785
Compare
k8s-ci-robot
removed
the
lgtm
|
/lgtm |
k8s-ci-robot
added
the
lgtm
|
/retest |
1 similar comment
|
/retest |
|
/lgtm again! |
|
Since the exception request was approved and the PR is already LGTM'ed and Approved by the owning SIG, adding the PR back into the v1.19 milestone. /milestone v1.19 |
|
/retest |
2 similar comments
|
/retest |
|
/retest |
|
/retest Review the full test history for this PR. Silence the bot with an |
1 similar comment
|
/retest Review the full test history for this PR. Silence the bot with an |
|
/retest |
1 similar comment
|
/retest |
|
/retest
…On Wed, Jul 22, 2020 at 7:04 PM Kubernetes Prow Robot < ***@***.***> wrote:
@RenaudWasTaken <https://github.com/RenaudWasTaken>: The following tests
*failed*, say /retest to rerun all failed tests:
Test name Commit Details Rerun command
pull-kubernetes-kubemark-e2e-gce-big 34dc785
<34dc785>
link
<https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/91930/pull-kubernetes-kubemark-e2e-gce-big/1285900026907725826/> /test
pull-kubernetes-kubemark-e2e-gce-big
pull-kubernetes-e2e-gce-100-performance 34dc785
<34dc785>
link
<https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/91930/pull-kubernetes-e2e-gce-100-performance/1285900026815451136/> /test
pull-kubernetes-e2e-gce-100-performance
Full PR test history
<https://prow.k8s.io/pr-history?org=kubernetes&repo=kubernetes&pr=91930>. Your
PR dashboard. Please help us cut down on flakes by linking to
<https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests>
an open issue
<https://github.com/kubernetes/kubernetes/issues?q=is:issue+is:open> when
you hit one in your PR.
Instructions for interacting with me using PR comments are available here
<https://git.k8s.io/community/contributors/guide/pull-requests.md>. If
you have questions or suggestions related to my behavior, please file an
issue against the kubernetes/test-infra
<https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:>
repository. I understand the commands that are listed here
<https://go.k8s.io/bot-commands>.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#91930 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AD24BUE7ES3QEV7Q2M2FGFLR43THXANCNFSM4NZCX7JA>
.
|
|
/retest |
|
/retest
…On Wed, Jul 22, 2020 at 8:53 PM Kubernetes Prow Robot < ***@***.***> wrote:
@RenaudWasTaken <https://github.com/RenaudWasTaken>: The following test
*failed*, say /retest to rerun all failed tests:
Test name Commit Details Rerun command
pull-kubernetes-typecheck 34dc785
<34dc785>
link
<https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/91930/pull-kubernetes-typecheck/1285956146762354688> /test
pull-kubernetes-typecheck
Full PR test history
<https://prow.k8s.io/pr-history?org=kubernetes&repo=kubernetes&pr=91930>. Your
PR dashboard. Please help us cut down on flakes by linking to
<https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests>
an open issue
<https://github.com/kubernetes/kubernetes/issues?q=is:issue+is:open> when
you hit one in your PR.
Instructions for interacting with me using PR comments are available here
<https://git.k8s.io/community/contributors/guide/pull-requests.md>. If
you have questions or suggestions related to my behavior, please file an
issue against the kubernetes/test-infra
<https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:>
repository. I understand the commands that are listed here
<https://go.k8s.io/bot-commands>.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#91930 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AD24BUF5YI6UGTKHCTMFWKDR4376VANCNFSM4NZCX7JA>
.
|
|
It merged! 🚀 |
|
Wooooooooooooooooooooo! Thanks for running the retest command 😛! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Renaud Gaubert rgaubert@nvidia.com
What type of PR is this?
/kind feature
What this PR does / why we need it: Adds the "DisableAcceleratorUsageMetrics" Feature Gate.
TLDR: Kubelet collects GPU metrics when it sees that the NVIDIA driver is present on the node.
Kubelet should no longer be collecting metrics from devices. The expected path is to use the pod resources API and have device vendors expose metrics through their own metrics container. This path is here for legacy reasons and we are deprecating it.
Furthermore because Kubelet now has an open handle on the NVIDIA driver, this breaks any infrastructure interactions (e.g: Removing or Updating the driver) with the NVIDIA driver. In other words any actions related to the NVIDIA driver cannot be taken without killing the kubelet.
See google/cadvisor#2574 for more details
Special notes for your reviewer:
This change does not remove the handle cadvisor has on the NVIDIA driver. To do so we will also need to wait for cadvisor to cut a new release and re-vendor cadvisor (so that we have the following PR in: google/cadvisor#2574).
/cc @dashpole
Does this PR introduce a user-facing change?:
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:
None, does this require a KEP / Doc / ... ?