Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce the number of expensive calls in the Windows stats queries for dockershim #104287

Merged
merged 1 commit into from
Nov 16, 2021

Conversation

jsturtevant
Copy link
Contributor

@jsturtevant jsturtevant commented Aug 11, 2021

What type of PR is this?

/kind bug

What this PR does / why we need it:

This improves the response time and minimizes the calls to the system stats which reduces overall cpu for kubelet. This is done by passing the container object, which has the information required, instead of querying for it on every container via the dockerservice. It also caches some information like the runtime info so it is not queried on every container.

Dockershim create a new namepiped instance to docker every call causing reads and higher cpu usage. (Containerd re-uses the same namedpipe instance so doesn't see the same cpu spike).

Which issue(s) this PR fixes:

Fixes #104283

Special notes for your reviewer:

I initially included the commits from #105744 but moved it to a separate PR. That PR also improves the overall CPU used here but not opening and list all the containers twice.

Does this PR introduce a user-facing change?

Reduce the number of calls to docker for stats via dockershim. For Windows this reduces the latency when calling docker, for Linux this saves cpu cycles. 

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/bug Categorizes issue or PR as related to a bug. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/apiserver area/cloudprovider area/code-generation area/dependency Issues or PRs related to dependency changes area/kubectl area/kubelet sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/cli Categorizes an issue or PR as relevant to SIG CLI. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/instrumentation Categorizes an issue or PR as relevant to SIG Instrumentation. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/storage Categorizes an issue or PR as relevant to SIG Storage. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Aug 11, 2021
@jsturtevant jsturtevant marked this pull request as draft August 11, 2021 17:48
@marosset
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 10, 2021
@jsturtevant
Copy link
Contributor Author

unrelated flake #106326
/retest

@jsturtevant
Copy link
Contributor Author

/assign @dchen1107

@marosset
Copy link
Contributor

/assign @mrunalp

@varshaprasad96
Copy link
Contributor

Hi @jsturtevant,
k8s 1.23 bug triage here 👋,
Just wanted to follow up and check if this issue is on track for 1.23?

@marosset
Copy link
Contributor

Hi @jsturtevant, k8s 1.23 bug triage here 👋, Just wanted to follow up and check if this issue is on track for 1.23?

We would to have this merged for 1.23. We just need final approval from area owners.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jsturtevant, mrunalp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@marosset
Copy link
Contributor

/priority critical-urgent

@ehashman
Copy link
Member

/remove-priority backlog
/priority critical-urgent

@k8s-ci-robot k8s-ci-robot added priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. and removed priority/backlog Higher priority than priority/awaiting-more-evidence. labels Nov 29, 2021
k8s-ci-robot added a commit that referenced this pull request Dec 1, 2021
…#104287-upstream-release-1.20

Automated cherry pick of #104287: Reduce calls to docker from dockershim for stats
k8s-ci-robot added a commit that referenced this pull request Dec 1, 2021
…#104287-upstream-release-1.22

Automated cherry pick of #104287: Reduce calls to docker from dockershim for stats
k8s-ci-robot added a commit that referenced this pull request Dec 1, 2021
…#104287-upstream-release-1.21

Automated cherry pick of #104287: Reduce calls to docker from dockershim for stats
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubelet cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/windows Categorizes an issue or PR as relevant to SIG Windows. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Context Deadline exceeded when Metrics Server calls Windows Stats endpoint