Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The kubelet /stats/summary is too expensive on windows #82522

Open
wawa0210 opened this issue Sep 10, 2019 · 1 comment

Comments

@wawa0210
Copy link

commented Sep 10, 2019

What happened:

Collect host and container information via kubelet stats/summay. By looking the kubelet code ,it's collected by cadvisor and the stats/summary request is kept within 50 ms.

But the implementation in the Windows environment is very cumbersome。First it will get all the pods managed by kubelet ,and call docker stats request one by one (very very expensive)

This is my set of test data

The pod is very sample just the mcr.microsoft.com/windows/nanoserver:1903 and loop ping

pod nums stats/summay times
5 10s
10 20s
15 30.50s
20 41.38s

On the Linux platform, the time cost will not change too much and remain within 100ms.
This is unacceptable on windows .

By studying the kubelet code, the main time consuming is to request the stats of each container through docker, which takes up almost 95% of the entire request time.

statsJSON, err := ds.client.GetContainerStats(containerID)

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Deploy 5 to 10 pods to the windows host then request kubelet 10255 stats/summay

Anything else we need to know?:

Environment:

  • Kubernetes version (use kubectl version): 1.15.3
  • Windows server version : windows server 2019 17763.678
  • Docker version : docker ee 18.09.6

/sig node

@rhockenbury

This comment has been minimized.

Copy link

commented Sep 10, 2019

This has been reported previously in #75752 and #74991

@PatrickLang PatrickLang added this to Backlog (v1.17) in SIG-Windows Sep 11, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
4 participants
You can’t perform that action at this time.