"kubectl top node" does not return resource usage for Windows nodes

[aravindhp]

What happened: kubectl top node does not return resource usage for Windows nodes

What you expected to happen: kubectl top node should return CPU/Memory/Storage resource usage for Windows nodes also.

How to reproduce it (as minimally and precisely as possible): Add a Windows node to a cluster and execute kubectl top node

kubectl top node 
NAME                                         CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%     
ip-10-0-131-234.us-east-2.compute.internal   595m         17%    6204Mi          42%         
ip-10-0-158-32.us-east-2.compute.internal    511m         34%    4625Mi          70%         
ip-10-0-164-33.us-east-2.compute.internal    155m         10%    2085Mi          31%         
ip-10-0-191-246.us-east-2.compute.internal   551m         15%    6018Mi          41%         
ip-10-0-197-247.us-east-2.compute.internal   127m         8%     1693Mi          25%         
ip-10-0-198-97.us-east-2.compute.internal    708m         20%    6664Mi          45%         
ip-10-0-156-117.us-east-2.compute.internal   <unknown>                           <unknown>               <unknown>               <unknown>               

Here ip-10-0-156-117.us-east-2.compute.internal is a Windows node.

kubectl get node ip-10-0-156-117.us-east-2.compute.internal -o wide
NAME                                         STATUS   ROLES    AGE   VERSION                       INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                         KERNEL-VERSION    CONTAINER-RUNTIME
ip-10-0-156-117.us-east-2.compute.internal   Ready    worker   78m   v1.19.2-1007+ad738ba548b6d6   10.0.156.117   <none>        Windows Server 2019 Datacenter   10.0.17763.1518   docker://19.3.12

Environment:

• Kubernetes version (use kubectl version): v1.20.0 (OpenShift 4.7.0-fc.5)
• Cloud provider or hardware configuration: AWS

[k8s-ci-robot]

@aravindhp: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[aravindhp]

/sig windows
/sig cli

[jsturtevant]

Can share the logs from the metric server and the kubelet?

There could be a couple things going on. This is similiar to #95735 which can be reduced via #95735 and setting reserves for kubelet.

I also found #98509 happens and metric server fails to scrape metrics which #98509 should help.

[kevin-shelaga]

This does not appear to be an issue for windows nodes in gke.

 k top nodes

NAME                                           CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
gke-dev-cluster-1-primary-pool-e3c5e6f5-14qw   1669m        10%    13355Mi         24%
gke-dev-cluster-1-primary-pool-e3c5e6f5-4wkd   2115m        13%    19515Mi         35%
gke-dev-cluster-1-primary-pool-e3c5e6f5-n4pt   1184m        7%     7291Mi          13%
gke-dev-cluster-1-windows-pool-3823a07c-fjqm   150m         7%     793Mi           15%
gke-dev-cluster-1-windows-pool-3823a07c-obtv   112m         5%     790Mi           15%

[jsturtevant]

@kevin-shelaga Thanks for the report. For the most part it is working I believe, we have seen it sporadically particularly under heavy load or when lots of containers are being created.

[marosset]

Can you make sure metrics-server is healthy for the node?

[mansikulkarni96]

Please retitle to "kubectl top node" does not* return resource usage for Windows nodes

[k8s-ci-robot]

@mansikulkarni96: Re-titling can only be requested by trusted users, like repository collaborators.

In response to this:

/retitle "kubectl top node" does not return resource usage for Windows nodes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[aravindhp]

/retitle "kubectl top node" does not return resource usage for Windows nodes

[aravindhp]

/assign @mansikulkarni96

[aravindhp]

/close

Fixes are in prometheus-operator/kube-prometheus#1058 and openshift/cluster-monitoring-operator#1127. Nice work @mansikulkarni96

[k8s-ci-robot]

@aravindhp: Closing this issue.

In response to this:

/close

Fixes are in prometheus-operator/kube-prometheus#1058 and openshift/cluster-monitoring-operator#1127. Nice work @mansikulkarni96

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.