New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add network stats for Windows containers #70121

Merged
merged 2 commits into from Jan 25, 2019

Conversation

@feiskyer
Copy link
Member

feiskyer commented Oct 23, 2018

What this PR does / why we need it:

An alternative way for #67884 and #69920.

  • To get node network stats, it invokes 'Get-NetAdapterStatistics | ConvertTo-Json' to fetch all adapters statistics
  • To get pod stats, it invokes hcsshim directly to fetch container's network statistics.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #64901

Special notes for your reviewer:

Refer #67884 (comment).

Release note:

Add network stats for Windows nodes and pods.

/assign @yujuhong @PatrickLang
/milestone v1.13
/sig windows

@k8s-ci-robot k8s-ci-robot added this to the v1.13 milestone Oct 23, 2018

@feiskyer feiskyer changed the title Win net stats3 Add network stats for Windows containers Oct 23, 2018

@k8s-ci-robot k8s-ci-robot requested review from yifan-gu and yujuhong Oct 23, 2018

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Oct 23, 2018

/kind bug

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Oct 23, 2018

@k8s-ci-robot k8s-ci-robot requested a review from dineshgovindasamy Oct 23, 2018

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Oct 23, 2018

/test pull-kubernetes-verify

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Oct 23, 2018

/retest

@dineshgovindasamy
Copy link

dineshgovindasamy left a comment

Thanks Pengfei. PR looks good.
/lgtm

Show resolved Hide resolved pkg/kubelet/winstats/network_stats.go

@k8s-ci-robot k8s-ci-robot added the lgtm label Oct 24, 2018

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Oct 24, 2018

/test pull-kubernetes-integration

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Oct 25, 2018

ping @yujuhong Could you have a look at this?

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Nov 6, 2018

ping @yujuhong

@benmoss

This comment has been minimized.

Copy link
Member

benmoss commented Nov 7, 2018

This all seems reasonable to me. The cri_stats_provider.go code seems like it might benefit from a higher abstraction, since it's now weaving between the Linux-only cadvisor calls and the Windows-only listContainerNetworkStats call, but I'm not sure that needs to happen now.

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Nov 8, 2018

The cri_stats_provider.go code seems like it might benefit from a higher abstraction, since it's now weaving between the Linux-only cadvisor calls and the Windows-only listContainerNetworkStats call, but I'm not sure that needs to happen now.

The best way I think is abstracting the windows features also under cadvisor interface, but that's not possible yet because there's no enough information from hcsshim (which means CRI interface should be changed to fetch them).

And there is also another impl #69920 for this feature which I think is cleaner, but that requires changes of CRI.

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Nov 19, 2018

Rebased and conflicts solved.

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Jan 9, 2019

ping @yujuhong @derekwaynecarr Help to take at the PR again?

@PatrickLang PatrickLang added this to In Review in SIG-Windows Jan 11, 2019

@yujuhong
Copy link
Member

yujuhong left a comment

Looks good mostly with some comments.

Show resolved Hide resolved pkg/kubelet/winstats/network_stats.go Outdated
Show resolved Hide resolved pkg/kubelet/winstats/network_stats.go
Show resolved Hide resolved pkg/kubelet/stats/cri_stats_provider_windows.go
Show resolved Hide resolved pkg/kubelet/stats/cri_stats_provider_windows.go Outdated
@yujuhong

This comment has been minimized.

Copy link
Member

yujuhong commented Jan 11, 2019

/cc @pjh FYI

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Jan 11, 2019

@yujuhong: GitHub didn't allow me to request PR reviews from the following users: pjh, FYI.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @pjh FYI

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@feiskyer feiskyer force-pushed the feiskyer:win-net-stats3 branch from 1d2151a to c33fdbe Jan 18, 2019

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Jan 18, 2019

@yujuhong Addressed comments. PTAL

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Jan 18, 2019

/retest

@yujuhong

This comment has been minimized.

Copy link
Member

yujuhong commented Jan 18, 2019

@feiskyer does the existing summary API test check for this, and does the test pass after the change?
Since there is no presubmit testing yet, I'd like to make sure the features added will be tested.

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Jan 19, 2019

The test covers some, but not all of this, because of invoking powershell commands. We may figure out how to mock powershell later.

@feiskyer

This comment has been minimized.

Copy link
Member Author

feiskyer commented Jan 19, 2019

/retest

@feiskyer feiskyer force-pushed the feiskyer:win-net-stats3 branch from c33fdbe to 9cf38de Jan 24, 2019

@yujuhong

This comment has been minimized.

Copy link
Member

yujuhong commented Jan 25, 2019

The test covers some, but not all of this, because of invoking powershell commands. We may figure out how to mock powershell later.

Actually I was talking about e2e tests, but having unit test is good too. We need something to prevent regression. I'm slightly uncomfortable that we can't run node e2e on windows since summary API test is also there.
Can you file a bug to add test coverage for stats in general?

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm label Jan 25, 2019

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Jan 25, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: feiskyer, yujuhong

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 697c231 into kubernetes:master Jan 25, 2019

17 of 18 checks passed

pull-kubernetes-local-e2e-containerized Job failed.
Details
cla/linuxfoundation feiskyer authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-kops-aws Job succeeded.
Details
pull-kubernetes-e2e-kubeadm-gce Skipped
pull-kubernetes-godeps Skipped
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
tide In merge pool.
Details

SIG-Windows automation moved this from In Review to Done (v.1.14) Jan 25, 2019

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Jan 25, 2019

@feiskyer: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-kubernetes-local-e2e-containerized 9cf38de link /test pull-kubernetes-local-e2e-containerized

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@feiskyer feiskyer deleted the feiskyer:win-net-stats3 branch Jan 26, 2019

@jterry75

This comment has been minimized.

Copy link

jterry75 commented Jan 29, 2019

@kevpar - FYI - You should look at this to match CRI stats

k8s-ci-robot added a commit that referenced this pull request Feb 15, 2019

Merge pull request #74114 from feiskyer/revert-70121
Revert #70121: Add network stats for Windows containers #70121
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment