Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add all-pods log flag to kubectl #124732

Merged
merged 1 commit into from
May 14, 2024
Merged

feat: add all-pods log flag to kubectl #124732

merged 1 commit into from
May 14, 2024

Conversation

cmwylie19
Copy link
Contributor

@cmwylie19 cmwylie19 commented May 7, 2024 •
edited

What type of PR is this?

/kind feature

ref: #123874

What this PR does / why we need it:

This adds an --all-pods option to get all pod logs from replicaSets or services by looking at the label selector and finding corresponding pods in the namespace. It reduces the need to look for logs based on labels.

Which issue(s) this PR fixes:

Fixes kubernetes/kubectl#1520

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Added flag to `kubectl logs` called `--all-pods` to get all pods from a object that uses a pod selector.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

kubectl logs <deploy/sts/daemonset/svc>/<name> --all-pods

usage

┌─[cmwylie19@Cases-MacBook-Pro] - [~/kubernetes] - [2024-03-11 06:37:09]
└─[0] <git:(1520 570a1ede3fd✱) > go run cmd/kubectl/kubectl.go logs svc/y --all-pods
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2024/03/07 21:10:16 [notice] 1#1: using the "epoll" event method
2024/03/07 21:10:16 [notice] 1#1: nginx/1.25.4
2024/03/07 21:10:16 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14) 
2024/03/07 21:10:16 [notice] 1#1: OS: Linux 5.15.49-linuxkit
2024/03/07 21:10:16 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2024/03/07 21:10:16 [notice] 1#1: start worker processes
2024/03/07 21:10:16 [notice] 1#1: start worker process 29
2024/03/07 21:10:16 [notice] 1#1: start worker process 30
2024/03/07 21:10:16 [notice] 1#1: start worker process 31
2024/03/07 21:10:16 [notice] 1#1: start worker process 32
2024/03/07 21:10:16 [notice] 1#1: start worker process 33
2024/03/07 21:10:16 [notice] 1#1: start worker process 34
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2024/03/07 21:15:54 [notice] 1#1: using the "epoll" event method
2024/03/07 21:15:54 [notice] 1#1: nginx/1.25.4
2024/03/07 21:15:54 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14) 
2024/03/07 21:15:54 [notice] 1#1: OS: Linux 5.15.49-linuxkit
2024/03/07 21:15:54 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2024/03/07 21:15:54 [notice] 1#1: start worker processes
2024/03/07 21:15:54 [notice] 1#1: start worker process 29
2024/03/07 21:15:54 [notice] 1#1: start worker process 30
2024/03/07 21:15:54 [notice] 1#1: start worker process 31
2024/03/07 21:15:54 [notice] 1#1: start worker process 32
2024/03/07 21:15:54 [notice] 1#1: start worker process 33
2024/03/07 21:15:54 [notice] 1#1: start worker process 34
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2024/03/07 21:15:54 [notice] 1#1: using the "epoll" event method
2024/03/07 21:15:54 [notice] 1#1: nginx/1.25.4
2024/03/07 21:15:54 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14) 
2024/03/07 21:15:54 [notice] 1#1: OS: Linux 5.15.49-linuxkit
2024/03/07 21:15:54 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2024/03/07 21:15:54 [notice] 1#1: start worker processes
2024/03/07 21:15:54 [notice] 1#1: start worker process 29
2024/03/07 21:15:54 [notice] 1#1: start worker process 30
2024/03/07 21:15:54 [notice] 1#1: start worker process 31
2024/03/07 21:15:54 [notice] 1#1: start worker process 32
2024/03/07 21:15:54 [notice] 1#1: start worker process 33
2024/03/07 21:15:54 [notice] 1#1: start worker process 34
┌─[cmwylie19@Cases-MacBook-Pro] - [~/kubernetes] - [2024-03-11 06:38:24]
└─[0] <git:(1520 570a1ede3fd✱) > go run cmd/kubectl/kubectl.go logs deploy/y --all-pods --prefix
[pod/y-844fbc9777-vhl79/nginx] /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
[pod/y-844fbc9777-vhl79/nginx] /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
[pod/y-844fbc9777-vhl79/nginx] /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
[pod/y-844fbc9777-vhl79/nginx] 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
[pod/y-844fbc9777-vhl79/nginx] 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
[pod/y-844fbc9777-vhl79/nginx] /docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
[pod/y-844fbc9777-vhl79/nginx] /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
[pod/y-844fbc9777-vhl79/nginx] /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
[pod/y-844fbc9777-vhl79/nginx] /docker-entrypoint.sh: Configuration complete; ready for start up
[pod/y-844fbc9777-vhl79/nginx] 2024/03/07 21:10:16 [notice] 1#1: using the "epoll" event method
[pod/y-844fbc9777-vhl79/nginx] 2024/03/07 21:10:16 [notice] 1#1: nginx/1.25.4
[pod/y-844fbc9777-vhl79/nginx] 2024/03/07 21:10:16 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14) 
[pod/y-844fbc9777-vhl79/nginx] 2024/03/07 21:10:16 [notice] 1#1: OS: Linux 5.15.49-linuxkit
[pod/y-844fbc9777-vhl79/nginx] 2024/03/07 21:10:16 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
[pod/y-844fbc9777-vhl79/nginx] 2024/03/07 21:10:16 [notice] 1#1: start worker processes
[pod/y-844fbc9777-vhl79/nginx] 2024/03/07 21:10:16 [notice] 1#1: start worker process 29
[pod/y-844fbc9777-vhl79/nginx] 2024/03/07 21:10:16 [notice] 1#1: start worker process 30
[pod/y-844fbc9777-vhl79/nginx] 2024/03/07 21:10:16 [notice] 1#1: start worker process 31
[pod/y-844fbc9777-vhl79/nginx] 2024/03/07 21:10:16 [notice] 1#1: start worker process 32
[pod/y-844fbc9777-vhl79/nginx] 2024/03/07 21:10:16 [notice] 1#1: start worker process 33
[pod/y-844fbc9777-vhl79/nginx] 2024/03/07 21:10:16 [notice] 1#1: start worker process 34
[pod/y-844fbc9777-hz4x4/nginx] /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
[pod/y-844fbc9777-hz4x4/nginx] /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
[pod/y-844fbc9777-hz4x4/nginx] /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
[pod/y-844fbc9777-hz4x4/nginx] 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
[pod/y-844fbc9777-hz4x4/nginx] 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
[pod/y-844fbc9777-hz4x4/nginx] /docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
[pod/y-844fbc9777-hz4x4/nginx] /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
[pod/y-844fbc9777-hz4x4/nginx] /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
[pod/y-844fbc9777-hz4x4/nginx] /docker-entrypoint.sh: Configuration complete; ready for start up
[pod/y-844fbc9777-hz4x4/nginx] 2024/03/07 21:15:54 [notice] 1#1: using the "epoll" event method
[pod/y-844fbc9777-hz4x4/nginx] 2024/03/07 21:15:54 [notice] 1#1: nginx/1.25.4
[pod/y-844fbc9777-hz4x4/nginx] 2024/03/07 21:15:54 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14) 
[pod/y-844fbc9777-hz4x4/nginx] 2024/03/07 21:15:54 [notice] 1#1: OS: Linux 5.15.49-linuxkit
[pod/y-844fbc9777-hz4x4/nginx] 2024/03/07 21:15:54 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
[pod/y-844fbc9777-hz4x4/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker processes
[pod/y-844fbc9777-hz4x4/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker process 29
[pod/y-844fbc9777-hz4x4/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker process 30
[pod/y-844fbc9777-hz4x4/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker process 31
[pod/y-844fbc9777-hz4x4/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker process 32
[pod/y-844fbc9777-hz4x4/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker process 33
[pod/y-844fbc9777-hz4x4/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker process 34
[pod/y-844fbc9777-95rhd/nginx] /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
[pod/y-844fbc9777-95rhd/nginx] /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
[pod/y-844fbc9777-95rhd/nginx] /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
[pod/y-844fbc9777-95rhd/nginx] 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
[pod/y-844fbc9777-95rhd/nginx] 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
[pod/y-844fbc9777-95rhd/nginx] /docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
[pod/y-844fbc9777-95rhd/nginx] /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
[pod/y-844fbc9777-95rhd/nginx] /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
[pod/y-844fbc9777-95rhd/nginx] /docker-entrypoint.sh: Configuration complete; ready for start up
[pod/y-844fbc9777-95rhd/nginx] 2024/03/07 21:15:54 [notice] 1#1: using the "epoll" event method
[pod/y-844fbc9777-95rhd/nginx] 2024/03/07 21:15:54 [notice] 1#1: nginx/1.25.4
[pod/y-844fbc9777-95rhd/nginx] 2024/03/07 21:15:54 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14) 
[pod/y-844fbc9777-95rhd/nginx] 2024/03/07 21:15:54 [notice] 1#1: OS: Linux 5.15.49-linuxkit
[pod/y-844fbc9777-95rhd/nginx] 2024/03/07 21:15:54 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
[pod/y-844fbc9777-95rhd/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker processes
[pod/y-844fbc9777-95rhd/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker process 29
[pod/y-844fbc9777-95rhd/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker process 30
[pod/y-844fbc9777-95rhd/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker process 31
[pod/y-844fbc9777-95rhd/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker process 32
[pod/y-844fbc9777-95rhd/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker process 33
[pod/y-844fbc9777-95rhd/nginx] 2024/03/07 21:15:54 [notice] 1#1: start worker process 34

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels May 7, 2024
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 7, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @cmwylie19. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/kubectl area/test sig/cli Categorizes an issue or PR as relevant to SIG CLI. sig/testing Categorizes an issue or PR as relevant to SIG Testing. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels May 7, 2024
@eddiezane
Copy link
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 7, 2024
@eddiezane eddiezane mentioned this pull request May 7, 2024
Closed
@eddiezane
Copy link
Member

eddiezane commented May 10, 2024 •
edited

Aside from the linter, the test failures are legitimate timeouts. 0 1 2 . Need to investigate them a bit deeper tomorrow.

@eddiezane
Copy link
Member

/retest

@eddiezane
Copy link
Member

@aojea any idea why the tests would be failing like this? The test runs fine locally but is hitting the 10 minute timeout while trying to spin up a StatefulSet. It looks like it manages to get 1 pod up at one point but then falls back to 0.

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/124732/pull-kubernetes-e2e-kind/1788771572169838592#1:build-log.txt%3A667-741

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/124732/pull-kubernetes-e2e-gce/1788771572048203776#1:build-log.txt%3A781-855

@eddiezane
Copy link
Member

It looks like both StatefulSets were actually created and logged out successfully

https://gcsweb.k8s.io/gcs/kubernetes-jenkins/pr-logs/pull/124732/pull-kubernetes-e2e-kind/1788771572169838592/artifacts/kubectl-logs-2956/

eddiezane
eddiezane suggested changes May 10, 2024
View reviewed changes
test/e2e/kubectl/logs.go Show resolved Hide resolved
test/e2e/kubectl/logs.go Show resolved Hide resolved
@cmwylie19 cmwylie19 force-pushed the 1520 branch 3 times, most recently from 083b9a1 to d699212 Compare May 13, 2024 22:11
eddiezane
eddiezane suggested changes May 14, 2024
View reviewed changes
Copy link
Member

@eddiezane eddiezane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small change then lgtm!

/approve

test/e2e/kubectl/logs.go Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 14, 2024
@cmwylie19
feat: add all-pods log flag to kubectl
6db859e 
Signed-off-by: Case Wylie <cmwylie19@defenseunicorns.com>
@cmwylie19
Copy link
Contributor Author

/retest

eddiezane
eddiezane approved these changes May 14, 2024
View reviewed changes
Copy link
Member

@eddiezane eddiezane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work on this!

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 14, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: f06afdfd7c71bd6719291a0312fd93b34d434dcc

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cmwylie19, eddiezane

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 9f39314 into kubernetes:master May 14, 2024
15 checks passed
@k8s-ci-robot k8s-ci-robot added this to the v1.31 milestone May 14, 2024
@ah8ad3 ah8ad3 mentioned this pull request May 21, 2024
Merged
@BenTheElder
Copy link
Member

the e2e test seems flaky?

https://go.k8s.io/triage?test=Kubectl%20logs%20all%20pod%20logs%20the%20Deployment%20has%202%20replicas%20

example: https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/125035/pull-kubernetes-e2e-kind-ipv6/1793029056485134336

@BenTheElder
Copy link
Member

(this happens all the time, making e2e tests reliable is hard, but if I'm right and it is this test it would be appreciated to investigate and attempt to increase e2e test reliability 🙏 , at a quick glance nothing jumps out in the test code)

This was referenced May 21, 2024
Merged
Closed
@eddiezane
Copy link
Member

Spoke with Ben. Going to submit something.

@eddiezane eddiezane mentioned this pull request May 21, 2024
Merged
@Rajalakshmi-Girish Rajalakshmi-Girish mentioned this pull request May 22, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wangyysde wangyysde wangyysde left review comments

@eddiezane eddiezane eddiezane approved these changes

@brianpursley brianpursley Awaiting requested review from brianpursley

Assignees

@eddiezane eddiezane

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubectl area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/cli Categorizes an issue or PR as relevant to SIG CLI. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
SIG CLI
Status: Done
Milestone
v1.31
Development

Successfully merging this pull request may close these issues.

Make kubectl log deployment/foo read logs from **all** pods
5 participants
@cmwylie19 @k8s-ci-robot @eddiezane @BenTheElder @wangyysde