Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sync CNI config in goroutine #74389

Merged

Conversation

@answer1991
Copy link
Contributor

commented Feb 22, 2019

What type of PR is this?

/kind bug

What this PR does / why we need it:

This PR fixes #74388 .
Kubelet run a new goroutine to sync CNI config if the runtime is dockershim, which fix the unexpected NotReady status when Node's iops is full.

Which issue(s) this PR fixes:

Fixes #74388

Special notes for your reviewer:
Before this fix, sync CNI config in the status check is inappropriate.

Does this PR introduce a user-facing change?:

Fix the unexpected NotReady status when Node's iops is full if the runtime is dockershim.
@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Feb 22, 2019

Hi @answer1991. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@answer1991

This comment has been minimized.

Copy link
Contributor Author

commented Feb 22, 2019

@answer1991

This comment has been minimized.

Copy link
Contributor Author

commented Feb 22, 2019

@answer1991

This comment has been minimized.

Copy link
Contributor Author

commented Feb 22, 2019

@dchen1107

PTAL

@zhangxiaoyu-zidif

This comment has been minimized.

Copy link
Member

commented Feb 22, 2019

/ok-to-test

@zhangxiaoyu-zidif

This comment has been minimized.

Copy link
Member

commented Feb 22, 2019

/kind bug

@k8s-ci-robot k8s-ci-robot added kind/bug and removed needs-kind labels Feb 22, 2019
@changyaowei

This comment has been minimized.

Copy link
Contributor

commented Feb 22, 2019

We encounter the same question, LGTM

@zhangxiaoyu-zidif

This comment has been minimized.

Copy link
Member

commented Feb 22, 2019

exec hack/update-bazel.sh please, if you modify import.

@answer1991 answer1991 force-pushed the answer1991:sync-cni-config-in-goroutine branch from a6f07eb to 4fe13c2 Feb 22, 2019
@answer1991

This comment has been minimized.

Copy link
Contributor Author

commented Feb 22, 2019

@zhangxiaoyu-zidif

cool, already done. Thanks

@answer1991

This comment has been minimized.

Copy link
Contributor Author

commented Feb 22, 2019

/retest

@resouer

This comment has been minimized.

Copy link
Member

commented Feb 22, 2019

The change looks good from kubelet view, but should be defer to Dan (@dcbw)

/assign @dcbw

@dcbw

This comment has been minimized.

Copy link
Member

commented Feb 22, 2019

Discussed with @squeed this morning. I think this is an acceptable workaround for now.

In the very near future we'd like to kill the periodic resync entirely and make the plugin choice more declarative, whether through a disk file or the kubelet ConfigMap (since dockershim doesn't have its own config options yet, they still go through Kubelet).

/approve
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm label Feb 22, 2019
@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Feb 22, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: answer1991, dcbw

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@fejta-bot

This comment has been minimized.

Copy link

commented Feb 22, 2019

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

1 similar comment
@fejta-bot

This comment has been minimized.

Copy link

commented Feb 22, 2019

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@k8s-ci-robot k8s-ci-robot merged commit b77e4e2 into kubernetes:master Feb 23, 2019
16 checks passed
16 checks passed
cla/linuxfoundation answer1991 authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-godeps Skipped
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
pull-publishing-bot-validate Skipped
tide In merge pool.
Details
@answer1991

This comment has been minimized.

Copy link
Contributor Author

commented Feb 23, 2019

@dcbw

Actually, we had already refactor CNI config using ConfigMap in our downstream K8S. But the implement has already broken the CNI specification. So this PR use this workaround to follow the CNI specification.

Please feel free to let me know if you need any help when you start to refactor these codes, me and @changyaowei would like to provide some helps.

@dcbw

This comment has been minimized.

Copy link
Member

commented Feb 23, 2019

@answer1991 I'm actually rethinking the ConfigMap bits; depending on the runtime you use (dockershim or crio or something else) kubelet may not be involved in the CNI decisions at all. So having any network plugin options in the kubelet config would be useless with a remote runtime like CRIO. Plus, dockershim is supposed to eventually also be a remote runtime and wouldn't use the kubelet options.

Which leads me to believe that perhaps the file-based approach is a better one for now, even if it isn't as "kubernetes-y".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.