Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Revert kubelet to default to ttl cache secret/configmap behavior #74755

Merged
merged 1 commit into from
Mar 2, 2019
Merged

Revert kubelet to default to ttl cache secret/configmap behavior #74755

merged 1 commit into from
Mar 2, 2019

Conversation

liggitt
Copy link
Member

@liggitt liggitt commented Feb 28, 2019 •
edited
Loading

What type of PR is this?
/kind bug

What this PR does / why we need it:
Reverts the kubelet to pre-1.12 behavior for fetching configmaps/secrets, due to http/2 stream saturation issues (golang/go#27044, fixed in go1.12, but not available to k8s 1.12/1.13). Once we update to golang 1.12+, and have scale tests in place to test this scenerio more thoroughly, we can re-enable the watching manager by default.

Which issue(s) this PR fixes:
xref #74412

Does this PR introduce a user-facing change?:

kubelet: resolved hang/timeout issues when running large numbers of pods with unique configmap/secret references

/cc @wojtek-t
@kubernetes/sig-scalability-pr-reviews

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/scalability Categorizes an issue or PR as relevant to SIG Scalability. kind/bug Categorizes issue or PR as related to a bug. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/kubelet sig/node Categorizes an issue or PR as relevant to SIG Node. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Feb 28, 2019
logicalhan
logicalhan approved these changes Feb 28, 2019
View reviewed changes
Copy link
Member

@logicalhan logicalhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 28, 2019
@liggitt
Copy link
Member Author

liggitt commented Feb 28, 2019

/hold

would like @kubernetes/sig-scalability-pr-reviews @kubernetes/sig-node-pr-reviews input

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 28, 2019
@k8s-ci-robot k8s-ci-robot added area/kubeadm sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Feb 28, 2019
@liggitt
Copy link
Member Author

liggitt commented Feb 28, 2019

/cc @yujuhong

@yujuhong
Copy link
Contributor

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 28, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt, yujuhong

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@liggitt
Copy link
Member Author

liggitt commented Mar 1, 2019

/retest

@yujuhong yujuhong added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Mar 1, 2019
@liggitt
Copy link
Member Author

liggitt commented Mar 2, 2019

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 2, 2019
This was referenced Mar 2, 2019
Merged
Merged
@k8s-ci-robot k8s-ci-robot merged commit d8d260a into kubernetes:master Mar 2, 2019
@DanyC97 DanyC97 mentioned this pull request Mar 2, 2019
Closed
@liggitt liggitt deleted the kubelet-watch branch March 5, 2019 00:02
k8s-ci-robot added a commit that referenced this pull request Mar 5, 2019
@k8s-ci-robot
Merge pull request #74842 from liggitt/automated-cherry-pick-of-#7475…
a85aa81 
…5-upstream-release-1.12

Automated cherry pick of #74755: Revert kubelet to default to ttl cache secret/configmap
@liggitt liggitt mentioned this pull request Mar 7, 2019
Closed
@krzysied krzysied mentioned this pull request Mar 8, 2019
Closed
@mm4tt
Copy link
Contributor

mm4tt commented Mar 8, 2019

@liggitt, this PR seems to be related to recent flakiness of our scalability tests, see #75171 (comment)

What is the long-term plan here? Are we going to revert this PR?
AFAIK, golang has been already updated to 1.12.

@liggitt
Copy link
Member Author

liggitt commented Mar 8, 2019

What is the long-term plan here? Are we going to revert this PR?
AFAIK, golang has been already updated to 1.12.

not until our http/2 dependency is bumped to go1.12 levels and we scale test the watching secret manager (see #74781 and #71501 (comment))

@mm4tt
Copy link
Contributor

mm4tt commented Mar 8, 2019

I see, thanks.

In that case we need to figure out what exactly is causing flakiness in #75171, there might be some other bug involved there.

@liggitt liggitt mentioned this pull request Mar 13, 2019
Merged
k8s-ci-robot added a commit that referenced this pull request Mar 18, 2019
@k8s-ci-robot
Merge pull request #74841 from liggitt/automated-cherry-pick-of-#7475…
da36728 
…5-upstream-release-1.13

Automated cherry pick of #74755: Revert kubelet to default to ttl cache secret/configmap
@ficcaglia ficcaglia mentioned this pull request Mar 21, 2019
Closed
@liggitt liggitt mentioned this pull request Apr 14, 2019
Closed
@liggitt liggitt added this to the v1.14 milestone Jun 10, 2019
@shyamjvs shyamjvs mentioned this pull request Oct 11, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@logicalhan logicalhan logicalhan approved these changes

@wojtek-t wojtek-t Awaiting requested review from wojtek-t

@yujuhong yujuhong Awaiting requested review from yujuhong

Assignees

@logicalhan logicalhan

@yujuhong yujuhong

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubeadm area/kubelet cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/scalability Categorizes an issue or PR as relevant to SIG Scalability. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Milestone
v1.14
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@liggitt @yujuhong @k8s-ci-robot @mm4tt @logicalhan