Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "Add better logging when iptables-restore fails" #77541

Conversation

krzysied
Copy link
Contributor

@krzysied krzysied commented May 7, 2019

Reverts #76171

@thockin @dcbw - FYI

@k8s-ci-robot
Copy link
Contributor

@krzysied: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. sig/network Categorizes an issue or PR as relevant to SIG Network. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels May 7, 2019
@wojtek-t wojtek-t self-assigned this May 7, 2019
@wojtek-t wojtek-t added release-note-none Denotes a PR that doesn't merit a release note. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. kind/bug Categorizes issue or PR as related to a bug. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-kind Indicates a PR lacks a `kind/foo` label and requires one. labels May 7, 2019
@krzysied
Copy link
Contributor Author

krzysied commented May 7, 2019

Example of logs:

E0502 12:20:18.998222       1 proxier.go:1374] Failed to execute iptables-restore: failed to acquire new iptables lock: timed out waiting for the condition
failed payload:
*filter
:KUBE-SERVICES - [0:0]
:KUBE-EXTERNAL-SERVICES - [0:0]
:KUBE-FORWARD - [0:0]
-A KUBE-SERVICES -m comment --comment "test-dlqat9-2/small-service-125: has no endpoints" -m tcp -p tcp -d 10.0.244.248/32 --dport 80 -j REJECT
-A KUBE-SERVICES -m comment --comment "test-dlqat9-1/small-service-72: has no endpoints" -m tcp -p tcp -d 10.0.195.129/32 --dport 80 -j REJECT
-A KUBE-SERVICES -m comment --comment "test-dlqat9-30/small-service-87: has no endpoints" -m tcp -p tcp -d 10.0.234.239/32 --dport 80 -j REJECT
-A KUBE-SERVICES -m comment --comment "test-dlqat9-32/small-service-79: has no endpoints" -m tcp -p tcp -d 10.0.132.31/32 --dport 80 -j REJECT
-A KUBE-SERVICES -m comment --comment "test-dlqat9-5/small-service-88: has no endpoints" -m tcp -p tcp -d 10.0.91.145/32 --dport 80 -j REJECT
-A KUBE-SERVICES -m comment --comment "test-dlqat9-41/small-service-94: has no endpoints" -m tcp -p tcp -d 10.0.198.154/32 --dport 80 -j REJECT
-A KUBE-SERVICES -m comment --comment "test-dlqat9-44/small-service-61: has no endpoints" -m tcp -p tcp -d 10.0.103.86/32 --dport 80 -j REJECT
-A KUBE-SERVICES -m comment --comment "test-dlqat9-18/small-service-89: has no endpoints" -m tcp -p tcp -d 10.0.209.224/32 --dport 80 -j REJECT
-A KUBE-SERVICES -m comment --comment "test-dlqat9-42/small-service-40: has no endpoints" -m tcp -p tcp -d 10.0.155.8/32 --dport 80 -j REJECT
....

The kube-proxy.log size increased over 1000X due to this change.

@wojtek-t
Copy link
Member

wojtek-t commented May 7, 2019

/lgtm
/approve
/sig scalability

@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. sig/scalability Categorizes an issue or PR as relevant to SIG Scalability. labels May 7, 2019
@k8s-ci-robot
Copy link
Contributor

@krzysied: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. label May 7, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: krzysied, wojtek-t

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. and removed release-note-none Denotes a PR that doesn't merit a release note. labels May 7, 2019
@wojtek-t
Copy link
Member

wojtek-t commented May 7, 2019

@thockin @dcbw

@wojtek-t wojtek-t added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels May 7, 2019
@k8s-ci-robot k8s-ci-robot merged commit 014a2d2 into kubernetes:master May 7, 2019
@dcbw
Copy link
Member

dcbw commented May 8, 2019

@wojtek-t this logging is pretty useful though, so shouldn't we investigate why the proxy failed to restore, rather than killing good debug logging that should almost never happen?

@wojtek-t
Copy link
Member

@wojtek-t this logging is pretty useful though, so shouldn't we investigate why the proxy failed to restore, rather than killing good debug logging that should almost never happen?

I think both - I wouldn't say that logging 40MB or sth of iptables (in very large cluster) contents is particularly useful...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. release-note-none Denotes a PR that doesn't merit a release note. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/scalability Categorizes an issue or PR as relevant to SIG Scalability. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants