Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ipvs: fix string check for IPVS protocol during graceful termination #78999

Merged
merged 1 commit into from
Jun 15, 2019
Merged

ipvs: fix string check for IPVS protocol during graceful termination #78999

merged 1 commit into from
Jun 15, 2019

Conversation

andrewsykim
Copy link
Member

@andrewsykim andrewsykim commented Jun 13, 2019 •
edited
Loading

Signed-off-by: Andrew Sy Kim kiman@vmware.com

What type of PR is this?
/kind bug

What this PR does / why we need it:
In #77802 we removed graceful termination for UDP (causes many issues when there are too many connections/real servers). It turns out that in pkg/util/ipvs we actually convert the protocol values from netlink to upper case strings ("TCP", "UDP", etc, see protocolToString) even though the string passed into docker/libnetwork/ipvs is lower case string (see stringToProtocol).

This PR fixes this bug by always checking upper case values. As a follow-up we should consolidate the string casing for utils/ipvs to ensure this doesn't happen again. The PR for this will be larger so wanted to get the bug fix only change in before we cut v1.15 and for easier cherry-picking for previous versions.

I'm missing context as to why we use lower case values for stringToProtocol and upper case values for protocolToString, would appreciate some background there @m1093782566 @lbernail.

Which issue(s) this PR fixes:
Fixes ##78993

DNS service in the reported issue went down due to too many stale real server entries being held by graceful termination.

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Fix a string comparison bug in IPVS graceful termination where UDP real servers are not deleted.

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. kind/bug Categorizes issue or PR as related to a bug. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jun 13, 2019
@andrewsykim
Copy link
Member Author

/assign @m1093782566 @lbernail

@k8s-ci-robot k8s-ci-robot added area/ipvs sig/network Categorizes an issue or PR as relevant to SIG Network. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jun 13, 2019
@andrewsykim
Copy link
Member Author

/priority critical-urgent
/milestone v1.15

@k8s-ci-robot k8s-ci-robot added the priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. label Jun 13, 2019
@k8s-ci-robot k8s-ci-robot added this to the v1.15 milestone Jun 13, 2019
@k8s-ci-robot k8s-ci-robot removed the needs-priority Indicates a PR lacks a `priority/foo` label and requires one. label Jun 13, 2019
@andrewsykim andrewsykim changed the title ipvs: fix string check for IPVS protocol ipvs: fix string check for IPVS protocol graceful termination Jun 13, 2019
@andrewsykim andrewsykim changed the title ipvs: fix string check for IPVS protocol graceful termination ipvs: fix string check for IPVS protocol during graceful termination Jun 13, 2019
@andrewsykim
ipvs: fix string check for IPVS protocol during graceful termination
bb95143 
Signed-off-by: Andrew Sy Kim <kiman@vmware.com>
@andrewsykim
Copy link
Member Author

More proper fix for a follow-up PR would be to change the string values returned in protocolToString to lower case values to be consistent with values passed into libnetwork/ipvs. Didn't do that in this PR since I'm not sure what the implications are yet.

@andrewsykim
Copy link
Member Author

Also created #79001 re: e2e coverage for IPVS graceful termination

@andrewsykim andrewsykim mentioned this pull request Jun 13, 2019
Merged
@andrewsykim
Copy link
Member Author

/retest

@lbernail
Copy link
Contributor

Nice catch
I tested with "!=tcp" before we decided to disable it only for UDP. So this worked (but would have broken TCP graceful termination...)

I agree that we should standardize on lower or upper everywhere to avoid this type of issue

@lbernail
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 14, 2019
@lbernail
Copy link
Contributor

We'll need to cherry-pick this in 1.14, 1.13 and 1.12

@bazzargh
Copy link

(for the future cleanup, not this patch) Would suggest moving those strings into constants for consistency.

@dims
Copy link
Member

dims commented Jun 14, 2019

/assign @johnbelamaric

@liggitt
Copy link
Member

liggitt commented Jun 14, 2019 •
edited
Loading

needs approval. once approved, go ahead and open the cherrypick against 1.15 if you want to make v1.15.0, since the master merge pool is full

@tpepper
Copy link
Member

tpepper commented Jun 14, 2019

@kubernetes/patch-release-team

@liggitt
Copy link
Member

liggitt commented Jun 14, 2019

@kubernetes/patch-release-team

note that this is targeting v1.15.0

@andrewsykim andrewsykim mentioned this pull request Jun 14, 2019
Merged
@andrewsykim
Copy link
Member Author

Created #79044 for faster turn over into 1.15 if/when this one gets approved (please approve this one first).

@thockin
Copy link
Member

thockin commented Jun 14, 2019

Thanks!

/lgtm
/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andrewsykim, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 14, 2019
@andrewsykim
Copy link
Member Author

@thockin can you approve the cherry-pick PR as well #79044, thank you

k8s-ci-robot added a commit that referenced this pull request Jun 15, 2019
@k8s-ci-robot
Merge pull request #79044 from andrewsykim/automated-cherry-pick-of-#…
e8462b5 
…78999-origin-release-1.15

Automated cherry pick of #78999: ipvs: fix string check for IPVS protocol during graceful
@k8s-ci-robot k8s-ci-robot merged commit 101f9ff into kubernetes:master Jun 15, 2019
This was referenced Jun 17, 2019
Merged
Merged
Merged
Closed
k8s-ci-robot added a commit that referenced this pull request Jun 18, 2019
@k8s-ci-robot
Merge pull request #79103 from andrewsykim/automated-cherry-pick-of-#…
55318c8 
…78999-origin-release-1.12

Automated cherry pick of #78999: ipvs: fix string check for IPVS protocol during graceful
k8s-ci-robot added a commit that referenced this pull request Jun 20, 2019
@k8s-ci-robot
Merge pull request #79101 from andrewsykim/automated-cherry-pick-of-#…
cb7218f 
…78999-origin-release-1.14

Automated cherry pick of #78999: ipvs: fix string check for IPVS protocol during graceful
k8s-ci-robot added a commit that referenced this pull request Jun 22, 2019
@k8s-ci-robot
Merge pull request #79102 from andrewsykim/automated-cherry-pick-of-#…
d952267 
…78999-origin-release-1.13

Automated cherry pick of #78999: ipvs: fix string check for IPVS protocol during graceful
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@khenidak khenidak khenidak approved these changes

@cmluciano cmluciano Awaiting requested review from cmluciano

@dnardo dnardo Awaiting requested review from dnardo

Assignees

@m1093782566 m1093782566

@lbernail lbernail

@thockin thockin

@johnbelamaric johnbelamaric

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/ipvs cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/network Categorizes an issue or PR as relevant to SIG Network. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Milestone
v1.15
Development

Successfully merging this pull request may close these issues.