New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ipvs broken because ipset on centos does not support comment extension #65461

Closed
honkiko opened this Issue Jun 26, 2018 · 2 comments

Comments

Projects
None yet
3 participants
@honkiko
Contributor

honkiko commented Jun 26, 2018

Is this a BUG REPORT or FEATURE REQUEST?:
/kind bug
/sig network
/area kube-proxy
/area ipvs
/assign @honkiko
I'll send a fixing pull request tomorrow.

What happened:
On centos 7.2 with ipset 6.29, kube-proxy in ipvs mode failed to create ipset sets.

Jun 25 20:50:00 VM_3_4_centos kube-proxy[3828]: E0625 20:50:00.312569    3828 ipset.go:156] Failed to make sure ip set: &{{KUBE-LOOP-BACK hash:ip,port,ip inet 1024 65536 0-65535 Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose} map[] 0xc42073e1d0} exist, error: error creating ipset KUBE-LOOP-BACK, error: exit status 2

After some dig, it turns out to be problem of ipset on centos/rh. See https://bugzilla.redhat.com/show_bug.cgi?id=1496859

I tried ipset 6.29 on ubuntu, it's OK. But when I tried ipset 6.29 on centos, I got errors:

[root@VM_3_4_centos ~]# ipset create foo hash:ip comment
ipset v6.29: Unknown argument: `comment'
Try `ipset help' for more information.

What you expected to happen:
Since the problematic ipset versions have very large installation base, we could remove ipset commet, or remove ipset commet with problematic ipset version.

The problematic ipset also says it support comment by --help. The problem could be detected by a test run of "ipset create foo hash:ip comment".

How to reproduce it (as minimally and precisely as possible):
Create some service in kubernetes, run kube-proxy with ipvs mode on Centos 7.x node.

Anything else we need to know?:

Environment:

  • Kubernetes version (use kubectl version): commit 67e7d4c (actually all commits after 10664ee)
  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release): CentOS Linux release 7.2.1511 (Core)
  • Kernel (e.g. uname -a): Linux VM_3_4_centos 3.10.0-514.26.2.el7.x86_64 #1 SMP Tue Jul 4 15:04:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools:
  • Others: ipset v6.29, protocol version: 6
@stewart-yu

This comment has been minimized.

Show comment
Hide comment
@stewart-yu

stewart-yu Jun 26, 2018

Contributor

could you test it before commit 10664ee, that's ok? @honkiko

Contributor

stewart-yu commented Jun 26, 2018

could you test it before commit 10664ee, that's ok? @honkiko

@honkiko

This comment has been minimized.

Show comment
Hide comment
@honkiko

honkiko Jun 27, 2018

Contributor

@stewart-yu , yes, I did. It's OK. But I think the we don't need to revert the iptables comments.

Contributor

honkiko commented Jun 27, 2018

@stewart-yu , yes, I did. It's OK. But I think the we don't need to revert the iptables comments.

k8s-merge-robot added a commit that referenced this issue Jul 3, 2018

Merge pull request #65533 from honkiko/fix-ipset-fails-on-centos
Automatic merge from submit-queue (batch tested with PRs 65094, 65533, 63522, 65694, 65702). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix ipset creation fails on centos. issue 65461

**What this PR does / why we need it**:
remove usage of ipset comment extension because ipset versions on centos 7.x don't support comment yet.
See the issue #65461

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #65461

**Special notes for your reviewer**:
The comments for corresponding iptable rules are left untouched.

**Release note**:

```
NONE
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment