New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kube-proxy v1.13.0 and 1.13.1 brokes services with externalIPs #72779

Open
nnz1024 opened this Issue Jan 10, 2019 · 2 comments

Comments

Projects
None yet
4 participants
@nnz1024
Copy link

nnz1024 commented Jan 10, 2019

/kind bug

Kubernetes version: 1.13.1.
kube-proxy mode: IPVS.
SDN: flannel.

We are used services like this:

---
apiVersion: v1
kind: Service
metadata:
  name: test
spec:
  type: NodePort
  externalIPs:
    - 192.168.10.201
  ports:
    - name: test-port
      protocol: TCP
      port: 80
      targetPort: test-port
  selector:
      app: test

where IP 192.168.10.201 is from our internal /24 network (in particular, our Kube nodes have IPs 192.168.10.1, 192.168.10.2 etc). Note that there is no conflicts between real IPs and virtual external ones.

Before updating from 1.12.3 to 1.13.1, it worked correctly. In partucular these IPs was accessible from other non-Kubernetes hosts from 192.168.10.0/24 network. After update kube-proxy to 1.13.1, this feature stops working: non-Kubernetes hosts cannot resolve service IPs via ARP:

# ip ne sh 192.168.10.201
192.168.10.201 dev eth1  FAILED

On Kubernetes hosts it still working, because not requires ARP resolving.

If we reset sysctl parameter arp_ignore on Kube hosts,

sysctl net.ipv4.conf.all.arp_ignore=0

all works normally even from non-Kube hosts.

Quck debugging shows that problem is in the commit 489e95b. Please make this behaviour opt-out. (Now we resets arp_ignore to 0 via cron every 1 min.)

@nnz1024

This comment has been minimized.

Copy link

nnz1024 commented Jan 10, 2019

/sig network

@k8s-ci-robot k8s-ci-robot added sig/network and removed needs-sig labels Jan 10, 2019

@liggitt liggitt added the area/ipvs label Jan 10, 2019

@Foxsa

This comment has been minimized.

Copy link

Foxsa commented Jan 11, 2019

Having the same problem on Kubernetes 1.13.2

btw Thanks for a workaround!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment