-
Notifications
You must be signed in to change notification settings - Fork 462
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
IPv6 support #307
Comments
@jpds honestly not much thought went in to supporting IPv6 at this point. But looks like at some point kube-router should start supporting it. If there is need from multiple users then cetainly support for IPv6 can be prioiritzed. |
+1 |
Please consider it. Ipvs, gobgb and ip6tables are supporting ipv6. I think the effort is smaller than one think since the commands are basically the same but with other addresses (don't know about the netlink though). |
https://github.com/vishvananda/netlink libray that is used by kube-router supports IPV6 as well. |
+1 - looking forward very much to ipv6 support in kube-router (we are on ipv6 only networks) |
There is backlog of issue currently for 0.2 release, We can work on this work 0.3 release. Meanwhile I am happy to accept any partial fixes towards IPv6 support. |
What is exactly missing / needed in kube-router to fully support IPv6? |
As a follow-up to issue #470 I've changed the address family temporarily and hit a few more issues/concerns/unexpected items, and figured I'd document them here FYI. First, I started with an existing cluster, that had bridge CNI plugin. The config called out a /72 pod subnet CIDR and the CNI configurations on nodes defined /72 subnets. When I started up kube-router, it created a new CNI config, that created a new bridge and /88 subnet, on each node. Should I have created the cluster w/o specifying the bridge CNI plugin (and static routes for pod networks to communicate with each node)? Second, it looks like the SyncNodeIPSets() has ipset call create with this /88 info and by default, ipset is doing IPv4. Looks like "family inet6" needs to be specified (thanks @murali-reddy). Here is the error:
Third, there is another error in kube-router, which combined with the previous error causes it to fail to start. The message is:
The netstat output shows some listening (on this host):
|
On the first issue, in syncNodeIPSets() I tried ensuring that the Create() call would specify that the IP family is IPv6. However, I found that the Create() method was not called (there is already a psSet and it was using ipv4. It then calls Refresh() with the IPv6 cluster CIDRs for the pod ets on each node, and complains of the incompatibility. Not sure where the initial sets are created, so that they can be made IPv6. P.S. I'll be on PTO for a week or so, but will resume investigation, when I get back. |
Here are some additional things that I see with the current kube-router code...
I've done some hacks to the code to force the above to use IPv6 equivalents. The current issue I have is that the GBP server does not start. For IPv4 mode, it seems like the local address list passed to the bgp server that is started has [ 10.192.0.2, ::], for IPv4 and IPv6. I see listening on the V4 address port 179, and listening on V6 any address for port 179 and two other ports (I have 3 nodes). I'm unsure how to set the local address list, when running in IPv6 (only) mode. I've done some guessing, without any success [fd00:20::2, fd00:20::2], [::, ::], [::1, ::1], [fd00:20::2, ::], [::, fd00:20::2], each saying port 179 is in use. If I try to use [0.0.0.0, fd00:20::2], I get a panic saying that the slice bounds is out o range. Can anyone explain how the local address list should be configured for use in IPv6 mode? Do I need to force an IPv4 address on the interface (it is IPv6 only right now)? |
I have not been able to figure out how to pass the right arguments for the local listen addresses to the go routine that starts goBGP, when operating in IPv6 mode. |
@pmichali ok let me give it a try. GoBGP folks also has a slack, https://slackin-gobgp.mybluemix.net they are pretty responsive. |
stay tuned. thanks to @uablrek IPv6 is taking good shape https://github.com/cloudnativelabs/kube-router/blob/master/docs/ipv6.md |
Bumping this up since IPv6 is now fully supported as of Kubernetes 1.16. We'll probably want to revisit this soon. |
Is it possible now to use ipv6 as an externalIP with kube-router? @murali-reddy |
@rearden-steel I belive no. Service proxy part is not yet compatible with IPv6. https://github.com/cloudnativelabs/kube-router/blob/master/docs/ipv6.md is the up to date status. CC: @uablrek |
+1 looking forward to this one |
BTW I switched from Kube-router to Cilium. |
@kvaps did you try ipv6 only in cilium? |
Well there is not much plugins which support IPv6, for now only Calico and Cilium, also Contiv. |
DANM also supports IPv6 very well. I hadn't had much luck with Cilium until 1.8. GoBGP itself supports IPv6 well. |
@kvaps Sorry, I did not mean to say "did you only test cilium", but I meant to say "did you test IPv6 only" in cilium :) Because what is of interest for me is IPv6 only k8s clusters without any IPv4 being involved. |
I have tested dual stack, but IPv6-only should be even simpler. |
@kvaps Last time I checked all the components had hardcoded IPv4 addresses for discovery, which won't work in IPv6 only environments |
Could someone summarize the state of ipv6 support? The docs says it's not supported, but those statements are quite old (~y2019). There's been quite a few ipv6 related PRs being merged in the past year or so hence I'm wondering to what extent it should be working already? Any references how people are using/testing kube-router with ipv6? |
+1 |
I tried it out and it breaks quite quickly:
with the setup where each node peers with a ToR router: args:
- --run-router=true
- --run-firewall=false
- --run-service-proxy=false
- --bgp-graceful-restart=true
- --cluster-asn=4288000094
- --peer-router-asns=64515
- --peer-router-ips=2001:19f0:ffff::1
- --peer-router-passwords=xxxxxxx
- --router-id=2001:19f0:ffff::1
- --metrics-port=8080 |
when it's an ipv6 address the format should be |
I added a few more fixes: #1249 |
What is the current state of IPv6 in kube-router? Is it possible to try using it an IPv6 only environments? |
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
IPv6 should be moving to beta in 1.10 and is in alpha in 1.9 - does kube-router have any integration with the protocol version?
The text was updated successfully, but these errors were encountered: