Service become unreachable after loadBalancerIP change with MetalLB in L2 mode #471
Checked few times and here the steps to reproduce:
Here is example, service worked well on IP 10.9.244.6 and then I changed loadBalancerIP to 10.9.244.7.
There are few ways to recover service access:
kube-proxy in iptables mode.
The text was updated successfully, but these errors were encountered:
Thanks for the report!
Okay, I see a logic bug in speaker, when an IP is assigned and then later changed. Basically the speaker internal state doesn't get cleaned up, so exported metrics are wrong and the speaker ends up doing a bit too much work for IPv6.
But... Even with that (which is definitely a bug!), the service should still be reachable on the new IP, based on the code I'm staring at...
Can you help me verify a few more things with your setup? I don't have time right now to set up a sandbox and repro (but should tomorrow).
Basically, looking at the code, I don't see how the new IP would not be working. There's definitely some stuff to fix here anyway, and with more steps (multiple services, moving IPs from one to another), there's maybe a way to trigger what you're seeing with just this logic bug... but if you don't see any ARP traffic with the clean steps above (that don't have borked internal speaker state), then there's something else going on.
@danderson thanks for reply.
Restarted all metallb pods, including controller.
Edited service and set LoadbalancerIP to 10.9.244.15. New IP assigned, according to logs and events.
For those that are hitting this issue, I did build a new speaker based off master with change #520 (fixed it for me, I don't know if there is any incompatibility between sepeaker built off master and controller base off latest release).
docker image name
How do I debug this further?
Sorry, I am not familiar how to debug such problem.
env: rancher 3.3.5, 1.17.2 (RKE) + , canel on bare metal 3x cs-24sc