-
Notifications
You must be signed in to change notification settings - Fork 910
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
IP-Masquerading for outgoing BGP traffic #2355
Comments
Whoa, this is pretty creative!
I think you could use the srcAddress field of the bgpPeer. This gets translated to the The downside is, you have to do some node selector gymnastic to have exactly one peer per node.
|
Wow, that was a quick response! :D Thank you! I've already considered the Do you know if the And for the bonus question: How is the L2 address actually bound to the node, and how can this be confirmed on the node itself without relying on inspecting the speaker logs? |
It should, although iirc we don't have e2e tests covering it at the moment. Also, the check of the existance of the ip on a node is related to the native bgp implementation, not the frr one (but I think is necessary). In any case should be trivial to test for one node (and to add the vip to the lo interface of the node)
MetalLB doesn't assign the ip to any interfaces of the node. But if you wait, #2351 should land soon and with that the support to expose the node exposing the VIP as a status CR. |
thanks, i am going to test that. What i found out in the meantime, in case somebody cares, what also works (but would be a hassle to maintain, is to add
and the other gets
more of a proof of concept but i will report back in case i find any suitable (kubernetes native) solution for that problem. i am honestly pretty surprised that nobody else seems to have this kind of use-case. |
I added some label-magic and metallb-config so that the L2-addresses for the peers (.6 and .7 ip address) are bound two two fixed nodes and also added my config looks likes this:
but metallb complains:
Have I misunderstood something here or is this the end of my creative solution? |
Talking (at least) about the asn and the src address: if we don't allow different properties on different nodes for the same peer, I don't see the point of having those fields. On the other hand, relying on the label selector is fragile, because even though we can check if those two peers (same ip, different src address) are not conflicting because they apply to different nodes today, a new node might come in and match both label selectors. I am keen to say we must rethink the api, either allowing a per node configuration by adding a node name field to the bgppeer, or having another structure that contains the per node configuration of a given frr instance. cc @oribon to open the discussion. |
@Zappelphilipp fwiw, I filed #2367 . After discussing offline with @oribon , we think it'd make sense to allow the configuration. I am closing this as I think we can refer to the other issue to track the implementation (which will happen on a best effort basis). |
Is your feature request related to a problem?
Running Kubernetes nodes with dynamic/changing VM IP's (as in Rancher, where nodes are regularly redeployed during updates, etc.) results in non-deterministic peering partner IP setups on the router. Apart from the Cisco-specific "BGP Peer Group" feature (which allows a whole subnet to be set as potential BGP peers), there is currently no known clean solution for using BGP with dynamic IP's.
Describe the solution you'd like
I have partially found a solution to the problem: MetalLB allows running in BGP and L2 mode simultaneously. So, I set up one or two virtual IP's in L2-Mode, which bind to the speaker-service of the MetalLB instance. These two IP's are then configured on the router as BGP peers. Thus, traffic flow from Firewall to BGP to MetalLB would consistently go through these two fixed virtual IP's in a direct way to the speaker-pods (which in this case are 2 on 2 worker nodes)
The second problem arises with all outgoing peering traffic:
MetalLB --> BGP --> Router
This traffic always uses the source IP's of the Kubernetes nodes, which change regularly. It would be beneficial if the source IP of the BGP traffic could be masqueraded to statically defined IP's. These would be the same two peering addresses used in L2-Mode for incoming BGP traffic.
Additional context
No response
I've read and agree with the following
The text was updated successfully, but these errors were encountered: