FRR/BGP + CARP/VIP

[ciroiriarte]

This issue has been automatically timed-out (after 180 days of inactivity).

For more information about the policies for this repository,
please read https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md for further details.

If someone wants to step up and work on this issue,
just let us know, so we can reopen the issue and assign an owner to it.

Originally posted by @OPNsense-bot in #2106 (comment)

[ciroiriarte]

Hello!, the request timeout but my understading is this improvement is needed.

Usually you would run active/active BGP peering but that the asymetric traffic can break some types of connectivity in a OPNSense CARP setup (VPN landing on the active master, same for local gateways).

FRR plugin already allows integration with CARP where it starts on the master only, and starts on the slave as soon as we switch roles or the current master fails. That requires that we setup two sessions (one per node) and the other party will see one session always down, which is usually unacceptable (it triggers unexpected alarms when a "green dashboard" is desired).

The "solution" (give an ECMP cluster without any other traffic breaking appart is not possible), would be to keep the "run on master only" but drive all the traffic with the CARP IP (outgoing & incoming), not the base/static IP of the node. This way the other party will see an "always stablished" session and not two sessions (one dead, the other active).

Also, I'm not sure if it's too much to ask for this integration, but usually in common routing environments, it's expected to be able to use multiple interaces (multiples CARP IPs in this case?) for different integrations: one interface going to network party A, a second one to network party B, etc.

For my usecase, I need BGP + BFD, but I assume the same would apply for OSPF.

[mimugmail]

Did you read my question in the other issue and tried to reproduce successfully?

[OPNsense-bot]

Thank you for creating an issue.
Since the ticket doesn't seem to be using one of our templates, we're marking this issue as low priority until further notice.

For more information about the policies for this repository,
please read https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md for further details.

The easiest option to gain traction is to close this ticket and open a new one using one of our templates.

[ciroiriarte]

Hello, I did the test.

The thing is that it still seems to initiate connection from the base IP.

root@vfw01:~ # grep bgpd_flags /etc/rc.conf.d/frr
bgpd_flags="${bgpd_flags} -M snmp -l 172.16.10.1"

root@vfw01:~ # ps auxw|grep  bgpd
frr          65770   0.0  0.7   55960 28080  -  Ss   13:41       0:00.07 /usr/local/sbin/bgpd -M snmp -l 172.16.10.1 -d
root         99077   0.0  0.1 1061152  3232  1  S+   13:43       0:00.00 grep bgpd

root@vfw01:~ # sockstat | grep bgp
frr      bgpd       65770 4  dgram  -> /var/run/logpriv
frr      bgpd       65770 18 stream -> /var/run/frr/zserv.api
frr      bgpd       65770 21 tcp6   *:2605                *:*
frr      bgpd       65770 22 tcp4   *:2605                *:*
frr      bgpd       65770 23 stream /var/run/frr/bgpd.vty
frr      bgpd       65770 24 stream -> /var/run/frr/zserv.api
frr      bgpd       65770 25 tcp4   172.16.10.2:41479     172.16.10.5:179
frr      bgpd       65770 26 tcp4   172.16.10.1:179       *:*
frr      bgpd       65770 27 tcp4   172.16.10.2:45163     172.16.10.6:179

As you can see, the process is listening on 172.16.10.1:179, but the connection to the remote 172.16.10.5:179 & 172.16.10.6:179 is initiated from the base/fixed IP (172.160.10.2).

[mimugmail]

Can you try outbound Nat?
Source WAN address, destination BGP peer, translated VIP

[ciroiriarte]

Well, checking with the other team, they kept both the base IP & the CARP IP. After dropping all the base IPs and keeping only the VIP, the session could be initiated (initiated by the other party):

root@vfw01:~ # sockstat | grep bgp
frr      bgpd       65770 4  dgram  -> /var/run/logpriv
frr      bgpd       65770 18 stream -> /var/run/frr/zserv.api
frr      bgpd       65770 21 tcp6   *:2605                *:*
frr      bgpd       65770 22 tcp4   *:2605                *:*
frr      bgpd       65770 23 stream /var/run/frr/bgpd.vty
frr      bgpd       65770 24 stream -> /var/run/frr/zserv.api
frr      bgpd       65770 25 tcp4   172.16.10.1:179       172.16.10.6:35037
frr      bgpd       65770 26 tcp4   172.16.10.1:179       *:*
frr      bgpd       65770 27 tcp4   172.16.10.1:179       172.16.10.5:42527

Now, also after adding the outgoing NAT, the session could be initiated from my (OPNSense) side:

root@vfw01:~ # sockstat | grep bgp
frr      bgpd       85143 4  dgram  -> /var/run/logpriv
frr      bgpd       85143 18 stream -> /var/run/frr/zserv.api
frr      bgpd       85143 21 tcp6   *:2605                *:*
frr      bgpd       85143 22 tcp4   *:2605                *:*
frr      bgpd       85143 23 stream /var/run/frr/bgpd.vty
frr      bgpd       85143 24 stream -> /var/run/frr/zserv.api
frr      bgpd       85143 25 tcp4   172.16.10.2:61223     172.16.10.5:179
frr      bgpd       85143 26 tcp6   *:179                 *:*
frr      bgpd       85143 27 tcp4   *:179                 *:*
frr      bgpd       85143 28 tcp4   172.16.10.2:22466     172.16.10.6:179

Is my understanding that we don't really need the "bind" parameter, since this can be accomodated with firewall rules + NAT:
1- Keep default of listening in all interfaces
2- Blocking undesired traffic to the base/fixed IP
3- Only allowing incomming connections to the VIP
4- Setting up outgoing NAT to the remote routers?

Is this correct?

[mimugmail]

Yes, I dont think there is a stable way to so auto nat when adding a checkbox and not breaking other inteded dual peer setups