FRR not support configures security associations in the SA/SP database automatically on BSD (FreeBSD)

[pautiina]

Hello.
On BSD systems, keys for TCP MD5 authentication are stored in the global SA/SP database, which can be accessed by external utilities (e.g. setkey(8))

On FreeBSD kernel must add next:

options         IPSEC                   # IP (v4/v6) security
options         IPSEC_SUPPORT           # Allow kldload of ipsec and tcpmd5
# The crypto framework is required by IPSEC
device          crypto                  # Required by IPSEC
device          cryptodev
options        TCP_SIGNATURE

setup ipsec key:

cat /etc/ipsec.conf
flush;
# For IPv4
add x.x.x.x y.y.y.y tcp 0x1000 -A tcp-md5 "some_password";
add y.y.y.y x.x.x.x tcp 0x1000 -A tcp-md5 "some_password";
# For IPv4
add XX:XX:XX::XX YY:YY:YY::YY tcp 0x1000 -A tcp-md5 "some_password";
add YY:YY:YY::YY XX:XX:XX::XX  tcp 0x1000 -A tcp-md5 "some_password";

Sutup of the IPSEC keys must be done directly through FRR without change /etc/ipsec.conf file.

[pautiina]

The problem is solved.
A full description of the problem and its solution is described here: Bug 223835 - BGP session not established with md5 password via FRRouting
Many thanks to Marek Zarychta mailto:zarychtam@plan-b.pwste.edu.pl for the help.
He showed me a similar problem: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219453.
Аnd indeed, the problem was that on the interface TX/RX checksums was disabled.

Now everything works with the following settings:

On interface ix0 or ixl0 or other must be turn ON: rxcsum txcsum; (ifconfig ixl0 rxcsum txcsum)

At /etc/rc.conf:

ifconfig_ixl0="up -tso -lro -vlanhwtso" #(I disabled only tso and lro)
ipsec_enable="YES"
ipsec_file="/etc/ipsec.conf"

At /etc/ipsec.conf:
flush;
add 185.1.62.241 185.1.62.69 tcp 0x1000 -A tcp-md5 "some_password";
add 185.1.62.69 185.1.62.241 tcp 0x1001 -A tcp-md5 "some_password";

On kernel you must add next:

options         IPSEC                   # IP (v4/v6) security
options         IPSEC_SUPPORT           # Allow kldload of ipsec and tcpmd5
# The crypto framework is required by IPSEC
device          crypto                  # Required by IPSEC
device          cryptodev
options         TCP_SIGNATURE

And need set password for neighbor on FRRouting, for example:
neighbor 185.1.62.69 password some_password

I think it's necessary to describe all this in wiki.
This would be good, as this problem arises for many or you can simply forget about it :)

[pautiina]

Maybe it's bug BGP session not established with md5 password via FRRouting

[donaldsharp]

@pautiina - I've quickly skimmed the included link and am a bit confused about what I should be focusing on to understand the bug. Can you help me out here? Why do you think this is a bug still?

[pautiina]

I'm not sure if this is a bug. Maybe just for this faket must be mentioned in the documentation.

[pautiina]

At first I thought it was a bug of the FRR, so I wrote it here.
On FreeBSD found that this is a bug drivers and network card is an issue that the TCP MD5 signatures is not validated and is not set until the network interface will not set TX/RXcsum options.
Concerning the frr, I think it is necessary to add support for SA / SP key control, as it was done in the bird for FreeBSD.
If I am not right about managing the SA / SP key database, please close the issue.

[donaldsharp]

@rwestphal to look into understanding this issue better and see if FRR needs to do anything

[pguibert6WIND]

from http://bird.network.cz/?get_doc&f=bird-6.html
On BSD systems, keys for TCP MD5 authentication are stored in the global SA/SP database, which can be accessed by external utilities (e.g. setkey(8)).
I guess this should be a store that BGP could get directly from it.

[pautiina]

@pguibert6WIND you're right.
It would be great if the FRR managed keys for TCP MD5 authentication on BSD system.
I think this issue should be attributed to the enhancement.

[pautiina]

Hello. What about enchantment for this issue?

[neirbowj]

For the benefit of others who find this issue while troubleshooting TCP-MD5 on FreeBSD, if you encounter "% Error while applying TCP-Sig to session(s)", then load the tcpmd5 kernel module (sysrc kld_list+=" tcpmd5", kldload tcpmd5).

Now, hopefully to elucidate this issue.

Per tcp(4) on TCP_MD5SIG (FreeBSD 12.1R): "In order for this option to function correctly, it is necessary for the administrator to add a tcp-md5 key entry to the system's security associations database (SADB) using the setkey(8) utility. This entry can only be specified on a per-host basis at this time.

If an SADB entry cannot be found for the destination, the system does not send any outgoing segments and drops any inbound segments."

It turns out that, unlike the IPsec protocols that keep credentials in the SAD and policy in the SPD, TCP-MD5 keeps credentials in the SAD and policy on the socket itself. As of FRR 7.3, it does not matter what password is set in bgpd.conf, because when bgpd enables TCP-MD5 that only has the effect of establishing policy, not storing credentials.

I tested with two VNET jails connected by an epair interface. The relevant FRR config on each is:

hostname prior.example.com
!
router-id 192.0.2.1
!
router bgp 65100
 neighbor 192.0.2.2 remote-as 65200
 neighbor 192.0.2.2 password aaaaa
 neighbor 2001:db8:f00::2 remote-as 65200
 neighbor 2001:db8:f00::2 password bbbbb
 !
 address-family ipv4 unicast
  network 198.51.100.0/24
  no neighbor 2001:db8:f00::2 activate
 exit-address-family
 !
 address-family ipv6 unicast
  network 2001:db8:1000::/64
  neighbor 2001:db8:f00::2 activate
  neighbor 2001:db8:f00::2 soft-reconfiguration inbound
 exit-address-family
!

hostname wilder.example.com
!
router-id 192.0.2.2
!
router bgp 65200
 neighbor 192.0.2.1 remote-as 65100
 neighbor 192.0.2.1 password ccccc
 neighbor 2001:db8:f00::1 remote-as 65100
 neighbor 2001:db8:f00::1 password ddddd
 !
 address-family ipv4 unicast
  network 203.0.113.0/24
  no neighbor 2001:db8:f00::1 activate
 exit-address-family
 !
 address-family ipv6 unicast
  network 2001:db8:2000::/64
  neighbor 2001:db8:f00::1 activate
  neighbor 2001:db8:f00::1 soft-reconfiguration inbound
 exit-address-family
!

Before installing the SAD entries, neither node emits any TCP traffic per the manpage description.

Both nodes use the same SAD config.

#!/sbin/setkey -f

flush;

add 192.0.2.1 192.0.2.2 tcp 0x1000 -A tcp-md5 "swordfish" ;
add 192.0.2.2 192.0.2.1 tcp 0x1000 -A tcp-md5 "swordfish" ;

add 2001:db8:f00::1 2001:db8:f00::2 tcp 0x1000 -A tcp-md5 "swordfish" ;
add 2001:db8:f00::2 2001:db8:f00::1 tcp 0x1000 -A tcp-md5 "swordfish" ;

The SPI does not seem to matter. Both BGP sessions come up and tcpdump -M swordfish -i epair0a confirms that TCP-MD5 is active.

In summary, I think the desired enhancement to FRR would be to implement management of the SAD entries, presumably via the PF_KEY interface.