BGP: Routes not learned from Bird with IPv6 link-local peering

[jeremyvisser]

When I peer an FRR and Bird host over BGP with IPv6 link-local addresses (no global addresses assigned to either interface), FRR cannot learn routes from Bird.

When peering an FRR and FRR together, it works fine.

Here's minimal config which can reproduce the problem:

interface peer-bird-2222
  ipv6 address fe80::1111/64

interface peer-frr-3333
  ipv6 address fe80::1111/64

router bgp 65111
 neighbor fe80::2222 description Bird
 neighbor fe80::3333 description FRR

 address-family ipv6 unicast
  network fd11:1111::/32

  neighbor fe80::2222 interface peer-bird-2222
  neighbor fe80::2222 activate
  neighbor fe80::2222 soft-reconfiguration inbound

  neighbor fe80::3333 interface peer-bird-2222
  neighbor fe80::3333 activate
  neighbor fe80::3333 soft-reconfiguration inbound
 exit-address-family

If I inspect the received-routes from Bird, I see incoming routes:

# show bgp neighbors fe80::2222 received-routes
   Network          Next Hop            Metric LocPrf Weight Path
*> fd22:2222::/32   ::                                    10 65222 i

But the nexthop is unexpectedly ::. I would expect the nexthop to be fe80::2222.

Understandably, there are no routes learned:

# show bgp neighbors fe80::2222 routes
<not a sausage>

If I inspect the UPDATE messages I receive from the Bird host, the nexthop information in the MP_REACH_NLRI is:

Next hop network address (32 bytes)
    Next Hop: ::
    Next Hop: fe80::2222

It sounds like FRR is getting confused about the :: nexthop, and should instead be trying to use the fe80::2222 nexthop.

If I look at the FRR to FRR peer instead, everything works as expected:

# show bgp neigh fe80::3333 routes
   Network          Next Hop            Metric LocPrf Weight Path
*> fd33:3333::/32   fe80::3333               0             0 65333 i

And in the UPDATE messages from the FRR host, the MP_REACH_NLRI has a different set of nexthops:

Next hop network address (32 bytes)
    Next Hop: fe80::3333
    Next Hop: fe80::3333

The route is installed correctly, and all works.

Expected behaviour

I expected an IPv6 link-local BGP peering (with no global address present on either interface) to work. However, with a Bird peer, FRR gets confused about which nexthop to use.

I believe it should ignore the :: nexthop and try the link-local address instead.

Possible workaround

I theorised that I could override the nexthop by using a route-map:

(config)# route-map BIRD-IN permit 10
(config-route-map)# set ipv6 next-hop global fe80::2222
% Invalid global nexthop address

This is expected behaviour. A link-local address is not a valid global nexthop.

Therefore, the MP_REACH_NLRI that FRR is sending by default...

Next hop network address (32 bytes)
    Next Hop: fe80::3333
    Next Hop: fe80::3333

...is doing what the route-map is forbidding! FRR is being fairly hypocritical here.

(For what it's worth, set ipv6 next-hop peer-address or set ipv6 next-hop local fe80::2222 are not effective as workarounds; they don't affect the nexthop that FRR learns.)

Versions

• OS: Fedora 31 kernel (Linux 5.5.13-200.fc31.x86_64); Fedora 32 userspace
• FRR Version frr-7.3-1.fc32.x86_64

[ton31337]

@jeremyvisser could you check with the latest master?

[jeremyvisser]

Thanks for that! I tested with latest master last night. I can confirm that incoming routes are now successfully being learnt from Bird as expected!

Now, as I originally wrote the bug report, 0355b41 solves it.

However, what I additionally noticed was that Bird is also not learning routes from FRR, due to what seems to be a similar reason.

Bird is showing this log entry:

bird[7402]: frrpeer: Invalid NEXT_HOP attribute

I suspect this is a result of the MP_REACH_NLRI nexthop contents:

Next hop network address (32 bytes)
    Next Hop: fe80::3333
    Next Hop: fe80::3333

Bird is probably trying to interpret the first address as global, and the second as local, but is rejecting the route because fe80::3333 isn't a valid global address (understandably so).

My guess is that sending this instead might work (again, just to clarify, I have no global address configured on this network interface):

Next hop network address (32 bytes)
    Next Hop: ::
    Next Hop: fe80::3333

[Napsterbater]

Well what frr receives from the other side should be able to work in both ways, which seems like maybe that is the case now with the latest change.

What FRR sends might be best set as a knob on a per peer basis maybe.

Like "neighbor next-hop-omit-v6-global" and this would be under the IPv6 AFI ofcourse.
Or something to that effect, just an example.

Because apparently there is ambiguity in the standard in regards to the situation and it might be best keep the current behavior and give the option when needed for alternative behavior. At least until the spec is clarified.

Just my two cents as a user of FRR.

P.s. here is a draft that is currently expired that explains an even different way handling IPv6 link local bgp peering, that might come up again.
https://tools.ietf.org/id/draft-white-linklocalnh-00.html

[ton31337]

@Napsterbater this draft is in progress to be updated, we agreed on some updates there, I hope the next revision will be published soon.

[ton31337]

@polychaeta autoclose in 1 day.

[chriselsen]

I'm still facing an issue with this on frr-7.4-dev-465-g3dab0aea0.

I'm also using IPv6 link-local peering and all learned routes are marked as invalid:

dn42-us-pdx1(config-router-af)# do sh ip bgp ipv6 neighbors fe80::1588 prefix-counts
Prefix counts for fe80::1588, IPv6 Unicast
PfxCt: 281

Counts from RIB table walk:

              Adj-in: 282
              Damped: 0
             Removed: 0
             History: 0
               Stale: 0
               Valid: 0
             All RIB: 281
       PfxCt counted: 281
             Useable: 281

As the other side is not setting a global address but :: instead, it seems like frr is choking on that:

dn42-us-pdx1# show ip bgp nexthop
Current BGP nexthop cache:
 172.20.16.140 valid [IGP metric 0], #paths 488, peer 172.20.16.140
 :: invalid
  Must be Connected
  Is not Registered
  Last update: Wed May 20 16:23:23 2020

 fe80::1588 valid [IGP metric 0], #paths 0, peer fe80::1588