Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

radvd: Implement AdvRASrcAddress option #5185

Closed
wants to merge 2 commits into from

Conversation

vnxme
Copy link
Contributor

@vnxme vnxme commented Aug 29, 2021

Idea:

  • Make it possible to send RA from any link-local IPv6 address (e.g. fe80::1).

Issues:

Solution:

For discussion:

  • VIPv6 may technically be LL, ULA or GUA, but only LL addresses seem to work correctly for me. E.g. if I put a GUA in AdvRASrcAddress, radvd starts normally and sends RAs, but a sample client (MacOS 11) shows an empty IPv6 gateway. That's why I suggest we limit this option to fe80::/64 addresses, this way I have encountered no issues so far.

Make it possible to send RA from any link-local IPv6 address (e.g. fe80::1)
@fichtner
Copy link
Member

To my knowledge:

a) CARP support for Router Advertisements works
b) CARP is not allowed to actively send from the address, only respond

What am I missing?

@vnxme
Copy link
Contributor Author

vnxme commented Aug 29, 2021

@fichtner Thank you for entering this discussion.

a) CARP support for Router Advertisements works

I'm not sure I understand what you mean. When you set RA interface to anything other than dynamic or static, dhcpd.inc changes DeprecatePrefix and RemoveRoute options while the daemon keeps sending RAs from the first link-local address.

b) CARP is not allowed to actively send from the address, only respond

Do you have any proofing materials describing this behaviour? CARP seems to be similar to VRRP in Linux, and radvd + VRRP solution has already been described, e.g. this article. A few tests I have made so far in OPNsense didn't reveal any problems with sending RAs from CARP - radvdump on other machines and iPhone/MacOS interfaces correctly report fe80::1 gateway.

@fichtner
Copy link
Member

Well for both things see e.g. https://www.netbsd.org/docs/guide/en/chap-carp.html

CARP's design does require that the members of a group be on the same physical subnet with a static IP address [...].

That's why the CARP mode (RA interface set) does not deprecate prefixes or routes, also because:

Similarly, services that require a constant connection to the server (such as SSH or IRC) will not be transparently transferred to the other system--though in this case, CARP can help with minimizing downtime.

This is why we need to switch routers in this case for IPv6 too. GUA with prefix delegation can be a problem, but it's not clear from the description so I assume that these issues are known and properly set up beforehand.

@fichtner
Copy link
Member

The conclusion of this thread is also relatively easy to process: https://forum.netgate.com/topic/133306/ipv6-carp-vip-with-route-advertisements/8

@fichtner
Copy link
Member

What I don't quite understand, why not disable "Advertise Default Gateway" and add a custom default route through your link local address, e.g. fe80::111/0 ?

@vnxme
Copy link
Contributor Author

vnxme commented Aug 29, 2021

@fichtner Thanks, I will take some time to read through the materials.

As far as I can see, CARP is only one possible option here. What if I just want to send RAs from fe80::1 (for whatever reason which is beyond this discussion) which is my static IP alias (i.e. not CARP)? This is what I'm talking about in point (A) of #4953 - RA interface list should include all types of VIPv6, not only CARP ones.

@fichtner
Copy link
Member

fichtner commented Aug 29, 2021

radvd.conf is also interesting for AdvRASrcAddress, where it says:

This functionality will NOT spoof the source address

Which is part of the earlier point. It's a little magical the way it is described, but at this point it's unclear if setting an explicit AdvRASrcAddress that is not really a source address but just fudges the advertisement's default route to e.g. a CARP IP so it looks seamless is the required approach. In that case in particular we should, however, not start making this manually configurable as it's easy to push the CARP IP into the config and avoid setup problems.

Cheers,
Franco

@fichtner fichtner self-assigned this Aug 29, 2021
Copy link
Member

@fichtner fichtner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we need to use AdvRASrcAddress in CARP scenarios we must set it automatically.

@vnxme
Copy link
Contributor Author

vnxme commented Aug 29, 2021

What I don't quite understand, why not disable "Advertise Default Gateway" and add a custom default route through your link local address, e.g. fe80::111/0 ?

The "Advertise Default Gateway" switch is definitely on the Router Advertisements page. One can add more routes to advertise on the same page, but no gateways can be specified there, i.e. subnets are routed through the link-local address radvd sends RAs from. Where can I set a custom default route the way you suggest so that clients could get fe80::1 as their default route gateway?

@fichtner
Copy link
Member

fichtner commented Aug 29, 2021 via email

@vnxme
Copy link
Contributor Author

vnxme commented Aug 29, 2021

@fichtner I think I don't fully understand you again.

I can't see any decent way to add a default route for ipv6 via a custom link-local address in OPNsense now (say, host A advertises a default route via host B). But if I create a new custom link-local VIPv6 (not CARP), manually add AdvRASrcAddress to radvd.conf and manually restart the daemon, I get exactly what I want - automatic custom default gateway on clients (same host A, but different, more attractive address is used).

With this PR I can do the same with a new custom link-local CARP VIPv6, RAs are sent normally and clients configure their interfaces automatically with the chosen custom default gateway.

So, let me summarize my point:

  • AdvRASrcAddress is an integral option in order to make clients establish default routes to a custom gateway (still limited to the same host's link-local address, unless you know another way - please share);
  • This option should be probably implemented without direct link to CARP functionality though it technically works for me even on CARP VIPv6.

@fichtner
Copy link
Member

fichtner commented Aug 29, 2021

It’s clearer now after discussing it, but I’m not the one having brought up CARP in the first place. I don’t see a general issue here that makes AdvRASrcAddress a bare necessity.

Also, you are wrong about it: radvd.conf can set AdvRASrcAddress per route. Further evidence that this is a strange fix…

last but not least the radvd.conf manual itself mentions VRRP (CARP) for AdvRASrcAddress so that still points to special use case.

@fichtner
Copy link
Member

fichtner commented Aug 29, 2021

As for the route/AdvRASrcAddress combination that was a misconception by me after reading radvd.conf manual page which doesn't start a new section after configuration entries making it easier to misread.

@vnxme
Copy link
Contributor Author

vnxme commented Aug 29, 2021

I looked through the links you provided earlier and my thoughts are as follows:

  • I have found no mentions of 'CARP being not allowed to actively send from the address';
  • I have found no results of discussion at Netgate's forum impeding us from implementing AdvRASrcAddress option.

Please, don't hesitate to correct me if I missed the most important bits of information.

What I actually found is Netgate merged more or less the same logic of AdvRASrcAddress earlier this year. Their users reported the solution to be working as expected.

My idea is quite simple: if the user changes RA interface from dynamic/static to any VIP (be it simple alias or CARP), one expects radvd to send RAs from it. The daemon allows to do it, so why don't we make it possible in OPNsense?

@fichtner
Copy link
Member

My idea is quite simple: if the user changes RA interface from dynamic/static to any VIP (be it simple alias or CARP), one expects radvd to send RAs from it. The daemon allows to do it, so why don't we make it possible in OPNsense?

I think we want the same thing, but can't agree on a format: I don't think we should make AdvRASrcAddress accessible in the GUI. Otherwise we need docs and community support and countless questions follow how to set up something "so simple". 99% of those things will not know or care about AdvRASrcAddress.

As for technical details of CARP they really do not matter the way radvd seems to be wired to work so feel free to discard any knowledge gaps. As radvd.conf manual page states spoofing does not occur and that includes the CARP address or any bound virtual IP, but feel free to verify the opposite with wireshark.

@vnxme
Copy link
Contributor Author

vnxme commented Aug 29, 2021

I think we want the same thing, but can't agree on a format: I don't think we should make AdvRASrcAddress accessible in the GUI. Otherwise we need docs and community support and countless questions follow how to set up something "so simple". 99% of those things will not know or care about AdvRASrcAddress.

Great, now we are on the same side. This PR actually doesn't expose this option to the GUI. It just makes the already existing RA interface drop-down list more predictable/less confusing. Dynamic/static options require no AdvRASrcAddress at all. The remaining RA interface options (which I believe most users won't even have) are VIPs, and that's the point when our backend code should put AdvRASrcAddress to radvd.conf automatically.

Would you like to propose another logic?

fichtner added a commit that referenced this pull request Aug 29, 2021
@fichtner
Copy link
Member

Well, first we need to get rid of calling get_interface_ipv6() by hand-rolling the use case to allow for grabbing the ipalias as well as the carp address that is there using the first practical link-local ... unverified commit f11aa507acf -- it may need falling back to the carp address as well.

Afterwards you can use this to set AdvRASrcAddress in the particular cases. Thoughts?

@pv2b
Copy link
Contributor

pv2b commented Sep 21, 2021

Hi!

Great to see work on this, I am just independently going down this same rabbit hole again. I also came to the conclusion that if I want "IPv4-like" working failover with a CARP cluster, I have to set up a fe80::1 CARP VIP on my LAN and then somehow get radvd to use this LL IPv6 address to advertise.

It has to be a link-local address, as per RFC4861 section 4.2:

  Source Address
                 MUST be the link-local address assigned to the
                 interface from which this message is sent.

As far as I can tell, I don't think there's any harm in just always specifying AdvRASrcAddress in any case where a link-local address is explicitly configured, no matter if it's CARP, or some other type of VIP/IP Alias. You probably don't want it always set for the typical (non-CARP) case where a link-local address is dynamically assigned, but that's all. You probably also want to include a filter so that the IPv6 address has to be in fe80::/64 (or maybe fe80::/10, not sure on this one) to avoid non-compliant configurations.

As for the UX in this case, it would involve the user knowing they must set up a link local CARP VIP when setting up gateway redundancy with IPv6, and then select this interface when configuring router advertisements. The AdvRASrcAddress would not be directly exposed in the GUI.

@pv2b
Copy link
Contributor

pv2b commented Sep 21, 2021

I should also mentioned that I just tested the patch above, and at least for the purposes of getting LAN gateway redundancy with CARP to work, this doesn't appear to be sufficient. The MAC address learnt through ND will still be the MAC of the individual router sending out the RA which doesn't provide the desired failover behaviour.

For this kind of scenario you probably also need to add this option to the interface config in radvd.conf:

AdvSourceLLAddress off;

This way the MAC address of the router is not learnt through the RA (which would be the ethernet MAC of that particular router), and instead is learnt through ND, which hopefully would be the CARP MAC. However... in my own testing I've not quite got this to work. My OPNsense boxes don't respond to these ND requests, but that's an unrelated issue, so in the interests of not polluting this PR with this unrelated trouble I'll omit the details. Edit: This works after a reboot of the firewalls so it's probably just an unrelated issue. Now it works correctly, as in the IPv6 packets are addressed to the CARP MAC on the L2 layer.

@vnxme
Copy link
Contributor Author

vnxme commented Sep 28, 2021

If we need to use AdvRASrcAddress in CARP scenarios we must set it automatically.

@fichtner Hi, Franco! Sorry for my late reply. I would like to continue our work on this PR. Could you please elaborate on your comment where you say that we must set it automatically?

If the user adds a link-local IPv6 CARP, one may or may not intend to use the IPv6 RA high-availability feature radvd provides with AdvRASrcAddress. Besides, the user may also add the second/third/etc. link-local IPv6 CARPs, and then we are unable to decide which one to use as RA source.

@bimbar
Copy link

bimbar commented Oct 16, 2021

NM, just checked your commit and it seems perfectly fine to me, that's just what we need.

@bimbar
Copy link

bimbar commented Oct 16, 2021

I just copied that over to my firewalls, because I want that functionality, found a little problem:

interface igb1 {
	AdvSendAdvert on;
	MinRtrAdvInterval 200;
	MaxRtrAdvInterval 600;
	AdvLinkMTU 1500;
	AdvDefaultPreference medium;
	AdvRASrcAddress {
		fe80::1;
	};
	prefix x.y.z.a::/64 {
		DeprecatePrefix off;
		AdvOnLink on;
		AdvAutonomous on;
	};
	prefix fe80::/64 {
		DeprecatePrefix off;
		AdvOnLink on;
		AdvAutonomous on;
	};

It started to advertise fe80::/64, too, which probably doesn't hurt, but shouldn't be done anyway. This is probably something that existed before, but there was little need for link-local CARP addresses then.

@fichtner
Copy link
Member

@bimbar can you check 6970f4c for the link local advertise issue

@fichtner
Copy link
Member

We discussed this at the office a bit. The new proposal is 2c247d9 + f18d0b0, so

# opnsense-patch 2c247d978754 f18d0b0bf719

You do need to set up a link-local virtual IPv6 for CARP to work, manual setup (in case of tracking) and select that CARP address in the router advertisement configuration page.

Cheers,
Franco

@vnxme
Copy link
Contributor Author

vnxme commented Oct 22, 2021

@vnxme as far as I understand it can't be a pure alias as that doesn't serve any meaningful purpose? maybe I'm missing something

I would like to provide a use case for each VIP type (pure CARP, pure alias, mixed):

  1. Pure CARP. You have 2 routers and want an IPv6 failover setup, so configure fe80::1/64 CARP VIPs with the same vhid on both routers.
  2. Pure alias. You have 1 router, but want clients to see and use a short IPv6 as their default gateway, so you configure one fe80::1/64 alias VIP on the router. No multicast traffic is sent in this case, by the way, which is an advatage to using a similar CARP instead.
  3. Mixed alias/CARP. Same as 1, but you also want IPv4 failover setup on the same vhid, so you add two CARP VIPv4 and two alias VIPv6 with the same vhid on both routers. Using the same vhid for a dual stack failover setup you save vhids and decrease multicast traffic.

@fichtner
Copy link
Member

@pv2b I'll add a comment to the block and will work on this further to clean up the style of course

@vnxme fair enough, although we could argue that case 2 is more or less the feature request to set an interface link-local address manually instead of adding an alias in which case this is not a VIP relation and works by default. It's been on my wish list forever, but also not worth the amount of work. Case 2 is certainly easier to reach... Let me think about it. :)

@pv2b
Copy link
Contributor

pv2b commented Oct 22, 2021

I identified another small problem in the codebase, the current is_linklocal function only detects a subset of valid link-local addresses, i.e. only those starting with exactly fe80:, i.e. fe80::/16. I submited a seperate PR #5301 that resolves this. This fix (or something similar) will probably need to be put in to fully close this one out.

It's probably never been found before because link-local addresses that do not start with exactly fe80: are quite rare.

@fichtner
Copy link
Member

Just as a clarification:

Case 1 is implemented now. Case 3 will be soon. And case 2 requires an exception for AdvSourceLLAddress off; as @pv2b mentioned as well as other things in the static/dynamic department since the RA Interface select box was always designed specifically for CARP.

@bimbar
Copy link

bimbar commented Oct 23, 2021

Great work, I like it. Though, if clients just respected RA priorities, this likely would not have been needed. But then, seems you can't expect anything from ipv6 clients that transcends the simplest case of one subnet, one gateway.

@pmhausen
Copy link
Contributor

pmhausen commented Nov 6, 2021

Any ETA for this to be merged? We are desperately waiting for the feature, too, because router advertisements are completely broken in a HA setup in production at the moment.

@fichtner
Copy link
Member

fichtner commented Nov 6, 2021

This won’t be merged. The code was adapted to deal with case 1 already and is available in the current development release. More work is pending which is why this is still open.

@pmhausen
Copy link
Contributor

pmhausen commented Nov 6, 2021

Ah - thanks. Can I apply the change to a production release with opnsense-patch?

@pv2b
Copy link
Contributor

pv2b commented Nov 6, 2021

Ah - thanks. Can I apply the change to a production release with opnsense-patch?

Yes. See #5185 (comment)

@fichtner
Copy link
Member

fichtner commented Nov 9, 2021

8aa30d6 will be added to 21.7.5 since feedback has been good so far. This is only case (1) for now, but there's no need to hold that back.

Cheers,
Franco

fichtner added a commit that referenced this pull request Nov 18, 2021
…ist()

Pages only call these to get lists but get_interface_ip*() functions want
a specific one.  Handroll these cases and simplify the other end.

PR: #5185
@fichtner fichtner closed this in a47949f Nov 18, 2021
@fichtner
Copy link
Member

a47949f probably won't patch easily on 21.7.x, but it includes cases (2) and (3) in the next 22.1 beta or 21.7 development version.

@fichtner
Copy link
Member

Guys, follow-up question for 0b76f2a is if we should make the "static" mode completely independent of CARP? The code is doing is-it-static-or-carp conditionals so you can't ever have DeprecatePrefix on/RemoveRoute on in CARP and I'm not sure if somebody will ask for it later anyway...

Thanks,
Franco

@vnxme
Copy link
Contributor Author

vnxme commented Nov 18, 2021

Guys, follow-up question for 0b76f2a is if we should make the "static" mode completely independent of CARP? The code is doing is-it-static-or-carp conditionals so you can't ever have DeprecatePrefix on/RemoveRoute on in CARP and I'm not sure if somebody will ask for it later anyway...

Thanks, Franco

Hi, Franco! Thank you very much for implementing the missing features for cases (2) and (3). As for the "static" mode, this is exactly what I talked about in #4953 (see bullet C of section Expected behavior), but that time I was not precise enough to explain my idea. I strongly support the option which would let the user decide whether enable this mode or not without binding this decision to the selected interface address type.

@pv2b
Copy link
Contributor

pv2b commented Nov 18, 2021

Guys, follow-up question for 0b76f2a is if we should make the "static" mode completely independent of CARP? The code is doing is-it-static-or-carp conditionals so you can't ever have DeprecatePrefix on/RemoveRoute on in CARP and I'm not sure if somebody will ask for it later anyway...

Thanks, Franco

I really don't understand what is this "static mode". I think there is a far too much of an opaque abstraction between the network administrator and the firewall's GUI. I understand the intent to keep it simple, but it's oversimplified to the point of being incomprehensible, at least to me. :-)

So instead I think I will chime in about what I think OPNsense should do regarding DeprecatePrefix and RemoveRoute.

First of all, DeprecatePrefix and RemoveRoute both really only do anything when shutting down the firewall or restarting radvd.

The DeprecatePrefix on option tells the clients to discontinue use of the prefix currently being announced when radvd is shutting down, I guess the idea is that maybe it'll announce a different prefix on startup, and in this case it's good to tell the clients, so they don't cling on to an old useless prefix.

The RemoveRoute on option tells the clients to remove the route from the routing table when radvd is shutting down. Again, in this case, this can be useful in case the administrator intends to replace the router with a new one with a new link-local address, so that the old route to the old router doesn't stay around.

Both of these options are a pretty bad idea when you're doing high availability, because you don't want one router to start telling the clients to drop routes and prefixes just because one of the HA nodes is going down, this is obviously counterproductive. So in most cases, having these features turned OFF for CARP setups is probably a good idea.

Also, depending on the specific scenario, it might also not be very useful in a single-router scenario.

If the router is shutting down, is it because it's rebooting and coming back up again? In this case DeprecatePrefix on and RemoveRoute on just causes unneccessary work on the client, but perhaps is not harmful.

If the router is shutting down because it'll be replaced by a different router (OPNsense or other vendor, doesn't really matter) then yes, this option is maybe useful, given that you will properly shut down the old firewall (or stop radvd) when replacing it and not just unplug it from the network...

It is also maybe useful in case you're rebooting the router and you think that maybe you're going to get a different IPv6 prefix when you reboot.

However, you still have to consider that we are not in an IPv6-only world. As far as I know there is no similar mechanism for DHCP in IPv4, so in these cases you have to often reboot client machines (or at least do a dhcp release/renew) when replacing a router with a different one.

Since this feature is harmful or useless in many cases, and is only of very marginal utility (fairly useless in most dual-stack networks), I would consider completely removing this feature for all setups, CARP or not. In my opinion it's just not a useful feature to make all this complexity.

If there are still some specific scenarios where you really want this feature, maybe you can just remove any automation for this, leave it OFF by default, and then allow the admin to turn it on if they really want it. Probably you want to have seperate knobs for RemoveRoute and DeprecatePrefix even.

@fichtner
Copy link
Member

Well, for now let me drop the feature request for the static mode: #4328

Specifically:

I've got a static prefix from my ISP so my LAN prefix never changes. When OPNsense reboots I loose internet access (that's expected) but I also loose LAN connectivity due to the deprecated prefix. I need to fall back to the link local addresses.

@pmhausen
Copy link
Contributor

@fichtner, if I read @pv2b correctly, the idea is to hardwire "static mode", i.e. never enable DeprecatePrefix. Which would essentially satisfy that feature request, too.

@vnxme
Copy link
Contributor Author

vnxme commented Nov 18, 2021

@pv2b Agree that the "static" mode is an abstraction which might confuse some users. Can't agree that the options are useless, but good to see you suggest keeping the way to turn them on/off if required. I would prefer implementing DeprecatePrefix and RemoveRoute as separate switches with their default values equal to the radvd manual defaults.

@pmhausen Can't agree with you in terms of completely preventing the user from enabling/disabling DeprecatePrefix and/or RemoveRoute. The existing "automatic" mode which is the current default effectively provides for DeprecatePrefix off and RemoveRoute on. I believe those who will ever consider IPv6 HA setups have already read the manual or will do it and will know what to do with these options.

A clear help text referring to the manual is certainly required for both options in question.

@pmhausen
Copy link
Contributor

pmhausen commented Nov 18, 2021

I did not intend to suggest that. I just wanted to clarify that (again, if read correctly) @pv2b's idea was to default to off, and possibly add dedicated switches. So your summary looks like the best solution to me.

@pv2b
Copy link
Contributor

pv2b commented Nov 19, 2021

To clarify: I don't think the features RemoveRoute and DeprecatePrefix features are entirely useless when considered in isolation.

However when you consider that pretty much all IPv6 deployments with an OPNsense box would have an IPv4 deployment on the same network, the features don't add any clear operational advantages holistically when you consider the whole picture including IPv4, and end up serving to confuse administrators in edge cases where IPv6 ends up acting differently than IPv4.

There is therefore a case to defaulting to these features being turned off for the purpose of inducing the least possible surprise in the end users.

There is also a case to be made that the use cases for these features is so niche that it makes sense to consider not expose them at all, at least not with the current state of networking.

Personally, I like having a lot of knobs to tweak, and for those knobs to be as transparent as possible to the underlying software, but then again it's not my job to provide support for the resulting GUI. I also recognise that the OPNsense devs have a lot of work and perhaps better places to put their time than adding these particular options. :-) So while removing the features completely is not my preferred option, it is an acceptable option to me if nobody wants to put in the work to make it happen.

In a magical dreamland where the devs were at my beck and call for my every whim, I'd like the ability to customize the configuration file completely, and for there to be some kind of GUI support to inhibit generation of configuration files, and warn me in case any configuration change made to the firewall might neccessitate the update of these config files. But I'm not about to put in that work, and I don't expect anyone to do this for me.

As an aside: I'd also advocate for putting these knobs under an "advanced" option if they do end up making it into the GUI.

@pmhausen
Copy link
Contributor

Seconded. Sections of advanced options are a well established concept and everyone wanting to tweak these options probably knows what they are doing. I also like as many knobs of the underlying software transparently exposed as possible.

@vnxme
Copy link
Contributor Author

vnxme commented Nov 19, 2021

Agree, moving the options in question to the advanced section is a good idea.

There is therefore a case to defaulting to these features being turned off for the purpose of inducing the least possible surprise in the end users.

@pv2b @pmhausen Please don't consider me a nitpicker. In fact, I second the underlying idea both of you are talking about and I really appreciate your contribution. But I would like to clarify that for the purposes of radvd config the "off" state should imply their default values which are:

  • DeprecatePrefix off
  • RemoveRoute on

In general, the issue with all such little options could have been easily resolved if the radvd configuration file syntax supported inclusions in a way it is realised by unbound. I understand it is barely expectable in the near future.

Talking about any options of whatever software under the hood of OPNsense, I would prefer to compare the latter against other solutions by Cisco, Juniper, Mikrotik, etc. available on the market. As for me, I'm upset when there is something easily achievable in e.g. RouterOS or even bare Debian/FreeBSD, but hardly doable in OPNsense because of some limitations of its web interface or config management system. I strongly believe that the OPNsense webgui is its key competitive edge. The more advanced setups it enables the user to build keeping it clear and consistent graphically and logically, the better.

@pv2b
Copy link
Contributor

pv2b commented Nov 19, 2021

If you're going to do RemoveRoute on by default, this will likely interfere with a CARP setup where you're using a common address like fe80::1 for routing to.

In case you want to default to this, we will definitely need a knob for this one specifically, for CARP setups.

@pmhausen
Copy link
Contributor

I don't see much of a problem here.

  1. You reboot the CARP slave. The slave does not have the CARP link local address so there is no route to retract.
  2. You reboot the CARP master. The master retracts the route, the slave takes the CARP link local address, the slave announces the route.

So possibly "on" is even the better setting for CARP setups.

@pv2b
Copy link
Contributor

pv2b commented Nov 20, 2021

@> I don't see much of a problem here.

1. You reboot the CARP slave. The slave does not have the CARP link local address so there is no route to retract.

2. You reboot the CARP master. The master retracts the route, the slave takes the CARP link local address, the slave announces the route.

So possibly "on" is even the better setting for CARP setups.

As long as this is how it ends up working, then sure, RemoveRoute on wouldn't be harmful in a CARP setup.

However I'm not convinced this is what would actually happen.

In my own environment (21.7.2_1 with 2c247d9 + f18d0b0 applied, CARP address is fe80::1), both routers end up sending router advertisements from fe80::1, even though one of the routers is the slave.

See the below packet capture, taken from a steady state. (In this case, the firewall with MAC xx:xx:xx:xx:xx:f6 is the CARP master, and xx:xx:xx:xx:xx:ae is the CARP slave.)

12:18:53.622844 xx:xx:xx:xx:xx:ae > 33:33:00:00:00:01, ethertype IPv6 (0x86dd), length 166: (hlim 255, next-header ICMPv6 (58) payload length: 112) fe80::1 > ff02::1: [icmp6 sum ok] ICMP6, router advertisement, length 112
        hop limit 64, Flags [other stateful], pref medium, router lifetime 1800s, reachable time 0s, retrans time 0s
          prefix info option (3), length 32 (4): 2a00:xxxx:xxxx:xxxx::/64, Flags [onlink, auto], valid time 86400s, pref. time 14400s
          prefix info option (3), length 32 (4): fe80::/64, Flags [onlink, auto], valid time 86400s, pref. time 14400s
          dnssl option (31), length 24 (3):  lifetime 600s, domain(s): xxxx.xxxx.net.
          mtu option (5), length 8 (1):  1500
12:19:18.561832 xx:xx:xx:xx:xx:f6 > 33:33:00:00:00:01, ethertype IPv6 (0x86dd), length 166: (hlim 255, next-header ICMPv6 (58) payload length: 112) fe80::1 > ff02::1: [icmp6 sum ok] ICMP6, router advertisement, length 112
        hop limit 64, Flags [other stateful], pref medium, router lifetime 1800s, reachable time 0s, retrans time 0s
          prefix info option (3), length 32 (4): 2a00:xxxx:xxxx:xxxx::/64, Flags [onlink, auto], valid time 86400s, pref. time 14400s
          prefix info option (3), length 32 (4): fe80::/64, Flags [onlink, auto], valid time 86400s, pref. time 14400s
          dnssl option (31), length 24 (3):  lifetime 600s, domain(s): xxxx.xxxx.net.
          mtu option (5), length 8 (1):  1500

The fact that the CARP slave doesn't "have" the fe80::1 address doesn't stop it from sending out router advertisements. Which logically would mean there's nothing stopping it from sending out a "remove route" from fe80::1 if RemoveRoute were enabled and radvd were restarted. Unfortunately I don't have a convenient lab set up at this very moment to validate this completely.

(This is why AdvSourceLLAddress off is required in a CARP setup! Otherwise the router advertisements end up containing an incorrect MAC address inside the router advertisement itself.)

Radvd is simply not aware of CARP, and the operating system does nothing to prevent applications like radvd from sending out packets with a source address where that machine is currently a CARP slave.

fichtner added a commit that referenced this pull request Jan 17, 2022
While technically not correct to make flags "AdvDeprecatePrefix"
and "AdvRemoveRoute" we simply use it to overwrite the existing
defaults which depend on CARP-or-not situation in the address
being used.

There was an extensive discussion about what it should and should
not do but the only way to keep the current defaults AND provide
requested overrides is this one.

The valid values are either "off" or "on", but no validation
takes place as customary to this page's advanced options. Use
with care.
fichtner added a commit that referenced this pull request Jan 18, 2022
While technically not correct to make flags "AdvDeprecatePrefix"
and "AdvRemoveRoute" we simply use it to overwrite the existing
defaults which depend on CARP-or-not situation in the address
being used.

There was an extensive discussion about what it should and should
not do but the only way to keep the current defaults AND provide
requested overrides is this one.

The valid values are either "off" or "on", but no validation
takes place as customary to this page's advanced options. Use
with care.

(cherry picked from commit a94c63b)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants