New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
radvd: Implement AdvRASrcAddress option #5185
Conversation
Make it possible to send RA from any link-local IPv6 address (e.g. fe80::1)
To my knowledge: a) CARP support for Router Advertisements works What am I missing? |
@fichtner Thank you for entering this discussion.
I'm not sure I understand what you mean. When you set RA interface to anything other than dynamic or static, dhcpd.inc changes DeprecatePrefix and RemoveRoute options while the daemon keeps sending RAs from the first link-local address.
Do you have any proofing materials describing this behaviour? CARP seems to be similar to VRRP in Linux, and radvd + VRRP solution has already been described, e.g. this article. A few tests I have made so far in OPNsense didn't reveal any problems with sending RAs from CARP - radvdump on other machines and iPhone/MacOS interfaces correctly report fe80::1 gateway. |
Well for both things see e.g. https://www.netbsd.org/docs/guide/en/chap-carp.html
That's why the CARP mode (RA interface set) does not deprecate prefixes or routes, also because:
This is why we need to switch routers in this case for IPv6 too. GUA with prefix delegation can be a problem, but it's not clear from the description so I assume that these issues are known and properly set up beforehand. |
The conclusion of this thread is also relatively easy to process: https://forum.netgate.com/topic/133306/ipv6-carp-vip-with-route-advertisements/8 |
What I don't quite understand, why not disable "Advertise Default Gateway" and add a custom default route through your link local address, e.g. fe80::111/0 ? |
@fichtner Thanks, I will take some time to read through the materials. As far as I can see, CARP is only one possible option here. What if I just want to send RAs from fe80::1 (for whatever reason which is beyond this discussion) which is my static IP alias (i.e. not CARP)? This is what I'm talking about in point (A) of #4953 - RA interface list should include all types of VIPv6, not only CARP ones. |
radvd.conf is also interesting for AdvRASrcAddress, where it says:
Which is part of the earlier point. It's a little magical the way it is described, but at this point it's unclear if setting an explicit AdvRASrcAddress that is not really a source address but just fudges the advertisement's default route to e.g. a CARP IP so it looks seamless is the required approach. In that case in particular we should, however, not start making this manually configurable as it's easy to push the CARP IP into the config and avoid setup problems. Cheers, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we need to use AdvRASrcAddress in CARP scenarios we must set it automatically.
The "Advertise Default Gateway" switch is definitely on the Router Advertisements page. One can add more routes to advertise on the same page, but no gateways can be specified there, i.e. subnets are routed through the link-local address radvd sends RAs from. Where can I set a custom default route the way you suggest so that clients could get fe80::1 as their default route gateway? |
Actually that is the precise problem description now. ;)
… On 29. Aug 2021, at 16:01, vnxme ***@***.***> wrote:
What I don't quite understand, why not disable "Advertise Default Gateway" and add a custom default route through your link local address, e.g. fe80::111/0 ?
The "Advertise Default Gateway" switch is definitely on the Router Advertisements page. One can add more routes to advertise on the same page, but no gateways can be specified there, i.e. subnets are routed through the link-local address radvd sends RAs from. Where can I set a custom default route the way you suggest so that clients could get fe80::1 as their default route gateway?
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
|
@fichtner I think I don't fully understand you again. I can't see any decent way to add a default route for ipv6 via a custom link-local address in OPNsense now (say, host A advertises a default route via host B). But if I create a new custom link-local VIPv6 (not CARP), manually add AdvRASrcAddress to radvd.conf and manually restart the daemon, I get exactly what I want - automatic custom default gateway on clients (same host A, but different, more attractive address is used). With this PR I can do the same with a new custom link-local CARP VIPv6, RAs are sent normally and clients configure their interfaces automatically with the chosen custom default gateway. So, let me summarize my point:
|
It’s clearer now after discussing it, but I’m not the one having brought up CARP in the first place. I don’t see a general issue here that makes AdvRASrcAddress a bare necessity.
last but not least the radvd.conf manual itself mentions VRRP (CARP) for AdvRASrcAddress so that still points to special use case. |
As for the route/AdvRASrcAddress combination that was a misconception by me after reading radvd.conf manual page which doesn't start a new section after configuration entries making it easier to misread. |
I looked through the links you provided earlier and my thoughts are as follows:
Please, don't hesitate to correct me if I missed the most important bits of information. What I actually found is Netgate merged more or less the same logic of AdvRASrcAddress earlier this year. Their users reported the solution to be working as expected. My idea is quite simple: if the user changes RA interface from dynamic/static to any VIP (be it simple alias or CARP), one expects radvd to send RAs from it. The daemon allows to do it, so why don't we make it possible in OPNsense? |
I think we want the same thing, but can't agree on a format: I don't think we should make AdvRASrcAddress accessible in the GUI. Otherwise we need docs and community support and countless questions follow how to set up something "so simple". 99% of those things will not know or care about AdvRASrcAddress. As for technical details of CARP they really do not matter the way radvd seems to be wired to work so feel free to discard any knowledge gaps. As radvd.conf manual page states spoofing does not occur and that includes the CARP address or any bound virtual IP, but feel free to verify the opposite with wireshark. |
Great, now we are on the same side. This PR actually doesn't expose this option to the GUI. It just makes the already existing RA interface drop-down list more predictable/less confusing. Dynamic/static options require no AdvRASrcAddress at all. The remaining RA interface options (which I believe most users won't even have) are VIPs, and that's the point when our backend code should put AdvRASrcAddress to radvd.conf automatically. Would you like to propose another logic? |
Well, first we need to get rid of calling get_interface_ipv6() by hand-rolling the use case to allow for grabbing the ipalias as well as the carp address that is there using the first practical link-local ... unverified commit f11aa507acf -- it may need falling back to the carp address as well. Afterwards you can use this to set AdvRASrcAddress in the particular cases. Thoughts? |
Hi! Great to see work on this, I am just independently going down this same rabbit hole again. I also came to the conclusion that if I want "IPv4-like" working failover with a CARP cluster, I have to set up a fe80::1 CARP VIP on my LAN and then somehow get radvd to use this LL IPv6 address to advertise. It has to be a link-local address, as per RFC4861 section 4.2:
As far as I can tell, I don't think there's any harm in just always specifying AdvRASrcAddress in any case where a link-local address is explicitly configured, no matter if it's CARP, or some other type of VIP/IP Alias. You probably don't want it always set for the typical (non-CARP) case where a link-local address is dynamically assigned, but that's all. You probably also want to include a filter so that the IPv6 address has to be in fe80::/64 (or maybe fe80::/10, not sure on this one) to avoid non-compliant configurations. As for the UX in this case, it would involve the user knowing they must set up a link local CARP VIP when setting up gateway redundancy with IPv6, and then select this interface when configuring router advertisements. The AdvRASrcAddress would not be directly exposed in the GUI. |
I should also mentioned that I just tested the patch above, and at least for the purposes of getting LAN gateway redundancy with CARP to work, this doesn't appear to be sufficient. The MAC address learnt through ND will still be the MAC of the individual router sending out the RA which doesn't provide the desired failover behaviour. For this kind of scenario you probably also need to add this option to the interface config in radvd.conf:
This way the MAC address of the router is not learnt through the RA (which would be the ethernet MAC of that particular router), and instead is learnt through ND, which hopefully would be the CARP MAC. |
@fichtner Hi, Franco! Sorry for my late reply. I would like to continue our work on this PR. Could you please elaborate on your comment where you say that we must set it automatically? If the user adds a link-local IPv6 CARP, one may or may not intend to use the IPv6 RA high-availability feature radvd provides with AdvRASrcAddress. Besides, the user may also add the second/third/etc. link-local IPv6 CARPs, and then we are unable to decide which one to use as RA source. |
NM, just checked your commit and it seems perfectly fine to me, that's just what we need. |
I just copied that over to my firewalls, because I want that functionality, found a little problem:
It started to advertise fe80::/64, too, which probably doesn't hurt, but shouldn't be done anyway. This is probably something that existed before, but there was little need for link-local CARP addresses then. |
We discussed this at the office a bit. The new proposal is 2c247d9 + f18d0b0, so
You do need to set up a link-local virtual IPv6 for CARP to work, manual setup (in case of tracking) and select that CARP address in the router advertisement configuration page. Cheers, |
I would like to provide a use case for each VIP type (pure CARP, pure alias, mixed):
|
@pv2b I'll add a comment to the block and will work on this further to clean up the style of course @vnxme fair enough, although we could argue that case 2 is more or less the feature request to set an interface link-local address manually instead of adding an alias in which case this is not a VIP relation and works by default. It's been on my wish list forever, but also not worth the amount of work. Case 2 is certainly easier to reach... Let me think about it. :) |
I identified another small problem in the codebase, the current is_linklocal function only detects a subset of valid link-local addresses, i.e. only those starting with exactly fe80:, i.e. fe80::/16. I submited a seperate PR #5301 that resolves this. This fix (or something similar) will probably need to be put in to fully close this one out. It's probably never been found before because link-local addresses that do not start with exactly fe80: are quite rare. |
Just as a clarification: Case 1 is implemented now. Case 3 will be soon. And case 2 requires an exception for |
Great work, I like it. Though, if clients just respected RA priorities, this likely would not have been needed. But then, seems you can't expect anything from ipv6 clients that transcends the simplest case of one subnet, one gateway. |
Any ETA for this to be merged? We are desperately waiting for the feature, too, because router advertisements are completely broken in a HA setup in production at the moment. |
This won’t be merged. The code was adapted to deal with case 1 already and is available in the current development release. More work is pending which is why this is still open. |
Ah - thanks. Can I apply the change to a production release with |
Yes. See #5185 (comment) |
8aa30d6 will be added to 21.7.5 since feedback has been good so far. This is only case (1) for now, but there's no need to hold that back. Cheers, |
…ist() Pages only call these to get lists but get_interface_ip*() functions want a specific one. Handroll these cases and simplify the other end. PR: #5185
a47949f probably won't patch easily on 21.7.x, but it includes cases (2) and (3) in the next 22.1 beta or 21.7 development version. |
Guys, follow-up question for 0b76f2a is if we should make the "static" mode completely independent of CARP? The code is doing is-it-static-or-carp conditionals so you can't ever have Thanks, |
Hi, Franco! Thank you very much for implementing the missing features for cases (2) and (3). As for the "static" mode, this is exactly what I talked about in #4953 (see bullet C of section Expected behavior), but that time I was not precise enough to explain my idea. I strongly support the option which would let the user decide whether enable this mode or not without binding this decision to the selected interface address type. |
I really don't understand what is this "static mode". I think there is a far too much of an opaque abstraction between the network administrator and the firewall's GUI. I understand the intent to keep it simple, but it's oversimplified to the point of being incomprehensible, at least to me. :-) So instead I think I will chime in about what I think OPNsense should do regarding DeprecatePrefix and RemoveRoute. First of all, DeprecatePrefix and RemoveRoute both really only do anything when shutting down the firewall or restarting radvd. The The Both of these options are a pretty bad idea when you're doing high availability, because you don't want one router to start telling the clients to drop routes and prefixes just because one of the HA nodes is going down, this is obviously counterproductive. So in most cases, having these features turned OFF for CARP setups is probably a good idea. Also, depending on the specific scenario, it might also not be very useful in a single-router scenario. If the router is shutting down, is it because it's rebooting and coming back up again? In this case If the router is shutting down because it'll be replaced by a different router (OPNsense or other vendor, doesn't really matter) then yes, this option is maybe useful, given that you will properly shut down the old firewall (or stop radvd) when replacing it and not just unplug it from the network... It is also maybe useful in case you're rebooting the router and you think that maybe you're going to get a different IPv6 prefix when you reboot. However, you still have to consider that we are not in an IPv6-only world. As far as I know there is no similar mechanism for DHCP in IPv4, so in these cases you have to often reboot client machines (or at least do a dhcp release/renew) when replacing a router with a different one. Since this feature is harmful or useless in many cases, and is only of very marginal utility (fairly useless in most dual-stack networks), I would consider completely removing this feature for all setups, CARP or not. In my opinion it's just not a useful feature to make all this complexity. If there are still some specific scenarios where you really want this feature, maybe you can just remove any automation for this, leave it OFF by default, and then allow the admin to turn it on if they really want it. Probably you want to have seperate knobs for RemoveRoute and DeprecatePrefix even. |
Well, for now let me drop the feature request for the static mode: #4328 Specifically:
|
@pv2b Agree that the "static" mode is an abstraction which might confuse some users. Can't agree that the options are useless, but good to see you suggest keeping the way to turn them on/off if required. I would prefer implementing DeprecatePrefix and RemoveRoute as separate switches with their default values equal to the radvd manual defaults. @pmhausen Can't agree with you in terms of completely preventing the user from enabling/disabling DeprecatePrefix and/or RemoveRoute. The existing "automatic" mode which is the current default effectively provides for DeprecatePrefix off and RemoveRoute on. I believe those who will ever consider IPv6 HA setups have already read the manual or will do it and will know what to do with these options. A clear help text referring to the manual is certainly required for both options in question. |
I did not intend to suggest that. I just wanted to clarify that (again, if read correctly) @pv2b's idea was to default to off, and possibly add dedicated switches. So your summary looks like the best solution to me. |
To clarify: I don't think the features RemoveRoute and DeprecatePrefix features are entirely useless when considered in isolation. However when you consider that pretty much all IPv6 deployments with an OPNsense box would have an IPv4 deployment on the same network, the features don't add any clear operational advantages holistically when you consider the whole picture including IPv4, and end up serving to confuse administrators in edge cases where IPv6 ends up acting differently than IPv4. There is therefore a case to defaulting to these features being turned off for the purpose of inducing the least possible surprise in the end users. There is also a case to be made that the use cases for these features is so niche that it makes sense to consider not expose them at all, at least not with the current state of networking. Personally, I like having a lot of knobs to tweak, and for those knobs to be as transparent as possible to the underlying software, but then again it's not my job to provide support for the resulting GUI. I also recognise that the OPNsense devs have a lot of work and perhaps better places to put their time than adding these particular options. :-) So while removing the features completely is not my preferred option, it is an acceptable option to me if nobody wants to put in the work to make it happen. In a magical dreamland where the devs were at my beck and call for my every whim, I'd like the ability to customize the configuration file completely, and for there to be some kind of GUI support to inhibit generation of configuration files, and warn me in case any configuration change made to the firewall might neccessitate the update of these config files. But I'm not about to put in that work, and I don't expect anyone to do this for me. As an aside: I'd also advocate for putting these knobs under an "advanced" option if they do end up making it into the GUI. |
Seconded. Sections of advanced options are a well established concept and everyone wanting to tweak these options probably knows what they are doing. I also like as many knobs of the underlying software transparently exposed as possible. |
Agree, moving the options in question to the advanced section is a good idea.
@pv2b @pmhausen Please don't consider me a nitpicker. In fact, I second the underlying idea both of you are talking about and I really appreciate your contribution. But I would like to clarify that for the purposes of radvd config the "off" state should imply their default values which are:
In general, the issue with all such little options could have been easily resolved if the radvd configuration file syntax supported inclusions in a way it is realised by unbound. I understand it is barely expectable in the near future. Talking about any options of whatever software under the hood of OPNsense, I would prefer to compare the latter against other solutions by Cisco, Juniper, Mikrotik, etc. available on the market. As for me, I'm upset when there is something easily achievable in e.g. RouterOS or even bare Debian/FreeBSD, but hardly doable in OPNsense because of some limitations of its web interface or config management system. I strongly believe that the OPNsense webgui is its key competitive edge. The more advanced setups it enables the user to build keeping it clear and consistent graphically and logically, the better. |
If you're going to do In case you want to default to this, we will definitely need a knob for this one specifically, for CARP setups. |
I don't see much of a problem here.
So possibly "on" is even the better setting for CARP setups. |
@> I don't see much of a problem here.
As long as this is how it ends up working, then sure, RemoveRoute on wouldn't be harmful in a CARP setup. However I'm not convinced this is what would actually happen. In my own environment (21.7.2_1 with 2c247d9 + f18d0b0 applied, CARP address is fe80::1), both routers end up sending router advertisements from fe80::1, even though one of the routers is the slave. See the below packet capture, taken from a steady state. (In this case, the firewall with MAC xx:xx:xx:xx:xx:f6 is the CARP master, and xx:xx:xx:xx:xx:ae is the CARP slave.)
The fact that the CARP slave doesn't "have" the fe80::1 address doesn't stop it from sending out router advertisements. Which logically would mean there's nothing stopping it from sending out a "remove route" from fe80::1 if RemoveRoute were enabled and radvd were restarted. Unfortunately I don't have a convenient lab set up at this very moment to validate this completely. (This is why Radvd is simply not aware of CARP, and the operating system does nothing to prevent applications like radvd from sending out packets with a source address where that machine is currently a CARP slave. |
While technically not correct to make flags "AdvDeprecatePrefix" and "AdvRemoveRoute" we simply use it to overwrite the existing defaults which depend on CARP-or-not situation in the address being used. There was an extensive discussion about what it should and should not do but the only way to keep the current defaults AND provide requested overrides is this one. The valid values are either "off" or "on", but no validation takes place as customary to this page's advanced options. Use with care.
While technically not correct to make flags "AdvDeprecatePrefix" and "AdvRemoveRoute" we simply use it to overwrite the existing defaults which depend on CARP-or-not situation in the address being used. There was an extensive discussion about what it should and should not do but the only way to keep the current defaults AND provide requested overrides is this one. The valid values are either "off" or "on", but no validation takes place as customary to this page's advanced options. Use with care. (cherry picked from commit a94c63b)
Idea:
Issues:
Solution:
For discussion: