VRRP HLD

[dks19]

Hi all,
This is high level design for VRRP feature in SONiC. Currently, VRRPv2 along with interface tracking is supported. Please review and provide feedback/comments.

Thanks,
Dilip

[msftclas]

All CLA requirements met.

[dev-aviz]

It would be good to list the out the possible use cases for the sonic user to enable to this feature? Can this work on data center MLAG kind of deployments ?

[vvbrcm]

The use cases are networks needing backup of logical gateway. The network could be like described below or for a similar use case,

• 3-tier access layer, distribution layer & core layer network, where the redundancy for gateway is needed in distribution layer.
• Layer 3 CLOS network, where the redundancy is needed for the first hop gateways.

Generally static anycast gateway is used in MLAG deployment, but yes VRRP could also be used in MLAG deployment.

[dev-aviz]

what is preventing not to support vrrp3?
How is it different from FRR VRRP? Since FRR is part of SONiC stack, can't we stick to FRR VRRP (one stack)?

[vvbrcm]

Currently VRRPv3 (IPv6) support is added.

Since keepalived VRRP is robust, feature rich and well maintained in linux community, keepalived VRRP was chosen over FRR VRRP.

[dev-aviz]

How does the uplink tracking works? for instance let's say there are more than 8 uplink interfaces how do we does it effects on mastership?

[vvbrcm]

Uplink tracking affects the overall priority for VRRP router. If the tracked interface is in 'UP' state, then the VRRP priority for the router is increased by the weight associated with the tracked interface.

Currently only 8 interfaces can be tracked per VRRP instance. Please let us know if there is a real use case of tracking more than 8 interfaces per VRRP instance.

Note multiple VRRP instances could be configured on an VRRP interface.

[lguohan]

we should prevent warm reboot if there is vrrp configuraiton in place.

[vvbrcm]

In the new design VRRP master's will relinquish the mastership of VRRP instances (by sending priority 0 keepalives, so that the standby can take over the mastership instantaneously) before going to warm-reboot. During warm-reboot the neighboring routers will be master. Post warm-reboot the rebooted router could claim back the mastership.

This way the forwarding is kept functional (and not changing) during the process of warm-reboot.

[lguohan]

it is unclear. can you explain what does vrrporch do?

[vvbrcm]

vrrpsyncd listen to kernel and for each VRRP master instances, vrrpsyncd will add interface name, VIP & VMAC to APP_DB.
vrrporch will inturn listen to APP_DB and for entry in VRRP_TABLE, program the VIP as my IP and the VMAC as my MAC(virtual RIF) in ASIC_DB.

[lguohan]

break paragraph

[vvbrcm]

Ok, will split it.

[lguohan]

we need detailed integration test plan

[vvbrcm]

Ok.

[lguohan]

we need sai trap id for this. do we have it defined in the SAI, if yes, then add this as a sai requirement.

[vvbrcm]

The already defined SAI trap IDs viz., SAI_HOSTIF_TRAP_TYPE_VRRP & SAI_HOSTIF_TRAP_TYPE_VRRPV6 will be used to trap the VRRP packets.

[lguohan]

why not use interface table? why can we extend intforch? why do we need a separate vrrporch?

[vvbrcm]

Intforch was not used to separate the use case. In VRRP multiple my IPs and my MACs (virtual RIF) needs to be programmed and it could so happen that there could be multiple sources (different VRRP instances on the same interface), generating same my IP.

As intforch cannot handle it as is, creating vrrporch module to handle the functionality separately.

[lguohan]

macvlan interface can be used for other use cases, are we assuming all macvlan interface are created by keepalived?

[vvbrcm]

No, only MACVLANs whose interface name starts with 'vrrp' in kernel is associated with VRRP. keepalived adds MACVLANs with name starting with 'vrrp' substring.

[disaster123]

Does this one support active/active setups?

[dks19]

This implementation of VRRP doesn't support Active-Active mode.

…

On Wed, Oct 30, 2019 at 11:35 AM disaster123 ***@***.***> wrote: Does this one support active/active setups? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#475>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALQHQWHJ4GPBZEV5NGHSEYLQRHHYTANCNFSM4I374EJA> .

[disaster123]

@dks19 any chance to implement active active or vrr? I‘m willing to pay for it.

[nser77]

Quite interesting topic.

I personally find a lot of benefit into keepalived:

1. keepalived is heavily focused on the vrrp protocol (v2 and v3) and supports many configurations around it.
2. keepalived has many track controls to detect a failure: interfaces, script, process and files.
3. keepalived supports both iptables and nftables.

           # keepalived uses a firewall (either nftables or iptables) for two purposes:
           #  i)  To implement no_accept mode
           #  ii) To stop IGMP/MLD/Router-Solicit packets being sent on VMAC interfaces,
           #      and to move IGMP/MLD messages onto the underlying interface.

5. keepalived supports VRRP synchronization group(s): VRRP Sync Group is an extension to VRRP protocol (man).
6. keepalived supports static route creation - it may need some implementations like fpmsyncd with bgp (see above).
7. keepalived is a lightweight, robust and smart software.
8. keepalived uses SNMP.
9. keepalived works with namespaces.

Anyway, keepalived works quite close to the kernel so in this case the heavy problem might be related to the netlink integration and how keepalived could works natively into the SONiC environment.

[...]
Same as IPVS wrapper. Keepalived work with its own network interface representation. 
IP address and interface flags are set and monitored through kernel Netlink channel. 
The Netlink messaging sub-system is used for setting VRRP VIPs. 
On the other hand, the Netlink kernel messaging broadcast capability is used to reflect 
into our userspace Keepalived internal data representation any events related to interfaces. 
So any other userspace (others program) netlink manipulation is reflected to our 
Keepalived data representation via Netlink Kernel broadcast (RTMGRP_LINK & RTMGRP_IPV4_IFADDR).
[...]

As per SONiC achitecture, seems to be logic to manage the integration betwen keepalived and netlink trought the producer/consumer achitecture and with some of those processes that are already implemented for some of those scopes:

[...]

Portsyncd: Listens to port-related netlink events. 
During boot-up, portsyncd obtains physical-port information by parsing system's hardware-profile config files.
In all these cases, portsyncd ends up pushing all the collected state into APPL_DB.
Attributes such as port-speeds, lanes and mtu are transferred through this channel.
Portsyncd also inject state into STATE_DB. Refer to next section for more details.

Intfsyncd: Listens to interface-related netlink events and push collected state into APPL_DB. 
Attributes such as new/changed ip-addresses associated to an interface are handled by this process.

[...]

Orchagent: The most critical component in the Swss subsystem. 
Orchagent contains logic to extract all the relevant state injected by *syncd daemons, 
process and message this information accordingly, and finally push it towards its south-bound interface.
This south-bound interface is yet again another database within the redis-engine (ASIC_DB), so as we can see, 
Orchagent operates both as a consumer (for example for state coming from APPL_DB),
and also as a producer (for state being pushed into ASIC_DB).

IntfMgrd: Reacts to state arriving from APPL_DB, CONFIG_DB and STATE_DB to configure interfaces in the linux kernel. 
This step is only accomplished if there is no conflicting or inconsistent state within any of the databases being monitored.
Refer to the above database-container section for examples of this undesired behavior.

VlanMgrd: Reacts to state arriving from APPL_DB, CONFIG_DB and STATE_DB to configure vlan-interfaces in the linux kernel.
As in IntfMgrd's case, this step will be only attempted if there is no dependent state/conditions being unmet.

[...]

As @dks19 suggested in his HLD, a vrrpsyncd process should also be added and, as per SONiC architecture, it may need to act as a producer for kernel integration (routes, ipaddress, etc. ..) - like fpmsyncd acts for bgp - and also as consumer, for all of those required netlink messages in both directions; from my point of view, the integration betwen keepalived and SONiC should relay over the SAI ecosystem and keepalivedshould not directly operate to the kernel; on the other hand, I'm not sure if vrrporchd is required.

Remember that vrrp works over multicast, but keepalived also implements unicast.

The integration of keepalived in SONiC would be very useful if it were fully integrated into the producer/consumer architecture, that's the power of SONiC, but keepalived really works close to the kernel; so, before proceding, an handshake with the mainters of keepalived might be required.

At the moment, I'm quite interested but still skeptical about this integration (perhaps due to lack of knowledge), anyway, keepalived should not replace frr in the SONiC architecture, but simply provide a mission-critical vrrp support for the enterprise in a separate, dedicated container.

[linux-foundation-easycla]

• ❌ - login: @dks19 / name: Dilip . The commit (0aa1db5, 3ceee3a, 7c77c69, eb48cde, 33c2c86, b13dd68) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.
• ❌ - login: @vvbrcm . The commit (8a93483, 09f1143, 37b9d0d, bed3582) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.

[jefftant]

Please note that we (IETF RTGWG chairs) have just last called draft-ietf-rtgwg-vrrp-rfc5798bis, upon becoming RFC it will absolete RFC5798, please use it as a base. Cheers, Jeff

…

On Fri, Jul 14, 2023 at 16:55 linux-foundation-easycla[bot] ***@***.*** ***@***.***>> wrote: <https://api.easycla.lfx.linuxfoundation.org/v2/repository-provider/github/sign/27908000/51108542/475/#/?version=2> ❌ <https://api.easycla.lfx.linuxfoundation.org/v2/repository-provider/github/sign/27908000/51108542/475/#/?version=2> - login: @dks19 <https://github.com/dks19> / name: Dilip . The commit (0aa1db5 <0aa1db5>, 3ceee3a <3ceee3a>, 7c77c69 <7c77c69>, eb48cde <eb48cde>, 33c2c86 <33c2c86>) is not authorized under a signed CLA. Please click here to be authorized <https://api.easycla.lfx.linuxfoundation.org/v2/repository-provider/github/sign/27908000/51108542/475/#/?version=2>. For further assistance with EasyCLA, please submit a support request ticket <https://jira.linuxfoundation.org/servicedesk/customer/portal/4>. — Reply to this email directly, view it on GitHub <#475 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABSIDQ6QJZ4YYRNSPIIVE5TXQHL6PANCNFSM4I374EJA>. You are receiving this because you are subscribed to this thread.

[adyeung]

Converge VRRP proposal with #1446