wlb: T4470: Migrate WAN Load Balancer to XML/Python #4108

sarthurdev · 2024-09-29T11:30:35Z

Change Summary

This PR migrates the last remaining Vyatta perl module to XML/Python standard.

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Code style update (formatting, renaming)
Refactoring (no functional changes)
Migration from an old Vyatta component to vyos-1x, please link to related PR inside obsoleted component
Other (please describe):

Related Task(s)

Related PR(s)

Component(s) name

wanloadbalance

Proposed changes

Migrate perl scripts/daemon to Python
Migrate op-mode from vyatta-wanloadbalance package

How to test

Smoketest result

DEBUG - Running Testcase: /usr/libexec/vyos/tests/smoke/cli/test_load-balancing_wan.py
DEBUG - test_check_chains (__main__.TestLoadBalancingWan.test_check_chains) ... ok
DEBUG - test_table_routes (__main__.TestLoadBalancingWan.test_table_routes) ... ok
DEBUG - 
DEBUG - ----------------------------------------------------------------------
DEBUG - Ran 2 tests in 16.750s
DEBUG - 
DEBUG - OK

Checklist:

I have read the CONTRIBUTING document
I have linked this PR to one or more Phabricator Task(s)
I have run the components SMOKETESTS if applicable
My commit headlines contain a valid Task id
My change requires a change to the documentation
I have updated the documentation accordingly

github-actions · 2024-09-29T11:30:57Z

👍
No issues in PR Title / Commit Title

github-actions · 2024-09-29T11:31:13Z

✅ No issues found in unused-imports check.. Please refer the workflow run

python/vyos/wanloadbalance.py

src/helpers/vyos-load-balancer.py

sever-sever · 2024-10-09T08:43:35Z

python/vyos/wanloadbalance.py

+    rc = run(cmd_str)
+    return rc != 0
+
+def parse_dhcp_nexthop(ifname):


Will PPPoE interfaces also work?
I didn't find any code regarding PPP.

Good question, let me look into that.

github-actions · 2024-10-10T19:41:11Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2024-10-21T14:49:53Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

github-actions · 2024-10-21T16:01:07Z

CI integration ❌ failed!

Details

CI logs

CLI Smoketests (no interfaces) ❌ failed
CLI Smoketests (interfaces only) 👍 passed
Config tests 👍 passed
RAID1 tests 👍 passed
TPM tests 👍 passed

sever-sever · 2024-11-05T13:43:29Z

src/helpers/vyos-load-balancer.py

+                run(f'ip route replace table {table_num} default dev {ifname} via {health_conf["nexthop"]}')
+
+            run(f'ip route delete table {table_num}')
+            run(f'ip rule add fwmark {hex(table_num)} table {table_num}')


The ip rule are not deleted

vyos@router:~$ show conf com | match wan set load-balancing wan interface-health eth0 nexthop '10.0.1.1' set load-balancing wan interface-health eth1 nexthop '10.0.2.1' vyos@router:~$ vyos@router:~$ vyos@router:~$ ip rule 0: from all lookup local 32756: from all fwmark 0xca lookup 202 32757: from all fwmark 0xc9 lookup 201 32758: from all fwmark 0xca lookup 202 32759: from all fwmark 0xc9 lookup 201 32760: from all fwmark 0xca lookup 202 32761: from all fwmark 0xc9 lookup 201 32762: from all fwmark 0xca lookup 202 32763: from all fwmark 0xc9 lookup 201 32764: from all fwmark 0xca lookup 202 32765: from all fwmark 0xc9 lookup 201 32766: from all lookup main 32767: from all lookup default vyos@router:~$ vyos@router# delete load-balancing [edit] vyos@router# commit [edit] vyos@router# ip rule 0: from all lookup local 32756: from all fwmark 0xca lookup 202 32757: from all fwmark 0xc9 lookup 201 32758: from all fwmark 0xca lookup 202 32759: from all fwmark 0xc9 lookup 201 32760: from all fwmark 0xca lookup 202 32761: from all fwmark 0xc9 lookup 201 32762: from all fwmark 0xca lookup 202 32763: from all fwmark 0xc9 lookup 201 32764: from all fwmark 0xca lookup 202 32765: from all fwmark 0xc9 lookup 201 32766: from all lookup main 32767: from all lookup default [edit] vyos@router# vyos@router# set load-balancing wan interface-health eth0 nexthop '10.0.1.1' [edit] vyos@router# set load-balancing wan interface-health eth1 nexthop '10.0.2.1' [edit] vyos@router# commit [edit] vyos@router# ip rule 0: from all lookup local 32746: from all fwmark 0xca lookup 202 32747: from all fwmark 0xc9 lookup 201 32748: from all fwmark 0xca lookup 202 32749: from all fwmark 0xc9 lookup 201 32750: from all fwmark 0xca lookup 202 32751: from all fwmark 0xc9 lookup 201 32752: from all fwmark 0xca lookup 202 32753: from all fwmark 0xc9 lookup 201 32754: from all fwmark 0xca lookup 202 32755: from all fwmark 0xc9 lookup 201 32756: from all fwmark 0xca lookup 202 32757: from all fwmark 0xc9 lookup 201 32758: from all fwmark 0xca lookup 202 32759: from all fwmark 0xc9 lookup 201 32760: from all fwmark 0xca lookup 202 32761: from all fwmark 0xc9 lookup 201 32762: from all fwmark 0xca lookup 202 32763: from all fwmark 0xc9 lookup 201 32764: from all fwmark 0xca lookup 202 32765: from all fwmark 0xc9 lookup 201 32766: from all lookup main 32767: from all lookup default [edit] vyos@router#

sever-sever · 2024-11-05T13:50:05Z

ip rules not deleted (after deleting wlb config)
Delete stick-conections fails

set load-balancing wan interface-health eth0 nexthop '10.0.1.1'
set load-balancing wan interface-health eth1 nexthop '10.0.2.1'
set load-balancing wan sticky-connections
commit


vyos@router# delete load-balancing wan sticky-connections 
[edit]
vyos@router# commit
[ load-balancing wan ]
Traceback (most recent call last):
  File "/usr/libexec/vyos/services/vyos-configd", line 136, in run_script
    script.apply(c)
  File "/usr/libexec/vyos//conf_mode/load-balancing_wan.py", line 104, in apply
    cmd(f'systemctl restart {service}')
  File "/usr/lib/python3/dist-packages/vyos/utils/process.py", line 155, in cmd
    raise OSError(code, feedback)
PermissionError: [Errno 1] failed to run command: systemctl restart vyos-wan-load-balance.service
returned: 
exit code: 1

[[load-balancing wan]] failed
Commit failed
[edit]
vyos@router#

Reverse protocol match does not work:

set load-balancing wan interface-health eth0 nexthop '10.0.1.1'
set load-balancing wan interface-health eth1 nexthop '10.0.2.1'
set load-balancing wan rule 20 inbound-interface 'eth2'
set load-balancing wan rule 20 interface eth1
set load-balancing wan rule 20 protocol 'tcp'
set load-balancing wan sticky-connections inbound
commit
set load-balancing wan rule 20 protocol !tcp 
commit

rules:

        chain wlb_mangle_prerouting {
                type filter hook prerouting priority mangle; policy accept;
                iifname "eth0" ct state new ct mark set 0x000000c9
                iifname "eth1" ct state new ct mark set 0x000000ca
                iifname "eth2" meta l4proto tcp ct state new limit rate 5/second burst 5 packets counter packets 0 bytes 0 jump wlb_mangle_isp_eth1
                iifname "eth2" meta l4proto tcp counter packets 0 bytes 0 meta mark set ct mark
        }

Delete load-balancing does not flush nft rules

vyos@router:~$ show conf com | match load
set load-balancing wan enable-local-traffic
set load-balancing wan interface-health eth0 nexthop '10.0.1.1'
set load-balancing wan interface-health eth1 nexthop '10.0.2.1'
set load-balancing wan rule 20 inbound-interface 'eth2'
set load-balancing wan rule 20 interface eth1
set load-balancing wan rule 20 protocol '47'
set load-balancing wan sticky-connections inbound
vyos@router:~$ conf
d[edit]
vyos@router# delete load-balancing 
[edit]
vyos@router# commit
[edit]
vyos@router#

check:

vyos@router:~$ sudo nft list table vyos_wanloadbalance
table ip vyos_wanloadbalance {
	chain wlb_nat_postrouting {
		type nat hook postrouting priority srcnat - 1; policy accept;
		ct mark 0x000000c9 counter packets 0 bytes 0 snat to 10.0.1.2
		ct mark 0x000000ca counter packets 1 bytes 328 snat to 10.0.2.2
	}

	chain wlb_mangle_prerouting {
		type filter hook prerouting priority mangle; policy accept;
		iifname "eth0" ct state new ct mark set 0x000000c9
		iifname "eth1" ct state new ct mark set 0x000000ca
		iifname "eth2" meta l4proto gre ct state new limit rate 5/second burst 5 packets counter packets 0 bytes 0 jump wlb_mangle_isp_eth1
		iifname "eth2" meta l4proto gre counter packets 0 bytes 0 meta mark set ct mark
	}

	chain wlb_mangle_output {
		type filter hook output priority mangle; policy accept;
		meta mark != 0x00000000 counter packets 0 bytes 0 accept
		meta l4proto icmp counter packets 6096 bytes 552936 accept
		ip saddr 127.0.0.0/8 ip daddr 127.0.0.0/8 counter packets 0 bytes 0 accept
		oifname != "eth2" meta l4proto gre ct state new limit rate 5/second burst 5 packets counter packets 0 bytes 0 jump wlb_mangle_isp_eth1
		oifname != "eth2" meta l4proto gre counter packets 0 bytes 0 meta mark set ct mark
	}

	chain wlb_mangle_isp_eth0 {
		meta mark set 0x000000c9 ct mark set 0x000000c9 counter packets 0 bytes 0 accept
	}

	chain wlb_mangle_isp_eth1 {
		meta mark set 0x000000ca ct mark set 0x000000ca counter packets 0 bytes 0 accept
	}
}
vyos@router:~$

Delete load-balancing does not flush routes

vyos@router# delete load-balancing 
[edit]
vyos@router# commit
[edit]
vyos@router# 
[edit]
vyos@router# 
[edit]
vyos@router# run show ip route table all
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF default table 201:
K>* 0.0.0.0/0 [0/0] via 10.0.1.1, eth0, 02:32:50

VRF default table 202:
K>* 0.0.0.0/0 [0/0] via 10.0.2.1, eth1, 02:32:50

Next-hop DHCP does not add routes in the tables

set interfaces ethernet eth0 address 'dhcp'
set interfaces ethernet eth1 address 'dhcp'
set load-balancing wan interface-health eth0 nexthop 'dhcp'
set load-balancing wan interface-health eth1 nexthop 'dhcp'
set load-balancing wan rule 20 inbound-interface 'eth2'
set load-balancing wan rule 20 interface eth1
set load-balancing wan rule 20 protocol 'tcp'
set load-balancing wan sticky-connections inbound

Check:

vyos@router# run show ip route table all
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF default table 254:
S>* 0.0.0.0/0 [210/0] via 10.0.1.1, eth0, weight 1, 00:03:15
  *                   via 10.0.2.1, eth1, weight 1, 00:03:15
C>* 10.0.1.0/24 is directly connected, eth0, 00:03:15
C>* 10.0.2.0/24 is directly connected, eth1, 00:03:15
C>* 100.64.0.0/24 is directly connected, eth2, 00:07:34
C>* 192.168.122.0/24 is directly connected, eth4, 00:07:35
[edit]
vyos@router# 
[edit]
vyos@router# run show ip route table 201
[edit]
vyos@router# run show ip route table 202
[edit]
vyos@router# sudo ip route show table 201
Error: ipv4: FIB table does not exist.
Dump terminated
[edit]
vyos@router# sudo ip route show table 202
Error: ipv4: FIB table does not exist.
Dump terminated
[edit]
vyos@router#

sever-sever · 2024-11-05T14:27:33Z

src/op_mode/load-balancing_wan.py

+            'ifname': ifname,
+            'status': "active" if if_data['state'] else "failed",
+            'last_change': change_dt.strftime("%Y-%m-%d %H:%M:%S") if change_dt else 'N/A',
+            'last_success': str(now - success_dt) if success_dt else 'N/A',


What does the output format mean for Last Interface Success:?

vyos@router:~$ show wan-load-balance Interface: eth0 Status: active Last Status Change: 2024-11-05 14:21:30 Last Interface Success: 0:03:48.162705 Last Interface Failure: N/A Interface Failures: 0 Interface: eth1 Status: failed Last Status Change: 2024-11-05 14:21:35 Last Interface Success: 0:03:53.167816 Last Interface Failure: 0:03:43.160109 Interface Failures: 1 vyos@router:~$

HollyGurza · 2024-11-05T14:50:23Z

src/conf_mode/load-balancing_wan.py

-    else:
-        cmd('sudo sysctl -w net.netfilter.nf_conntrack_acct=1')
-        cmd(f'systemctl restart {systemd_service}')
+    cmd(f'systemctl restart {service}')

    call_dependents()


I think, call to call_dependents() remains valuable even when the load balancer configuration is empty. It can handle cases where an old configuration has been removed, ensuring all dependencies are correctly updated.

sever-sever · 2024-11-05T16:44:07Z

python/vyos/wanloadbalance.py

+
+from vyos.utils.process import run
+
+dhclient_lease = '/var/lib/dhcp/dhclient_{0}.lease'


This path seems incorrect:

vyos@router# ls -la /var/lib/dhcp/ total 8 drwxr-xr-x 2 root root 3 Oct 20 2023 . drwxr-xr-x 1 root root 4096 Nov 5 13:30 .. [edit] vyos@router#

The correct path:

vyos@router# cat /run/dhclient/dhclient_eth0.lease Tue Nov 5 16:19:41 UTC 2024 reason='BOUND' interface='eth0' new_expiry='1730909981' new_dhcp_lease_time='86400' medium='' alias_ip_address='' new_ip_address='10.0.1.10' new_broadcast_address='10.0.1.255' new_subnet_mask='255.255.255.0' new_domain_name='' new_network_number='10.0.1.0' new_domain_name_servers='1.1.1.1' new_routers='10.0.1.1' new_static_routes='' new_dhcp_server_identifier='10.0.1.1' new_dhcp_message_type='5' old_ip_address='' old_subnet_mask='' old_domain_name='' old_domain_name_servers='' old_routers='' old_static_routes=''

github-actions bot added the current label Sep 29, 2024

github-actions bot assigned sarthurdev Sep 29, 2024

sarthurdev force-pushed the wlb_python branch 3 times, most recently from b43023b to a139da3 Compare October 8, 2024 08:05

sever-sever reviewed Oct 8, 2024

View reviewed changes

python/vyos/wanloadbalance.py Outdated Show resolved Hide resolved

sever-sever reviewed Oct 8, 2024

View reviewed changes

src/helpers/vyos-load-balancer.py Outdated Show resolved Hide resolved

sarthurdev force-pushed the wlb_python branch 4 times, most recently from f201240 to bc76249 Compare October 8, 2024 21:36

sarthurdev marked this pull request as ready for review October 9, 2024 08:32

sarthurdev requested a review from a team as a code owner October 9, 2024 08:32

sarthurdev requested review from dmbaturin, zdc, jestabro, sever-sever, c-po and fett0 October 9, 2024 08:32

sever-sever reviewed Oct 9, 2024

View reviewed changes

sarthurdev force-pushed the wlb_python branch from bc76249 to f8176d7 Compare October 9, 2024 20:15

github-actions bot added the conflicts label Oct 10, 2024

sarthurdev added 2 commits October 21, 2024 16:48

wlb: T4470: Migrate WAN load balancer to Python/XML

c51efbf

wlb: T4470: Support WLB op-mode commands

b740f3e

sarthurdev force-pushed the wlb_python branch from f8176d7 to b740f3e Compare October 21, 2024 14:49

github-actions bot removed the conflicts label Oct 21, 2024

sever-sever reviewed Nov 5, 2024

View reviewed changes

HollyGurza reviewed Nov 5, 2024

View reviewed changes

sever-sever requested changes Nov 5, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

wlb: T4470: Migrate WAN Load Balancer to XML/Python #4108

wlb: T4470: Migrate WAN Load Balancer to XML/Python #4108

sarthurdev commented Sep 29, 2024 •

edited

Loading

github-actions bot commented Sep 29, 2024 •

edited

Loading

github-actions bot commented Sep 29, 2024 •

edited

Loading

sever-sever Oct 9, 2024

sarthurdev Oct 9, 2024

github-actions bot commented Oct 10, 2024

github-actions bot commented Oct 21, 2024

github-actions bot commented Oct 21, 2024

sever-sever Nov 5, 2024

sever-sever commented Nov 5, 2024 •

edited

Loading

sever-sever Nov 5, 2024

HollyGurza Nov 5, 2024

sever-sever Nov 5, 2024


		from vyos.utils.process import run

		dhclient_lease = '/var/lib/dhcp/dhclient_{0}.lease'

wlb: T4470: Migrate WAN Load Balancer to XML/Python #4108

Are you sure you want to change the base?

wlb: T4470: Migrate WAN Load Balancer to XML/Python #4108

Conversation

sarthurdev commented Sep 29, 2024 • edited Loading

Change Summary

Types of changes

Related Task(s)

Related PR(s)

Component(s) name

Proposed changes

How to test

Smoketest result

Checklist:

github-actions bot commented Sep 29, 2024 • edited Loading

github-actions bot commented Sep 29, 2024 • edited Loading

sever-sever Oct 9, 2024

Choose a reason for hiding this comment

sarthurdev Oct 9, 2024

Choose a reason for hiding this comment

github-actions bot commented Oct 10, 2024

github-actions bot commented Oct 21, 2024

github-actions bot commented Oct 21, 2024

Details

sever-sever Nov 5, 2024

Choose a reason for hiding this comment

sever-sever commented Nov 5, 2024 • edited Loading

sever-sever Nov 5, 2024

Choose a reason for hiding this comment

HollyGurza Nov 5, 2024

Choose a reason for hiding this comment

sever-sever Nov 5, 2024

Choose a reason for hiding this comment

sarthurdev commented Sep 29, 2024 •

edited

Loading

github-actions bot commented Sep 29, 2024 •

edited

Loading

github-actions bot commented Sep 29, 2024 •

edited

Loading

sever-sever commented Nov 5, 2024 •

edited

Loading