Dynamically assign master upon master fail or allow failover for master somehow. #3361

AGIinspired · 2024-05-17T07:47:44Z

Expected Behavior

"Failover" support for master interface also

Current Behavior

if master goes down, connection drops until its back up.

Specifications

OpenMPTCProuter version: OpenMPTCProuter v0.60
OpenMPTCProuter VPS version: latest
OpenMPTCProuter VPS provider: self hosted, Debian.
OpenMPTCProuter platform: RPI5

Is there a way to configure OMR such that if the master goes down there is still seemless failover? Its working with non-master. Is there a command to set master for example that I can put in a script to run when that interface goes down? (dont need the script, just thinking of solutions)

AGIinspired · 2024-05-17T09:08:45Z

okay I noticed its not necessarily if master goes down. Seems to be whichever connection is running the tunnel.

Ysurac · 2024-05-17T17:41:07Z

What is the VPN and Proxy used ? You can check in System->OpenMPTCProuter, "Wizard" tab, "Advanced settings" checkbox.
What do you have in Status->System log ?

AGIinspired · 2024-05-20T08:52:36Z

Using shadowsocksrust 2022 with openvpn TCP.

system log output when failover doesnt happen:


May 20 08:49:30 OpenMPTCProuter daemon.notice netifd: wan1 (9865): udhcpc: received SIGTERM
May 20 08:49:30 OpenMPTCProuter daemon.notice netifd: wan1 (9865): udhcpc: unicasting a release of 192.168.7.116 to 192.168.7.1
May 20 08:49:30 OpenMPTCProuter daemon.notice netifd: wan1 (9865): udhcpc: sending release
May 20 08:49:30 OpenMPTCProuter daemon.notice netifd: wan1 (9865): udhcpc: entering released state
May 20 08:49:30 OpenMPTCProuter daemon.notice netifd: wan1 (9865): Command failed: ubus call network.interface notify_proto { "action": 0, "link-up": false, "keep": false, "interface": "wan1" } (Permission denied)
May 20 08:49:30 OpenMPTCProuter daemon.notice netifd: Interface 'wan1' is now down
May 20 08:49:30 OpenMPTCProuter daemon.notice ttyd[24212]: [2024/05/20 08:49:30:0650] N: rops_handle_POLLIN_netlink: DELADDR
May 20 08:49:30 OpenMPTCProuter user.notice post-tracking-001-post-tracking: wan1 (eth1) switched off because wan1 may have ip issues, interface have no IPv4, interface have no IPv4 gateway, interface have no IPv6, interface have no IPv6 gateway
May 20 08:49:30 OpenMPTCProuter user.notice post-tracking-001-post-tracking: Delete default route to (server IP) dev eth1

Ysurac · 2024-05-20T16:37:47Z

What is the result of ip r when all is working ?
Maybe incorrect metric are set.

AGIinspired · 2024-05-21T07:08:26Z

default via 10.255.252.1 dev tun0
default metric 1
nexthop via 192.168.7.1 dev eth1 weight 100
nexthop via 192.168.8.1 dev eth2 weight 1
default via 192.168.7.1 dev eth1 metric 7
default via 192.168.8.1 dev eth2 metric 8
default via 192.168.9.1 dev wan3 metric 9
default via 10.255.252.1 dev tun0 metric 1500
10.255.252.0/24 dev tun0 proto kernel scope link src 10.255.252.2
10.255.252.0/24 dev tun0 scope link metric 1500
10.255.252.1 via 10.255.252.1 dev tun0
102.132.169.17 metric 1
nexthop via 192.168.7.1 dev eth1 weight 100
nexthop via 192.168.8.1 dev eth2 weight 1
127.0.0.0/8 dev lo proto static scope link metric 5
192.168.7.0/24 dev eth1 scope link metric 7
192.168.8.0/24 dev eth2 scope link metric 8
192.168.9.0/24 dev wan3 scope link metric 9

This is while aggregation is working well

Ysurac · 2024-05-21T07:32:17Z

And when it's not working well ?
Maybe the VPN take too long to detect any problem...

AGIinspired · 2024-05-21T07:45:39Z

This is after I turn off main interface running tunnel (also did ping before I turned it off):

64 bytes from 8.8.8.8: icmp_seq=25 ttl=118 time=36.7 ms
64 bytes from 8.8.8.8: icmp_seq=26 ttl=118 time=57.5 ms
^C
--- 8.8.8.8 ping statistics ---
49 packets transmitted, 26 received, 46.9388% packet loss, time 48833ms
rtt min/avg/max/mdev = 19.734/35.168/76.825/13.643 ms
root@OpenMPTCProuter:~# ip r
default via 10.255.252.1 dev tun0
default metric 1
nexthop via 192.168.7.1 dev eth1 weight 100 dead linkdown
nexthop via 192.168.8.1 dev eth2 weight 1
default via 192.168.8.1 dev eth2 metric 8
default via 192.168.9.1 dev wan3 metric 9
default via 10.255.252.1 dev tun0 metric 1500
10.255.252.0/24 dev tun0 proto kernel scope link src 10.255.252.2
10.255.252.0/24 dev tun0 scope link metric 1500
10.255.252.1 via 10.255.252.1 dev tun0
(server IP) via 192.168.8.1 dev eth2 metric 1
127.0.0.0/8 dev lo proto static scope link metric 5
192.168.8.0/24 dev eth2 scope link metric 8
192.168.9.0/24 dev wan3 scope link metric 9

And this is ip r (and ping) when I switch off the interface not running tunnel (This is after eth2 became default so I just switched off eth1 again) :

64 bytes from 8.8.8.8: icmp_seq=15 ttl=118 time=35.9 ms
64 bytes from 8.8.8.8: icmp_seq=16 ttl=118 time=41.7 ms
64 bytes from 8.8.8.8: icmp_seq=17 ttl=118 time=42.7 ms
^C
--- 8.8.8.8 ping statistics ---
17 packets transmitted, 17 received, 0% packet loss, time 16021ms
rtt min/avg/max/mdev = 19.017/37.197/86.874/15.991 ms
root@OpenMPTCProuter:~# ip r
default via 10.255.252.1 dev tun0
default via 192.168.8.1 dev eth2 metric 1
default via 192.168.8.1 dev eth2 metric 8
default via 192.168.9.1 dev wan3 metric 9
default via 10.255.252.1 dev tun0 metric 1500
10.255.252.0/24 dev tun0 proto kernel scope link src 10.255.252.2
10.255.252.0/24 dev tun0 scope link metric 1500
10.255.252.1 via 10.255.252.1 dev tun0
(server IP) via 192.168.8.1 dev eth2 metric 1
127.0.0.0/8 dev lo proto static scope link metric 5
192.168.8.0/24 dev eth2 scope link metric 8
192.168.9.0/24 dev wan3 scope link metric 9

AGIinspired · 2024-05-21T08:04:37Z

This is the output with both interfaces up and eth2 as default:

 root@OpenMPTCProuter:~# ip r
default via 10.255.252.1 dev tun0 
default via 192.168.8.1 dev eth2 metric 1 
default via 192.168.7.1 dev eth1 metric 7 
default via 192.168.8.1 dev eth2 metric 8 
default via 192.168.9.1 dev wan3 metric 9 
default via 10.255.252.1 dev tun0 metric 1500 
10.255.252.0/24 dev tun0 proto kernel scope link src 10.255.252.2 
10.255.252.0/24 dev tun0 scope link metric 1500 
10.255.252.1 via 10.255.252.1 dev tun0 
(server IP) metric 1 
        nexthop via 192.168.7.1 dev eth1 weight 100 
        nexthop via 192.168.8.1 dev eth2 weight 1 
127.0.0.0/8 dev lo proto static scope link metric 5 
192.168.7.0/24 dev eth1 scope link metric 7 
192.168.8.0/24 dev eth2 scope link metric 8 
192.168.9.0/24 dev wan3 scope link metric 9

After eth2 off:

root@OpenMPTCProuter:~# ip r
default via 10.255.252.1 dev tun0 
default via 192.168.7.1 dev eth1 metric 7 
default via 192.168.9.1 dev wan3 metric 9 
default via 10.255.252.1 dev tun0 metric 1500 
10.255.252.0/24 dev tun0 proto kernel scope link src 10.255.252.2 
10.255.252.0/24 dev tun0 scope link metric 1500 
10.255.252.1 via 10.255.252.1 dev tun0 
(server IP) metric 1 
        nexthop via 192.168.7.1 dev eth1 weight 100 
        nexthop via 192.168.8.1 dev eth2 weight 1 dead linkdown 
127.0.0.0/8 dev lo proto static scope link metric 5 
192.168.7.0/24 dev eth1 scope link metric 7 
192.168.9.0/24 dev wan3 scope link metric 9

Tried running:

root@OpenMPTCProuter:~# ip route replace default via 192.168.7.1 dev eth1 metric 1

which resulted in


root@OpenMPTCProuter:~# ip r
default via 10.255.252.1 dev tun0 
default via 192.168.7.1 dev eth1 metric 1 
default via 192.168.7.1 dev eth1 metric 7 
default via 192.168.9.1 dev wan3 metric 9 
default via 10.255.252.1 dev tun0 metric 1500 
10.255.252.0/24 dev tun0 proto kernel scope link src 10.255.252.2 
10.255.252.0/24 dev tun0 scope link metric 1500 
10.255.252.1 via 10.255.252.1 dev tun0 
(server IP) metric 1 
        nexthop via 192.168.7.1 dev eth1 weight 100 
        nexthop via 192.168.8.1 dev eth2 weight 1 dead linkdown 
127.0.0.0/8 dev lo proto static scope link metric 5 
192.168.7.0/24 dev eth1 scope link metric 7 
192.168.9.0/24 dev wan3 scope link metric 9

but still no internet

Then this is after it self corrects (about 1 min)


root@OpenMPTCProuter:~# ip r
default via 10.255.252.1 dev tun0 
default via 192.168.7.1 dev eth1 metric 1 
default via 192.168.7.1 dev eth1 metric 7 
default via 192.168.9.1 dev wan3 metric 9 
default via 10.255.252.1 dev tun0 metric 1500 
10.255.252.0/24 dev tun0 proto kernel scope link src 10.255.252.3 
10.255.252.0/24 dev tun0 scope link metric 1500 
10.255.252.1 via 10.255.252.1 dev tun0 
(server IP) metric 1 
        nexthop via 192.168.7.1 dev eth1 weight 100 
        nexthop via 192.168.8.1 dev eth2 weight 1 dead linkdown 
127.0.0.0/8 dev lo proto static scope link metric 5 
192.168.7.0/24 dev eth1 scope link metric 7 
192.168.9.0/24 dev wan3 scope link metric 9 
root@OpenMPTCProuter:~# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=118 time=35.2 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=118 time=28.2 ms

Ysurac · 2024-05-22T16:26:36Z

It self correct after an OpenVPN restart I think. You can try lower values in Services->OMR-Tracker Manager for omrvpn interface.

fareign · 2024-05-25T23:13:09Z

I find master is too high weight. if master not connect to the internet. omr cant connect to the server,although the other interface works very good, the network still very worst.so we need allow other interface connect to the server ,or a backup interface.

Ysurac · 2024-05-26T05:41:51Z

When a connection is down, it's removed from route table, so no issue with weight here.

fareign · 2024-05-27T00:43:49Z

When a connection is down, it's removed from route table, so no issue with weight here.

maybe you are right,if master no sim card or internet.it canbe remove, but a lot of time master can not be remove from route table.so this is the problem

fareign · 2024-05-27T01:19:51Z

I dont know what is the role of weight. but is real to reduce some work from master,

vempire-ghost · 2024-05-27T13:54:35Z

When a connection is down, it's removed from route table, so no issue with weight here.

maybe you are right,if master no sim card or internet.it canbe remove, but a lot of time master can not be remove from route table.so this is the problem

OMR tracker is quite inconsistent in removing a WAN when it is without internet but physically connected. Several times, it simply does not remove the route and stays in this no-connectivity situation. The only way I managed to solve this was by placing the master WAN behind another TPLINK router with failover, thus ensuring that the master never loses internet.

Ysurac · 2024-05-27T14:13:35Z

When there is an issue, everything is in Status->System log. Without this, nothing can be solve.

vempire-ghost · 2024-05-27T14:45:11Z

I already posted logs here #3122 (comment), the problem continue until today.

Ysurac · 2024-05-27T15:20:25Z

@vempire-ghost it's not related to this issue I think. And if this an issue, reopen the issue and add more logs.

AGIinspired added the help wanted label May 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dynamically assign master upon master fail or allow failover for master somehow. #3361

Dynamically assign master upon master fail or allow failover for master somehow. #3361

AGIinspired commented May 17, 2024

AGIinspired commented May 17, 2024

Ysurac commented May 17, 2024

AGIinspired commented May 20, 2024

Ysurac commented May 20, 2024

AGIinspired commented May 21, 2024

Ysurac commented May 21, 2024

AGIinspired commented May 21, 2024

AGIinspired commented May 21, 2024 •

edited

Ysurac commented May 22, 2024

fareign commented May 25, 2024

Ysurac commented May 26, 2024

fareign commented May 27, 2024

fareign commented May 27, 2024

vempire-ghost commented May 27, 2024

Ysurac commented May 27, 2024

vempire-ghost commented May 27, 2024

Ysurac commented May 27, 2024

Dynamically assign master upon master fail or allow failover for master somehow. #3361

Dynamically assign master upon master fail or allow failover for master somehow. #3361

Comments

AGIinspired commented May 17, 2024

Expected Behavior

Current Behavior

Specifications

AGIinspired commented May 17, 2024

Ysurac commented May 17, 2024

AGIinspired commented May 20, 2024

Ysurac commented May 20, 2024

AGIinspired commented May 21, 2024

Ysurac commented May 21, 2024

AGIinspired commented May 21, 2024

AGIinspired commented May 21, 2024 • edited

Ysurac commented May 22, 2024

fareign commented May 25, 2024

Ysurac commented May 26, 2024

fareign commented May 27, 2024

fareign commented May 27, 2024

vempire-ghost commented May 27, 2024

Ysurac commented May 27, 2024

vempire-ghost commented May 27, 2024

Ysurac commented May 27, 2024

AGIinspired commented May 21, 2024 •

edited