Aggregation Not working as Expected

[vinc32]

Expected Behavior

Link Aggregation around the capacity of all connections. (40-50M)
Maybe there are some settings wrong in my config? (Router / Server load around 3-10%)

Current Behavior

It seems like connections are always getting slowed down.

ETH1 DSL - 11.0M

not able to test but around 11.0M (hetzner speedtest not working)

ETH2 Starlink 19.0M

Best server is http://fra.download.datapacket.com/10000mb.bin, running test:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
82 9536M 82 7834M 0 0 19.0M 0 0:08:20 0:06:51 0:01:29 15.2M

ETH3 5G - 10.9M

Best server is http://ams.download.datapacket.com/10000mb.bin, running test:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
55 9536M 55 5326M 0 0 10.9M 0 0:14:32 0:08:07 0:06:25 11.2M

Aggregation - 18.4M

Best server is http://speedtest.frankfurt.linode.com/garbage.php?ckSize=10000, running test:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1024M 0 1024M 0 0 18.4M 0 --:--:-- 0:00:41 --:--:-- 16.6M

Upload

seems to be good / ok - Starlink not using complete power but ok.

Specifications

OpenMPTCProuter version: openmptcprouter v0.61-snapshot-6.6 r0+26018
OpenMPTCProuter VPS version: Version：6.1.0-20-cloud-amd64 0.1030
OpenMPTCProuter VPS provider: Ionos
OpenMPTCProuter platform: x86_64 / esxi
Country: Germany

Settings

IPv6 - disabled
Master DSL
Default Proxy - Shadowsocks - Shadowsocks Rust not getting an IP.
Defualt VPN - OpenVPN TCP
MTCP over VPN - Wireguard
Gateway Ping - disabled
Congestion Control - BBR

[Ysurac]

It's seems in aggregation graph that it's really aggregated. So I would say you have something that limit speed.
When using shadowsocks-Rust, what do you have in System log ? It's an update or a new install ?

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days

[vinc32]

Hi Ysurac, sorry for my late response - struggled a little the last month.
Tested a lot and still no aggregation working like it should.

Tests in Mbps Test with iperf 90 seconds

all 3 Connections enabled -> Down 220 / UP 150
DSL Solo -> Down 80 / UP 40
Starlink Solo -> Down 180 / UP 50
5G Solo -> Down 150 / UP 125

DSL (Master) + Starlink -> Down 170 / UP 50
Starlink (Master) + DSL -> Down 130 / Up 50
DSL (Master) + 5G -> Down 140 / UP 115
5G (Master) + DSL -> Down 155 / Up 110
Starlink (Master) + 5G -> Down 125 / UP 80
5G (Master) + Starlink -> Down 140 / UP 80

New Install, Shadowsuck Rust can't get public IP -> code below

Aug 6 20:09:57 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: /usr/libexec/openvpn-hotplug route-pre-down omr tun0 1420 0 10.255.252.2 255.255.255.0 init
Aug 6 20:09:57 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: net_addr_v4_del: 10.255.252.2 dev tun0
Aug 6 20:09:57 OpenMPTCProuter daemon.notice ttyd[8247]: [2024/08/06 22:09:57:2126] N: rops_handle_POLLIN_netlink: DELADDR
Aug 6 20:09:57 OpenMPTCProuter daemon.notice netifd: Network device 'tun0' link is down
Aug 6 20:09:57 OpenMPTCProuter daemon.notice netifd: Interface 'omrvpn' has link connectivity loss
Aug 6 20:09:57 OpenMPTCProuter daemon.notice netifd: Interface 'omrvpn' is now down
Aug 6 20:09:57 OpenMPTCProuter daemon.notice ttyd[8247]: [2024/08/06 22:09:57:2137] N: rops_handle_POLLIN_netlink: DELADDR
Aug 6 20:09:57 OpenMPTCProuter user.notice NET: hotplug (iface): action='remove' interface='tun0'
Aug 6 20:09:57 OpenMPTCProuter daemon.info ModemManager[32425]: hotplug: remove network interface tun0: event processed
Aug 6 20:09:57 OpenMPTCProuter user.notice firewall.omr-server: Firewall reload, set server part firewall reloading
Aug 6 20:09:57 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: /usr/libexec/openvpn-hotplug down omr tun0 1420 0 10.255.252.2 255.255.255.0 init
Aug 6 20:09:57 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: SIGUSR1[soft,connection-reset] received, process restarting
Aug 6 20:09:57 OpenMPTCProuter daemon.notice netifd: Interface 'omrvpn' is disabled
Aug 6 20:09:58 OpenMPTCProuter daemon.warn openvpn(omr)[31539]: WARNING: No server certificate verification method has been enabled. See http://openvpn.net/howto.html#mitm for more info.
Aug 6 20:09:58 OpenMPTCProuter daemon.warn openvpn(omr)[31539]: NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Aug 6 20:09:58 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: TCP/UDP: Preserving recently used remote address: [AF_INET]XXXXXXXXXX:65301
Aug 6 20:09:58 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: Attempting to establish TCP connection with [AF_INET]XXXXXXXX:65301
Aug 6 20:09:58 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: TCP connection established with [AF_INET]XXXXXXXXXX:65301
Aug 6 20:09:58 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: TCPv4_CLIENT link local: (not bound)
Aug 6 20:09:58 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: TCPv4_CLIENT link remote: [AF_INET]XXXXXXXXXXXX:65301
Aug 6 20:09:58 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: [server] Peer Connection Initiated with [AF_INET]XXXXXXXX:65301
Aug 6 20:09:58 OpenMPTCProuter daemon.warn openvpn(omr)[31539]: sitnl_route_save: rtnl: can't get ifname for index 0: No such device or address (errno=6)
Aug 6 20:09:58 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: TUN/TAP device tun0 opened
Aug 6 20:09:58 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: net_iface_mtu_set: mtu 1420 for tun0
Aug 6 20:09:58 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: net_iface_up: set tun0 up
Aug 6 20:09:58 OpenMPTCProuter daemon.notice netifd: Interface 'omrvpn' is enabled
Aug 6 20:09:58 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: net_addr_v4_add: 10.255.252.2/24 dev tun0
Aug 6 20:09:58 OpenMPTCProuter daemon.notice netifd: Network device 'tun0' link is up
Aug 6 20:09:58 OpenMPTCProuter daemon.notice netifd: Interface 'omrvpn' has link connectivity
Aug 6 20:09:58 OpenMPTCProuter daemon.notice netifd: Interface 'omrvpn' is setting up now
Aug 6 20:09:58 OpenMPTCProuter daemon.notice netifd: Interface 'omrvpn' is now up
Aug 6 20:09:58 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: /usr/libexec/openvpn-hotplug up omr tun0 1420 0 10.255.252.2 255.255.255.0 init
Aug 6 20:09:58 OpenMPTCProuter user.notice NET: hotplug (iface): action='add' interface='tun0'
Aug 6 20:09:58 OpenMPTCProuter daemon.info ModemManager[32602]: hotplug: add network interface tun0: event processed
Aug 6 20:10:02 OpenMPTCProuter user.notice firewall: Reloading firewall due to ifup of omrvpn (tun0)
Aug 6 20:10:03 OpenMPTCProuter user.notice firewall.omr-server: Firewall reload, set server part firewall reloading
Aug 6 20:10:03 OpenMPTCProuter daemon.notice openvpn(omr)[31539]: Initialization Sequence Completed
Aug 6 20:10:05 OpenMPTCProuter user.notice omr-tracker: Launched
Aug 6 20:10:05 OpenMPTCProuter daemon.info vnstatd[25648]: Info: SIGTERM received, exiting.
Aug 6 20:10:05 OpenMPTCProuter daemon.info vnstatd[1757]: Info: vnStat daemon 2.12 (pid:1757 uid:0 gid:0 64-bit, SQLite 3.41.2)
Aug 6 20:10:05 OpenMPTCProuter daemon.info vnstatd[1757]: Info: Data retention: 48 5MinuteHours, 4 HourlyDays, 62 DailyDays, 25 MonthlyMonths, -1 YearlyYears, 20 TopDayEntries
Aug 6 20:10:05 OpenMPTCProuter daemon.info vnstatd[1757]: Info: Monitoring (3): wan2 (1000 Mbit) wan1 (1000 Mbit) eth3 (10000 Mbit)
Aug 6 20:10:05 OpenMPTCProuter daemon.info vnstatd[1757]: Info: Interface "wan2" disabled.
Aug 6 20:10:05 OpenMPTCProuter daemon.info vnstatd[1757]: Info: Interface "wan1" disabled.
Aug 6 20:10:05 OpenMPTCProuter user.notice post-tracking-001-post-tracking: Tunnel up : Replace default route by 10.255.252.1 dev tun0 (was 100.64.0.1)
Aug 6 20:10:05 OpenMPTCProuter user.notice NET: hotplug (iface): action='remove' interface='ifb4eth3'
Aug 6 20:10:05 OpenMPTCProuter daemon.info ModemManager[2141]: hotplug: remove network interface ifb4eth3: event processed
Aug 6 20:10:05 OpenMPTCProuter daemon.info unbound: [26394:0] info: generate keytag query _ta-4f66. NULL IN
Aug 6 20:10:05 OpenMPTCProuter user.notice NET: hotplug (iface): action='add' interface='SQM_IFB_51468'
Aug 6 20:10:05 OpenMPTCProuter daemon.info ModemManager[2324]: hotplug: add network interface SQM_IFB_51468: event processed
Aug 6 20:10:05 OpenMPTCProuter user.notice NET: hotplug (iface): action='remove' interface='SQM_IFB_51468'
Aug 6 20:10:05 OpenMPTCProuter daemon.info ModemManager[2389]: hotplug: remove network interface SQM_IFB_51468: event processed
Aug 6 20:10:05 OpenMPTCProuter user.notice NET: hotplug (iface): action='add' interface='SQM_IFB_41811'
Aug 6 20:10:05 OpenMPTCProuter daemon.info ModemManager[2416]: hotplug: add network interface SQM_IFB_41811: event processed
Aug 6 20:10:05 OpenMPTCProuter user.notice NET: hotplug (iface): action='remove' interface='SQM_IFB_41811'
Aug 6 20:10:05 OpenMPTCProuter daemon.info ModemManager[2646]: hotplug: remove network interface SQM_IFB_41811: event processed
Aug 6 20:10:05 OpenMPTCProuter user.notice NET: hotplug (iface): action='add' interface='ifb4eth3'
Aug 6 20:10:05 OpenMPTCProuter daemon.info ModemManager[2711]: hotplug: add network interface ifb4eth3: event processed
Aug 6 20:10:06 OpenMPTCProuter user.notice firewall.omr-server: Firewall reload, set server part firewall reloading
Aug 6 20:10:07 OpenMPTCProuter user.notice omr-tracker: Launching...
Aug 6 20:10:16 OpenMPTCProuter user.notice omr-tracker: Launched
Aug 6 20:10:17 OpenMPTCProuter user.notice omr-schedule-010-services: Can't find omr-bypass rules, restart omr-bypass...
Aug 6 20:10:17 OpenMPTCProuter user.notice omr-bypass: Starting OMR-ByPass...
Aug 6 20:10:36 OpenMPTCProuter daemon.notice netifd: wan2 (5228): udhcpc: sending renew to server 100.64.0.1
Aug 6 20:10:40 OpenMPTCProuter user.notice firewall.omr-server: Firewall reload, set server part firewall reloading
Aug 6 20:10:40 OpenMPTCProuter user.notice omr-bypass: Restart dnsmasq...
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: exiting on receipt of SIGTERM
Aug 6 20:10:40 OpenMPTCProuter user.notice omr-bypass: OMR-ByPass is running
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: started, version 2.90 cachesize 1000
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: DNS service limited to local subnets
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP conntrack no-ipset nftset auth cryptohash DNSSEC no-ID loop-detect inotify dumpfile
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: UBus support enabled: connected to system bus
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq-dhcp[1]: DHCP, IP range 10.255.1.4 -- 10.255.1.6, lease time 12h
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: using nameserver 127.0.0.1#5353
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: using only locally-known addresses for test
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: using only locally-known addresses for onion
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: using only locally-known addresses for localhost
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: using only locally-known addresses for local
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: using only locally-known addresses for invalid
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: using only locally-known addresses for bind
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: using only locally-known addresses for use-application-dns.net
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: using only locally-known addresses for lan
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: using 1 more local addresses
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: read /etc/hosts - 12 names
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq[1]: read /tmp/hosts/dhcp.cfg01411c - 2 names
Aug 6 22:10:40 OpenMPTCProuter daemon.info dnsmasq-dhcp[1]: read /etc/ethers - 0 addresses
Aug 6 20:10:45 OpenMPTCProuter user.notice omr-schedule-010-services: Set firewall on server vps