Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download speed issue #104

Closed
SoGooDFR opened this Issue Jul 11, 2018 · 93 comments

Comments

Projects
None yet
8 participants
@SoGooDFR
Copy link

SoGooDFR commented Jul 11, 2018

Expected Behavior

Get full of my speed connection in dowload or youtube, stream...

Actual Behavior

I can get my full speed connection in a speedtest (80-100mbps)
I get 4-6 mbps on youtube.
I get 4-6 mbps on twitch.

I get only 4-6mpbs when i downlad file from http://ovh.net/files or http://ping.online.net...

Steps to Reproduce the Problem

  1. Fresh install on both
  2. Enter the basic configuration
  3. See the results.

Specifications

  • OpenMPTCProuter version: openmptcprouter v0.27 r7308-ebe1216c7c
  • OpenMPTCProuter VPS version: < OpenMPTCProuter VPS 0.34 >
  • OpenMPTCProuter platform: (RPI2/RPI3/x86/x86_64) Intel NUC x64 (Celeron J3455)
@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Jul 11, 2018

From what platform do you download file ? what file do you download ? what is the command line used ? How do you check speed on youtube ?

@SoGooDFR

This comment has been minimized.

Copy link
Author

SoGooDFR commented Jul 11, 2018

Some files.dat. On youtube, right-click and select "Advanced Statistics". I check the speed connection of the VPS with iperf = 1-3G/s.

@fnoir

This comment has been minimized.

Copy link

fnoir commented Jul 15, 2018

I have same problem. OpenMPTCProuter use only the master connection for download :(

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Jul 15, 2018

Without info, I can't reproduce and fix anything...

I need:

  • info on the type of connections used (ADSL,3G,4G, Pigeon over IP,...)
  • "steps to reproduce problem" (a command used with a URL, a iperf test,...)
  • check if OpenMPTCProuter status page is ok (green) for all
  • check also if MPTCP kernel is used on the VPS (uname -a on the VPS)
  • VPS used (OVH, FirstHeberg, Vultz,...)
  • VPN used (glorytun TCP, glorytun UDP, MLVPN,...)
  • ShadowSocks is enabled ?
@fnoir

This comment has been minimized.

Copy link

fnoir commented Jul 15, 2018

Sorry,

  • info on the type of connections used (ADSL,3G,4G, Pigeon over IP,...)
    I use ADSL (master) and 4G

  • "steps to reproduce problem" (a command used with a URL, a iperf test,...)
    Download a file for exemple (http://ovh.net/files/)

  • check if OpenMPTCProuter status page is ok (green) for all
    OpenMPTCProuter status page is ok => All green

  • check also if MPTCP kernel is used on the VPS (uname -a on the VPS)
    Ok => Linux 4.14.24-mptcp-64056fa+ . Fresh install, no tunning

  • VPS used (OVH, FirstHeberg, Vultz,...)
    FirstHeberg GP1

  • VPN used (glorytun TCP, glorytun UDP, MLVPN,...)
    ShadowSocks -> TCP
    Glorytun -> UDP and ICMP

  • ShadowSocks is enabled ?
    Yes for TCP

I use Raspberry Pi 3 Model B Rev 1.2 (openmptcprouter v0.27 r7308-ebe1216c7c) for router at home

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Jul 15, 2018

Can you run, on OpenMPTCProuter, iperf3 -c bouygues.iperf.fr -P 10 -4 and give me the result of the last two SUM line ?
Also tell me the speed of your links. What are the ADSL and 4G FAI ?

@fnoir

This comment has been minimized.

Copy link

fnoir commented Jul 15, 2018

[SUM]   0.00-10.00  sec   110 MBytes  92.6 Mbits/sec    0             sender
[SUM]   0.00-10.00  sec  22.2 MBytes  18.6 Mbits/sec                  receiver

Free for both

@SoGooDFR

This comment has been minimized.

Copy link
Author

SoGooDFR commented Jul 15, 2018

For me :

  • Only one connection : 4G
  • Download file from ovh.net/files or ping.online.net
  • Status page OK (Green)
  • Kernel : Linux 4.14.24-mptcp-64056fa+
  • VPS : OVH
  • VPN : TCP - SS UDP - Glorytun UDP
  • SS Enable
  • Intel NUC with Celeron J3455

Iperf :

[SUM]   0.00-10.00  sec   110 MBytes  92.3 Mbits/sec    0             sender
[SUM]   0.00-10.00  sec  31.6 MBytes  26.5 Mbits/sec                  receiver

Edit:
Same bandwidth limitation with Shadowsocks disabled and only Glorytun UDP.

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Jul 15, 2018

What is the result of iperf3 -c bouygues.iperf.fr -P 10 -4 -R ?

@fnoir

This comment has been minimized.

Copy link

fnoir commented Jul 15, 2018

[SUM]   0.00-10.00  sec  53.8 MBytes  45.1 Mbits/sec    3             sender
[SUM]   0.00-10.00  sec  31.9 MBytes  26.8 Mbits/sec                  receiver
@SoGooDFR

This comment has been minimized.

Copy link
Author

SoGooDFR commented Jul 15, 2018

[SUM]   0.00-10.00  sec  56.6 MBytes  47.5 Mbits/sec  171             sender
[SUM]   0.00-10.00  sec  47.1 MBytes  39.5 Mbits/sec                  receiver

@fnoir

This comment has been minimized.

Copy link

fnoir commented Jul 16, 2018

I retry this morning :

[SUM]   0.00-10.00  sec  73.7 MBytes  61.8 Mbits/sec    0             sender
[SUM]   0.00-10.00  sec  54.3 MBytes  45.5 Mbits/sec                  receiver
@boundouny

This comment has been minimized.

Copy link

boundouny commented Jul 16, 2018

I don't know if this is can helf, but I found a same problem in overthebox's user, it's a loadbalancing ping problem ... If you want i can send an URL where somme people have the same problem and the solution...

they always have same configuration: 1 x 4G and 1 x ADSL conection

@nileio

This comment has been minimized.

Copy link

nileio commented Jul 16, 2018

i am facing the same problem as described above , and i still have no solution, i am in a similar config to @SoGooDFR and @fnoir . I am following this thread ! what is the solution you refer to @boundouny ?

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Jul 16, 2018

If you set Multipath TCP SYN retries to 5 in Network->MPTCP, this should give a little bit more usable speed in some cases. You can also try to disable Multipath TCP checksum.

@SoGooDFR

This comment has been minimized.

Copy link
Author

SoGooDFR commented Jul 16, 2018

@boundouny But i only use one connection : 4G, so we don't have any load balancing here.

@fnoir

This comment has been minimized.

Copy link

fnoir commented Jul 16, 2018

I try solutions but not working.

When i test my connection with speedtest.net => 25Mbps, i download a file with ovh.net/files/ =>12Mbps

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Jul 16, 2018

What file do you download and what is the command used ?

@fnoir

This comment has been minimized.

Copy link

fnoir commented Jul 16, 2018

I use curl http://ovh.net/files/1Gio.dat --output 1Gio.dat => Current speed: 1445k

Speedtest:
speedtest

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Jul 16, 2018

You should use: curl http://ovh.net/files/1Gio.dat -o /dev/null

@fnoir

This comment has been minimized.

Copy link

fnoir commented Jul 16, 2018

On Windows, i use curl http://ovh.net/files/1Gio.dat -o nul => Current speed: 1398k

@catslab

This comment has been minimized.

Copy link

catslab commented Jul 16, 2018

speedtest.net is not accurate at all.
It shows me 4Mbps upload where I can only have 2 max.
Try nperf.com if you need web based test.

Repository owner deleted a comment from Scaff31 Jul 16, 2018

@SoGooDFR

This comment has been minimized.

Copy link
Author

SoGooDFR commented Jul 16, 2018

@catslab Ok, but for now, this is change nothing about the problem. With my 4G, i can get 120 mbps the night, but on navigation (stream, video, download,) i'm limited at 5-6 mpbs...

openmptcp_lol

@fnoir

This comment has been minimized.

Copy link

fnoir commented Jul 17, 2018

@catslab Same result with nperf.com

Download: 51.73 Mbps (av: 44.72)
Upload: 16.98 Mbps (av: 13.98)
Ping: 24 ms

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Jul 17, 2018

I think I find the issue:
sysctl -w net.ipv4.tcp_moderate_rcvbuf=1
It was set to 0. This will be fixed in next release that should be out tomorrow.

@SoGooDFR

This comment has been minimized.

Copy link
Author

SoGooDFR commented Jul 18, 2018

Thx man but i test your temporary fix and again in a speedtest it give to me 70-80 mbps. On navigation, i'm stuck with 20-25 mbps. I can see a littre improvement, so the bug/problem persist.

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Jul 18, 2018

After this change all networks services must be restarted (it's also possible to remove the line in /etc/sysctl.d/default.conf).

Without reproducible examples I can't test and fix anything. What is the test for "On navigation" ?

@SoGooDFR

This comment has been minimized.

Copy link
Author

SoGooDFR commented Jul 18, 2018

I apply the line fix, reload the sysctl config on modem and see the results.
Navigation test :

  • Stream (Twitch/Netflix/PrimeVideo)
  • Video (Youtube/Vimeo)
  • Download file (Direct download from ovh.net/files or ping.online.net)

For all, i will get the same limitation : 25 Mbps.

@linuzel

This comment has been minimized.

Copy link

linuzel commented Aug 1, 2018

Latency is stable at 33ms on wan1 and 39 on wan2.
I tried to do a ping test while doing a speed test and there is no large variation (33-40 ms)

I am using openmptcprouter v0.29.1-1 and OpenMPCTProuter VPS 0.40 but it was the same on every version I tried.

It seems to be a little bit faster with ShadowSocks off (and Glorytun TCP as fallback) :
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 41 1024M 41 423M 0 0 2658k 0 0:06:34 0:02:43 0:03:51 2756k

With Glorytun UDP as fallback it is again a little bit faster :
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 56 1024M 56 576M 0 0 3882k 0 0:04:30 0:02:32 0:01:58 5146k

This is a really strange behavior, I feared my router was the issue so I tried in an intel nuc and the result was similar.

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Aug 1, 2018

Interesting results.
Can you change method in /etc/shadowsocks-libev/config.json on the VPS to chacha20 and set same method on OpenMPTCProuter Shadowsocks interface ? Then test again (with shadowsocks enabled).
If not better, can you disable fast_open on both VPS and OMR, and test ?
It's all ideas I have for now ;)
Thanks

@linuzel

This comment has been minimized.

Copy link

linuzel commented Aug 1, 2018

It does not seems to change anything, I also tried salsa20 without any luck.

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Aug 1, 2018

You can try to change, in OMR MPTCP interface, the TCP congestion. By default it's olia, you can try cubic.

After that, the only think would be to run sysctl -a net.mptcp.mptcp_debug=1 on OMR, run omr-test-speed and check log about MPTCP.

@linuzel

This comment has been minimized.

Copy link

linuzel commented Aug 1, 2018

I tried a few times changing the congestion control without too much success but this time I did it on the VPS (I don't know why I haven't thought about this before) via :
sysctl -w net.ipv4.tcp_congestion_control=cubic

The result is a lot better :
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1024M 100 1024M 0 0 5891k 0 0:02:57 0:02:57 --:--:-- 5380k

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Aug 1, 2018

OK. A bit better.
Can you test BBR TCP Congestion also ?
On the VPS modprobe tcp_bbr && sysctl -w net.ipv4.tcp_congestion_control=bbr and BBR on OpenMPTCProuter too.

Seems to be better with my dirty 4G connection, but maybe not for you ?

@linuzel

This comment has been minimized.

Copy link

linuzel commented Aug 1, 2018

Ok, now we're talking !
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1024M 100 1024M 0 0 9653k 0 0:01:48 0:01:48 --:--:-- 9.8M

BBR does wonders here (with salsa20 method for shadowsocks).

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Aug 1, 2018

Very good (78Mb/s). Not sure if we can do better with your connection.
I will make some tests, put BBR as default (and maybe salsa20), and release a new version. This will permit more people to test :)

Thanks for all your tests and thanks to all others people that help here. Need more tests, but this seems to be the good way to solve this issue.

@SoGooDFR

This comment has been minimized.

Copy link
Author

SoGooDFR commented Aug 1, 2018

I can confirm that. The BBR tcp congestion solve the problem. On shadowsocks i use aes-128-gcm.

@fnoir

This comment has been minimized.

Copy link

fnoir commented Aug 1, 2018

The problem seems solved (BBR + aes-256-cfb). Good job !

Just a quick question, when the update will be released, how do we update the VPS? Use apt-get ?

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Aug 1, 2018

You will only have to run the OpenMPTCProuter VPS script again. This will update all and keep key.
New version should be out in the next 24 hours.

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Aug 2, 2018

OpenMPTCProuter v0.30 and OpenMPTCProuter VPS v0.43 are released with BBR set by default.

@Maudissure

This comment has been minimized.

Copy link

Maudissure commented Aug 2, 2018

It's worse for me with this version.

On Ookla speedtest and an ubuntu image download, only non master Wan connection is used with low rate.

With torrent download, bandwith used by all Wan ports is very low.

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Aug 2, 2018

Can you give me the result of omr-test-speed and omr-test-speed wan1 etc... for each interface ? Connections types (4G, xDSL, FTTH, IPoAC,...) ? and latency for each WAN (visible on OpenMPTCProuter status page) ?

@Maudissure

This comment has been minimized.

Copy link

Maudissure commented Aug 2, 2018

All Wan :

root@OpenMPTCProuter:~# omr-test-speed
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
1 10.0G 1 129M 0 0 2806k 0 1:02:16 0:00:47 1:01:29 3590k

Seems good with big variations (perharps the two LTE connections at this time).

Wan1 (ADSL) :

root@OpenMPTCProuter:~# omr-test-speed wan1
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 10.0G 0 14.4M 0 0 664k 0 4:22:54 0:00:22 4:22:32 665k

Ok at start (full bandwith) then :

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 10.0G 0 60.1M 0 0 572k 0 5:05:20 0:01:47 5:03:33 0

Completely crash

Wan 2 (LTE) :

**root@OpenMPTCProuter:~# omr-test-speed wan2
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
1 10.0G 1 189M 0 0 882k 0 3:18:04 0:03:39 3:14:25 1722k

2 10.0G 2 239M 0 0 846k 0 3:26:30 0:04:49 3:21:41 672k**

Big variation in download rates

Wan 3 (LTE):

root@OpenMPTCProuter:~# omr-test-speed wan3
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 10.0G 0 39.0M 0 0 925k 0 3:08:53 0:00:43 3:08:10 1741k

Seems better, more stable connection

Didn't find latency for each Wan on status page ...

Usualy i get approximatively 20Mbps for each LTE connection, max for my ADSL is 5.5Mbps.

While an Ubuntu image download, it's not quite good :

wan2 | Actuellement : | 6.17 Mbit/s(789.63 kB/s) | Moyenne : | 30.27 kbit/s(3.78 kB/s) | Pointe : | 8.58 Mbit/s(1.07 MB/s)
wan1 | Actuellement : | 5.17 Mbit/s(662.02 kB/s) | Moyenne : | 25.36 kbit/s(3.17 kB/s) | Pointe : | 10.3 Mbit/s(1.29 MB/s)
wan3 | Actuellement : | 8.53 Mbit/s(1.07 MB/s) | Moyenne : | 41.76 kbit/s(5.22 kB/s) | Pointe :

And eventually the router crash.

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Aug 2, 2018

If I ignore the crash (and use only real speed), aggregation seems quite good.
Can you make a screen capture of OpenMPTCProuter status page ? (remove VPS ip before posting)

What are the result if you set back olia in MPTCP interface and sysctl -w net.ipv4.tcp_congestion_control=olia on the VPS ?

@Maudissure

This comment has been minimized.

Copy link

Maudissure commented Aug 2, 2018

Hum, after crashing and RPI3B+ hard reboot, no more access to the router interface and no internet connection. The RPI3B+ booting is ok but the system seems completely down.

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Aug 2, 2018

Are you sure you don't have power problem ? If you use USB LTE on it they can use too much power for RPI.

@Maudissure

This comment has been minimized.

Copy link

Maudissure commented Aug 2, 2018

Nope, i have official RPI kit with 2.5A power block and my LTE devices are both routers linked to the home switch. All network home install is wired.

Fresh install in progress ...

@Maudissure

This comment has been minimized.

Copy link

Maudissure commented Aug 2, 2018

So, the status page cannot load.

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Aug 2, 2018

Can you give me the result of http://192.168.100.1/cgi-bin/luci/admin/system/openmptcprouter/interfaces_status (and remove your wan/vps ip if present)

@Maudissure

This comment has been minimized.

Copy link

Maudissure commented Aug 2, 2018

There :

Failed to execute call dispatcher target for entry '/admin/system/openmptcprouter/interfaces_status'.
The called action terminated with an exception:
/usr/lib/lua/luci/controller/openmptcprouter.lua:676: attempt to concatenate field 'RecursiveDnsServer' (a table value)
stack traceback:
/usr/lib/lua/luci/controller/openmptcprouter.lua:676: in function 'callback'
/usr/lib/lua/luci/model/uci.lua:295: in function 'foreach'
/usr/lib/lua/luci/controller/openmptcprouter.lua:571: in function </usr/lib/lua/luci/controller/openmptcprouter.lua:432>
(tail call): ?

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Aug 2, 2018

Seems you have other IPv6 announced, but this should not crash here :)

@Maudissure

This comment has been minimized.

Copy link

Maudissure commented Aug 2, 2018

Ok i just disabled IPV6 in Advanced configuration of openMPTCProuter page, and it works ! All ok but the message "Shadowsocks not working !"

And the pings :
ADSL : 14ms
LTE1 : 39ms
LTE2 : 38ms

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Aug 2, 2018

Check keys, and that chacha20 is used at both end.

@Maudissure

This comment has been minimized.

Copy link

Maudissure commented Aug 2, 2018

Keys and Chacha20 ok on router, how do i verify on VPS ?

the file openmptcprouter_config.txt on VPS contain :
Shadowsocks port: 65101
Shadowsocks encryption: aes-256-cfb
Your shadowsocks key:

Glorytun port: 65001
Glorytun encryption: chacha20

@Ysurac

This comment has been minimized.

Copy link
Owner

Ysurac commented Aug 2, 2018

in /etc/shadowsocks/config.json

@Maudissure

This comment has been minimized.

Copy link

Maudissure commented Aug 2, 2018

hum ! directory /etc/shadowsocks/config.json does not exist ...

It's seem that shadowsock was not installed by the script.

i retry an install of the vps with the script...

... well, still no /etc/shadowsocks/config.json BUT i have a /etc/shadowsocks-libev/config.json

{
"server":["[::0]", "0.0.0.0"],
"server_port":65101,
"local_port":1081,
"mode":"tcp_and_udp",
"key":"
"timeout":400,
"method":"chacha20",
"verbose":0,
"prefer_ipv6": false,
"fast_open": true,
"reuse_port": true,
"mptcp": true
}

Do i need the same method on the VPS and on the router ? i suppose so ...

OK after a VPS reboot, all good now .. :D

But download of the ubuntu image still very slow :( . Anyway, thanks for all.

@boundouny

This comment has been minimized.

Copy link

boundouny commented Aug 3, 2018

With the new version, i have downloading full power . Thanks for all.

Best regards

@Ysurac Ysurac closed this Aug 20, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.