too low utils/internetspeed.py result if CPU <<< Internetspeed

[sanderjo]

utils/internetspeed.py is a nice tool to measure Internet speed.The result is presented in the SABnzbd Wrench.
It works nicely with normal CPU's and normal Internet speed. So with an i3 you can probably measure 1000 Mbps.

However if your Internetspeed is much more than your CPU, the measured speed is too low. And, note, while downloading, that CPU can achieve much higher than the incorrectly measured speed.

On a 8 Gpbs connection(even with a fast i9), utils/internetspeed.py measures 2 Gbps or so.
On a very slow CPU (like a NAS), and only 75 Mbps, utils/internetspeed.py just measures 5.9 Mbps

In both cases:

So:

• do we care? So far no complaints in the forums ... but Internet speeds are going up.
• if so, what kind of solution is good / acceptable. For example: iperf3 always gives the correct Internet speed. About iperf3: nice tool, but not installed by default, and it needs to be run as external program.

EDIT: I see the iperf3 as ad additional tool/measurement; the current Internet speed measurement as "always-tool", the iperf3 as addtional ... if available and for the niche cases (low CPU, high Internet)

OK case:
OK CPU (pystone 125657), with about 800 Mbps Internet speed:

sander@haring1:~/git/sabnzbd$ python3 sabnzbd/utils/internetspeed.py
DEBUG:root:Log level is DEBUG
Starting speed test:
...
Speed in MB/s: 82.38
Speed in Mbps: 663.16

with about the same speed from iperf3

$ iperf3 -c ams.speedtest.clouvider.net -p 5205 -t3  --connect-timeout 300 --format m | grep sender
[  5]   0.00-3.00   sec   312 MBytes   873 Mbits/sec   31             sender

Edge cases: low cpu, high Internet

Slow CPU ... much too low measured speed:

sander@nezha:~/git/sabnzbd$ python3 sabnzbd/utils/internetspeed.py
DEBUG:root:Log level is DEBUG
Starting speed test:
DEBUG:root:Basic measurement, with small download:
DEBUG:root:URL is https://sabnzbd.org/tests/internetspeed/5MB.bin
DEBUG:root:Downloaded bytes: 5242880
DEBUG:root:Duration in seconds: 6.787108
DEBUG:root:Speed in MB/s: 0.74
DEBUG:root:Internet Bandwidth = 0.74 MB/s (in 6.80 seconds)
Speed in MB/s: 0.74
Speed in Mbps: 5.93

... only 5.93 Mbps which is not correct.

... and perfect result with iper3

sander@nezha:$ iperf3 -c ams.speedtest.clouvider.net -p 5205 -4 -t5 | grep sender
[  5]   0.00-5.00   sec  49.9 MBytes  83.7 Mbits/sec  133             sender

... so 83 Mbps, which is correct.

Synology:

admin@DiskStation:/$ iperf3 -c ams.speedtest.clouvider.net -p 5205 -4 -t5 | grep sender
[  5]   0.00-5.00   sec  33.6 MBytes  56.4 Mbits/sec    0             sender

VPS with 2-3 Gbps connection:

root@superkoning:~/git/sabnzbd# python3 sabnzbd/utils/internetspeed.py
DEBUG:root:Log level is DEBUG
Starting speed test:
DEBUG:root:Internet Bandwidth = 61.84 MB/s (in 0.95 seconds)
Speed in MB/s: 61.84
Speed in Mbps: 497.83

... but with iperf3:

$ iperf3 -c ams.speedtest.clouvider.net -p 5205 -t3  --connect-timeout 300 --format m | grep sender
[  5]   0.00-3.00   sec  1.18 GBytes  3373 Mbits/sec  4647             sender

so 3373 Mbps.

[thezoggy]

routinely its been noted that the speed test we include in the wrench is unreliable (its fine as a crude test, sometimes just fine but also sometimes just wildly wrong and we dont really offer anything to figure out why). So we've pushed users to go just test with a speed testing site instead.

so my vote would just to remove it..

or move to using one of the speedtest via cli tools and leverage that instead:
https://www.ookla.com/articles/introducing-speedtest-cli
https://www.speedtest.net/apps/cli

side note, i personally run this docker to test connection periodically
speedtest-tracker - https://github.com/alexjustesen/speedtest-tracker

[sanderjo]

routinely its been noted that the speed test we include in the wrench is unreliable

What a nonsense. The Internet speed test has proven to be very useful in most cases.

(its fine as a crude test, sometimes just fine but also sometimes just wildly wrong and we dont really offer anything to figure out why).

See my proposal for extreme cases.

So we've pushed users to go just test with a speed testing site instead.

Only if people don't believe SAB Wrench.

so my vote would just to remove it..

Clear.

About the alternatives you mention: do they run on ... Synology (with all different archs)? Linux (dfferent archs)? Are they open source?

[thezoggy]

Re-reading my response I must apologize as I meant to call out the disk speedtest one is the 'unreliable' one not the speedtest. Which is not the topic at hand and I should have just not said anything there. Also I can see my response probably sounds a bit more harsh than it should be.

The speed test / disk test are things that i've seen cause more questions than answers. Why in support I've pushed for them to test outside of sab to get 2nd opinion. Lots of time people are just confused with speed test with the whole MB vs Mbps but generally it is not so off. The times that it is off, there seems to just be something funky with their setup and since they can't pinpoint it to sab it never makes it as a github issue.

I suggested the speedtest cli as a solution vs using iperf3 just because it might be easier for us to include or leverage if installed I recall iperf3 being hard to find updated binaries for all os and a pain to compile. There are quite a few speedtest via cli type things out there.. that one i linked is just one example that is pretty popular and they seem to provide it across all os/distro and seems like it would be easy to include if we really wanted to go that route.

[jeffgeorge]

Quick update since this came up again in Discord. Ookla's Speedtest CLI binaries are not FOSS and do not appear to be safe to redistribute, so those are right out. However, it looks like we may be able to use https://github.com/sivel/speedtest-cli/, which is a black-box Python implementation of a speedtesting service running against Ookla's public infrastructure.

As a bonus, they have what appears to be a pretty simple python api so we may be able to just declare them as a dependency and go nuts.

However, as good as that seems, my primary concern is that the opensource speedtest is not quite as reliable as ookla's first-party binaries. Plus, even with ookla's first party tools, I've personally had trouble always getting a reliable reading. Sometimes when it prioritizes by distance I get a crappier server that's technically closer to me but objectively worse, and even when I do get one that's good it's still hard to get the full sustained gigabit speeds I know my network can handle. . . and if our whole problem here is trying to get this to work more reliably on the upper-end of network connections, would swapping it out with speedtest really help?

[sanderjo]

"and if our whole problem here is trying to get this to work more reliably on the upper-end of network connections, would swapping it out with speedtest really help?"

I have a 2500 Mbps line. Proof it's working:

$ iperf3 -R -P30 -p 5207 -t5 -4  --format m -c ams.speedtest.clouvider.net  | grep SUM | grep receiver
[SUM]   0.00-5.00   sec  1.36 GBytes  2340 Mbits/sec                  receiver

I tried the speedtest python sample code from https://github.com/sivel/speedtest-cli/wiki#python-api:~:text=Python%20API-,import%20speedtest,-servers%20%3D%20%5B%5D

Result:

$ python3 speedtest_with_api.py 
{'download': 944272083.2462982, 'upload': 1417902448.9589882, 

So 944 Mbps. Not enough. https://www.speedtest.net/result/15226600728.png

With the GUI version, I get 2305 Mbps: https://www.speedtest.net/nl/result/15226611319 So like you say: the GUI version is better.

Reference: a few SABnzbd Wrench reports ... which can't report my full Internet speed either:

Internet Bandwidth 93.3 MB/s 746.4 Mbps
Internet Bandwidth 111.3 MB/s 890.4 Mbps
Internet Bandwidth 102.8 MB/s 822.4 Mbps
Internet Bandwidth 96.3 MB/s 770.4 Mbps

[sanderjo]

With this python code, I get the correct speed. Works on Linux (and thus docker), needs libiperf.so, which is provided by iperf3. See https://iperf3-python.readthedocs.io/en/latest/

$ python3
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import iperf3
>>> client = iperf3.Client()
>>> client.duration = 3
>>> client.num_streams = 20
>>> client.server_hostname = 'ams.speedtest.clouvider.net'
>>> client.port = 5206
>>> client.reverse=True
>>> result = client.run()
>>> print("Downstream speed [Mbps]", int(result.received_Mbps))
Downstream speed [Mbps] 2316