Troubleshooting general #75

adolfintel · 2017-10-12T04:43:59Z

If you need troubleshooting, send me an email at info@fdossena.com, or use this thread.
Please don't make new issues if you just need help.

moeller0 · 2017-10-19T08:52:42Z

Hi adolftintel,

first of, nice project!

I have two smallish nits to pick:

Please have a look at line 38 of speedtest_worker.js:
overheadCompensationFactor: 1048576/925000, //compensation for HTTP+TCP+IP+ETH overhead. 925000 is how much data is actually carried over 1048576 (1mb) bytes downloaded/uploaded. This default value assumes HTTP+TCP+IPv4+ETH with typical MTUs over the Internet. You may want to change this if you're going through your local network with a different MTU or if you're going over IPv6 (see doc.md for some other values)

This really is just a guestimate as the speedtest has no real idea about the true overhead on the link. The HTTP part will be under control of the speedtest, sure, and probably also the IP part (assuming you force IPv4) but how about TCP options? Also I believe you do not measure the actual MSS/MTU so this part really should be reported as additional data IMHO; so present the real measured HTTP/TCP/IP Goodput first and then also present estimates about gross rates at lower OSI layers, that way the user can choose which numbers to trust.

And lines 235:
dlStatus = ((speed * 8 * settings.overheadCompensationFactor)/1048576).toFixed(2) // speed is multiplied by 8 to go from bytes to bits, overhead compensation is applied, then everything is divided by 1048576 to go to megabits/s
and 351:
ulStatus = ((speed * 8 * settings.overheadCompensationFactor)/1048576).toFixed(2) // speed is multiplied by 8 to go from bytes to bits, overhead compensation is applied, then everything is divided by 1048576 to go to megabits/s

actually assume Mebibytes/Mebibits (base 2^10) instead of Megabytes/Megabits (base 1000), as far as I can see in networking all rates are traditionally defined is SI units. It would make it much easier to compare this speedtests results with other tests if the conventional measures would be used (or at least the correct unit descriptors)

Best Regards

adolfintel · 2017-10-19T09:05:12Z

Yes, the overhead is an estimate of course. I estimated it from my ADSL, which seems to be the most common type over here.
Also yes, I could add a MB/MiB setting.

moeller0 · 2017-10-19T09:26:27Z

Ah, I believe the fact that it is an estimate should be reported prominently to the enduser, so one can understand the results better.
By the way, have a look at https://github.com/moeller0/ATM_overhead_detector for getting empirical data on the true overhead on your ADSL link, it is way more complex than one naively assumes... Also the actual encapsulations in use are quite diverse, too diverse, I believe, to list them all in the results.
BTW, can you get the number of packets used for the test, because it would be quite helpful to also show the user the number of packets and the volume of data transferred per direction.

moeller0 · 2017-10-19T09:52:57Z

Side note: Overhead compensation/reporting is I believe quite an interesting feature for endusers, especially since many ISPs only "promise" gross rates an so the ability to get a decent estimate of the gross rate from a speed test is quite helpful. The challenge for doing this generically is simply that there are a lot of different options that are hard to deduce empirically from the speedtest server side, and making the user manually select the used encapsulation will mostly confuse the "target" audience of normal users (this is not meant as an offense, nobody except a few axperts should have to know this ;))

adolfintel · 2017-10-19T10:30:47Z

It's probably overkill.
A good compromise could be to ignore overhead and output in MiB.
Either way, it's a miniscule difference, I doubt anyone cares.

moeller0 · 2017-10-19T10:37:09Z

Sure, reporting the measured rate seems like a decent idea, and maybe add the IP-level rate as well (as anyone is bound to use IP to reach the server), but anything higher like ethernet/ATM-ADSL/PTM-VDSL2 will just be confusing. But the number of packets sent would be great ;). The difference between Mb and Mib (100 - 100*1000^2/1024^2) = 4.6% will be ignored by a lot of users (those that look at the order of magnitude of the test result and their contracted rates) but there are users who will obsess over roughly 5%. Whether one wants to cater to those is i different question though ;).

geekism · 2018-01-09T03:47:37Z

Not really a issue, But I love the HTML5 Speedtest, I have about 50 mirrors. And i wanted something to show the test records from using 'example-telemetry.html'. So i wrote one.. You can view it here..

http://code.justla.me/index.php?file=php/html5speedtesttelemetry.php

adolfintel · 2018-01-09T08:36:17Z

@geekism I have an almost identical code that I use on my server. I haven't published it because it's very basic and generates very long pages. If you have time, you could make some page system and send a PR.

Also, a couple of suggestions for improving your page:

you can mix HTML and PHP, you don't need to echo HTML tags
don't ever ever ever use px, use em or another modern unit

cayxxx · 2018-01-17T09:52:34Z

Hello , thanks of your project . I've run it on my http website very well.
But recently I turned it to https , the speedtest page became abnormal. IP Address and Jitter cannot be displayed ,and the speed became very very slow.
How can i fix it ?

adolfintel · 2018-01-17T11:30:36Z

@cayxxx Do you get an error 500 if you try to open getIp.php or any other PHP file?

cayxxx · 2018-01-17T14:08:12Z

@adolfintel Thank you for reminding! It works perfectly after I installed php. But I am wondering why it could work before without php.

adolfintel · 2018-01-17T14:09:29Z

With the default settings, the test requires PHP. It's possible that your web server was configured to use PHP only with HTTP, and when you used HTTPS something changed.

mattorola7 · 2018-01-25T16:29:19Z

I really like this project. I would like to contribute back a command line client (instead of web browser) that could perform speed tests against the server. Has anybody developed a command line client for it already before we embark on the task?

adolfintel · 2018-01-25T16:30:48Z

This is a speedtest for browsers, if you can use the command line, iperf is a better solution

mattorola7 · 2018-01-25T16:44:57Z

We probably have a corner use case. I thought iPerf would enter the discussion :) I've been using iperf but I don't think it is possible to do exactly what we are trying to accomplish with it. If iperf would stream its JSON output rather than only output it at the end of the test, it would make it easier for us to build a "speed test API". Ultimately, we want to animate the speedtest result, even though it is run behind the scenes between server to server. This animation is just not going to happen in enough "real time" as we hoped.

adolfintel · 2018-01-25T16:49:42Z

Hmm, I don't know much about iperf, but I really can't recommend this project for your use case.
This test runs over HTTP, the result is not as accurate as iperf, and also the amount of processing that has to be done on both sides is greater by at least an order of magnitude, which means that if you have a really fast link like a 10 Gbit/s, you'll get a completely wrong result.
If you still want to use this, there is nothing stopping you, but I strongly recommend something else. Look at iperf's documentation, maybe there is a real time output that you can parse.

ketterling · 2018-03-08T20:36:03Z

Browser choice makes a difference. For a client, I'm using a recent iMac, and the server is on our local network (all gigabit Ethernet). I noticed that using Firefox, my reported download/upload speeds were consistently 990/600 Mbps. After troubleshooting for awhile, I tried Chrome and consistently get 995/995 Mbps. Something to consider if you're testing a local network.

adolfintel · 2018-03-08T20:39:38Z

@ketterling Yes, some browsers are a bit heavier on the CPU than others, therefore the speed may be limited on very fast connections.
Still, I wouldn't recommend this tool to test a local network, iperf would be better unless you absolutely want to run the test from a browser

ketterling · 2018-03-09T01:12:18Z

@adolfintel I was trying to understand what appeared to be a slow speed test for my new 200Mbit Internet service provided by my cable company. Having a solution such as yours (great solution, by the way) allowed me to test over the Internet and also to perform the same test locally and across my firewall. With this test, I was able to conclusively pinpoint my firewall, and a bit of trial and error pointed me to a particular setting. I'm very grateful for your solution!
And honestly, I'd never heard of iperf until finding this page, but I'll be reading up about it tonight.

niko2 · 2018-07-18T00:38:45Z

Great tool @adolfintel !

I would like to share a strange behavior with Chrome (whatever OS I use, Linux, android or Win) : each time it runs, my web server (apache mpm_event php7-fpm) answers (exactly and always) 3 bad requests atfer test is finished. I don't have this behavior with Firefox. I tried to disable parallel_downloading in chrome flags but got poor speed results and same apache errors:

access_log
xx.yy.zz.qq - [18/Jul/2018:01:53:22 +0200] "POST /empty.php?r=0.37227131738601105 HTTP/1.1" 400 226 "http://**********/speedtest_worker.js" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
error_log
[Wed Jul 18 01:53:44.285925 2018] [proxy_fcgi:error] [pid 8111:tid 139725453113088] (70008)Partial results are valid but processing is incomplete: [client xx.yy.zz.qq:57824] AH01075: Error dispatching request to : (reading input brigade), referer: http://**********/speedtest_worker.js

looking at the log timestamp, you may notice the access log with error 400 is older than telemetry.php (which is not shown here but trust me) but only appears at the same time as the php error, which is stamped after telemetry.php. So it looks like, the 3 requests are blocked somewhere in apache and never finish until error. There is about 35s between the requests and their php errors, which is not related to any (apache or php) timeout configured.

Strange isn't it ?

adolfintel · 2018-07-18T05:14:39Z

@niko2 I'm 99% sure that those are the incomplete upload XHRs that have been terminated at the end of the test. Nothing to worry about I'd say. Is it causing you problems?

niko2 · 2018-07-18T21:17:24Z

thank you @adolfintel for your quick answer; no it is not causing me problems except flooding my error logs which I prefer to avoid. Do you have any clue about the different behavior between Chrome and Firefox ?

adolfintel · 2018-07-19T05:10:10Z

@niko2 I don't know the implementation details. I guess Firefox just closes the socket so the server treats it as a request that the client failed to send, but Chromium just leaves it hanging. Pure speculation of course, but if you're interested you can use wireshark to see exactly what's being sent (or not sent).

GuojlWeb · 2018-07-19T06:45:28Z

Why can not upload when speed is detected in mobile browser?

adolfintel · 2018-07-19T07:38:44Z

@GuojlWeb what browser?

sistemabc · 2018-08-03T06:42:36Z

hey I can't get ISP data, there is written "unknown ISP", can you help me? @adolfintel

dunklesToast · 2018-08-03T06:44:40Z

@sistemabc Do you use this in your local network or is it hosted on a server outside your networ? If it’s in your local network the Speedtest can’t determine your ISP since we use the IP for that.

sistemabc · 2018-08-03T06:54:25Z

So how do I run so I get data ISP? I mean when running in the browser, is it "localhost/speedtest/" ?

adolfintel · 2019-04-30T17:44:53Z

@Ressy66 if you're not using garbage.php then you don't need it, however, you need to point the download test to your 10mb.dat file (see url_dl in customizing test settings in the wiki). Keep in mind that paths are relative to the JS file.

Ressy66 · 2019-04-30T20:23:58Z

@adolfintel Hi, yes I have done that, makes no difference still, download max speed is 0.10mbps when I should have 50-ish mbps like I get on iperf3, any other sugestions as to whats going on? or on how to debug this?

adolfintel · 2019-05-01T04:58:32Z

@Ressy66 Yes, press F12 and go to network, then run the test. Do you see a bunch of failed requests during the download test?

Halfeez92 · 2019-05-02T05:55:29Z

@Halfeez92
Your first issue is easily solved by changing overheadCompensationFactor to a lower value (the default of 1.06 is meant to compensate for the stack of a typical DSL connection, set it to 1 for raw throughput). See the wiki page on changing the test parameters if you don't know how to set it.

Your second issue requries some tweaking in getIP.php. Lines 10-19 detect the most typical HTTP headers added by gateways, firewalls, proxies, ... You need to find out the name of the HTTP header that your gateway is using to forward the original IP address and put it in that detection code. You can find it in the documentation of your gateway, or you can simply sniff HTTP traffic forwarded by the gateway with wireshark and look at the headers.

I am using Fortigate as my gateway. Guess need to study it first, you dont have any idea what is the Fortigate header right?

adolfintel · 2019-05-02T06:18:17Z

@Halfeez92 I don't know, sorry. The easiest way to figure it out is probably to sniff the HTTP traffic and see what headers it adds.

cnickle321 · 2019-06-05T11:57:38Z

Inactivity is the measure of postponement (or time) it takes to send data starting with one point then onto the next. Inertness is for the most part estimated in milliseconds or ms. It might likewise be alluded to as a ping rate.
For More Info:- https://testmyinternetspeed.org/ko/download-speed-test.html

baroescoeg · 2019-06-24T22:35:07Z

Hello
I have implemented it on my local server
when I speedtest, the results for Download 100Mbps, (According to 100Mbps LAN capability)
But the upload results only reach 20Mbps

Has uploading been set in the script / code at a limit of only 20Mbps?

Thank you

adolfintel · 2019-06-25T05:06:52Z

@baroescoeg There is no limit, and certainly not that low. You need to find out what's slowing it down: it could be a server problem (for instance, it's underpowered, like a raspberry), or a network problem (firewall, congestion, throttling, ...)
If you give me a link to your test I can make sure you configured it correctly

baroescoeg · 2019-06-25T05:22:07Z

@baroescoeg There is no limit, and certainly not that low. You need to find out what's slowing it down: it could be a server problem (for instance, it's underpowered, like a raspberry), or a network problem (firewall, congestion, throttling, ...)
If you give me a link to your test I can make sure you configured it correctly

@adolfintel Thank you for your reply
My local server uses a Mini PC (OrangePi-PC)
For the firewall I haven't changed it, or just a normal setting
System Operation, I use Ubuntu 16.04.6 LTS

For the speedtest link I can't give it
because I only made this for my local network

Is there anything I can do, thank you

adolfintel · 2019-06-25T05:24:13Z

@baroescoeg I'm pretty sure the OrangePi is underpowered, at least for apache, but let's make sure.
Run htop in the server and start the test, do you see high CPU usage on at least 1 of the cores?

baroescoeg · 2019-06-25T05:32:35Z

@adolfintel I check on htop, when doing a test (speedtest)
There is no high CPU usage, even in 1 core there is no high CPU usage only reaching around 15%

adolfintel · 2019-06-25T08:37:49Z

@baroescoeg 15% on a single core or globally?

baroescoeg · 2019-06-25T13:01:00Z

@adolfintel globally 15%, single core 56% for download, single core 5% for upload (only reach 20Mbps)

adolfintel · 2019-06-25T14:21:10Z

@baroescoeg That's weird, there's got to be something slowing you down.
Do you get full speed if you transfer a file via FTP or if you use iperf?

adolfintel · 2019-06-30T06:37:54Z

I'm closing this thread and making a new one because it's becoming too long

adolfintel · 2019-07-06T10:29:58Z

@robertjoh001 no spamming

muellerjoel · 2021-08-10T21:58:44Z

I have installed librespeed on a archlinux server with a 1 Gbit/s Uplink and sofar the fulltest is working but the results show me no ip-adresses.

It‘s this becaus of uBlock or Ghostery or need to install some additional software on the server?

If want to test it www.speedtesting.it have also a speedtest.net and other stuff running!

How i can protect the subfloder results with a htaccess file? I get a 500 error or nothing. Htacces is working on same apache server at other vhosts.

adolfintel · 2021-08-11T05:53:10Z

@Morta1985
You don't need an .htaccess file for the results folder, there's nothing that can be downloaded there, but if you want to add it you can, just allow access to the files in there and disallow listing of the directory.

About the IP address, try to open http://yourserver/backend/getIP.php?isp=true, see if you get an error.

muellerjoel · 2021-08-11T06:05:56Z

https://speedtesting.it/backend/getIP.php?isp=true

Seems to work on the iphone i will look on the Laptop later.

adolfintel · 2021-08-11T06:11:24Z

Works for me as well

muellerjoel · 2021-08-11T06:16:38Z

but still in the stats.php 0.0.0.0 as ip in every test. Can i enter a access token of ipinfo.io? Federico Dossena ***@***.***> schrieb am Mi. 11. Aug. 2021 um 08:11:

…

Works for me as well — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#75 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASKSDY65AW7IO5LT3IYUL6LT4IIBPANCNFSM4D6Z27BQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

adolfintel · 2021-08-11T06:24:05Z

Did you turn this on?

speedtest/results/telemetry_settings.php

Line 10 in eca7f06

$redact_ip_addresses = false;

Contact me at info@fdossena.com for the API key (only needed if you to more than ~500 tests/day)

muellerjoel · 2021-08-11T06:51:31Z

I fixed it with to turn false. Thanks 🙏 Federico Dossena ***@***.***> schrieb am Mi. 11. Aug. 2021 um 08:24:

…

Did you turn this on? https://github.com/librespeed/speedtest/blob/eca7f06e217b739e2c6cb14f3f4b4da9ea9be921/results/telemetry_settings.php#L10 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#75 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASKSDY5EZO5D3KGUWEIGZALT4IJRDANCNFSM4D6Z27BQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

ciroiriarte · 2021-09-13T22:08:28Z

Is this still on? :)

I'm seeing HTTP 400 errors in every test, would like to report it here before opening another issue:

[13/Sep/2021:16:16:56 -0500] "POST /backend/empty.php?r=0.6879673611457933 HTTP/1.1" 400 0 "http://speedtest.mydomain.com/speedtest_worker.js?r=0.722550769557101" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"

[13/Sep/2021:16:16:57 -0500] "POST /backend/empty.php?r=0.33775848954686616 HTTP/1.1" 400 0 "http://speedtest.mydomain.com/speedtest_worker.js?r=0.722550769557101" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"

[13/Sep/2021:16:16:58 -0500] "POST /backend/empty.php?r=0.4655761021776039 HTTP/1.1" 400 0 "http://speedtest.mydomain.com/speedtest_worker.js?r=0.722550769557101" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"

[13/Sep/2021:16:16:58 -0500] "POST /results/telemetry.php?r=0.742802661032028 HTTP/1.1" 200 20 "http://speedtest.mydomain.com/speedtest_worker.js?r=0.722550769557101" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"

There are always 3 x HTTP 400 before closing the test. For me it's an issue, because I have fail2ban blocking hosts generating multiple 4xx errors.

It's running with nginx+php-fpm:

server {

        listen 80;
        listen [::]:80;

        server_name speedtest.mydomain.com;
        root /srv/www/vhosts/speedtest.mydomain.com;
        access_log /var/log/nginx/speedtest.mydomain.com_access.log;
        error_log /var/log/nginx/speedtest.mydomain.com_error.log error;
        server_tokens off;

        modsecurity on;
        modsecurity_rules_file /etc/nginx/modsec/main.conf;

        client_max_body_size 50M;

        fastcgi_index index.php;
        #location ~* \.php$ {
        location ~ [^/]\.php(/|$) {
                fastcgi_split_path_info ^(.+?\.php)(/.*)$;
                if (!-f $document_root$fastcgi_script_name) {
                        return 404;
                }
                fastcgi_param HTTP_PROXY "";
                fastcgi_pass unix:/run/php-fpm/www.sock;
                fastcgi_param   SCRIPT_FILENAME    $document_root$fastcgi_script_name;
                include         fastcgi_params;
        }
        location /results/ {
                index index.php;
        }
        location / {
                limit_except GET HEAD POST { deny all; }
        }
}

All this using the stock example-singleServer-full.html + sqlite

adolfintel · 2021-09-14T07:02:04Z

@ciroiriarte It's normal: when the upload test ends all connections get cancelled, so the server will receive an incomplete POST request.

ciroiriarte · 2021-09-14T18:49:35Z

@ciroiriarte It's normal: when the upload test ends all connections get cancelled, so the server will receive an incomplete POST request.

Thanks, adding an exception to my fail2ban rule.

adolfintel mentioned this issue Oct 20, 2017

Ping count is Different from ookla speed test #78

Closed

adolfintel added the READ THIS! label Oct 24, 2017

adolfintel closed this as completed Jun 30, 2019

librespeed deleted a comment Jul 6, 2019

mckaygerhard mentioned this issue Dec 13, 2019

telemetry documentation does not property have doumented #260

Closed

Troubleshooting general #75

Troubleshooting general #75

Comments

adolfintel commented Oct 12, 2017 • edited Loading

moeller0 commented Oct 19, 2017

adolfintel commented Oct 19, 2017

moeller0 commented Oct 19, 2017

moeller0 commented Oct 19, 2017

adolfintel commented Oct 19, 2017

moeller0 commented Oct 19, 2017

geekism commented Jan 9, 2018

adolfintel commented Jan 9, 2018

cayxxx commented Jan 17, 2018

adolfintel commented Jan 17, 2018

cayxxx commented Jan 17, 2018

adolfintel commented Jan 17, 2018

mattorola7 commented Jan 25, 2018

adolfintel commented Jan 25, 2018

mattorola7 commented Jan 25, 2018

adolfintel commented Jan 25, 2018

ketterling commented Mar 8, 2018

adolfintel commented Mar 8, 2018

ketterling commented Mar 9, 2018

niko2 commented Jul 18, 2018

adolfintel commented Jul 18, 2018

niko2 commented Jul 18, 2018

adolfintel commented Jul 19, 2018

GuojlWeb commented Jul 19, 2018

adolfintel commented Jul 19, 2018

sistemabc commented Aug 3, 2018

dunklesToast commented Aug 3, 2018

sistemabc commented Aug 3, 2018

adolfintel commented Apr 30, 2019 • edited Loading

Ressy66 commented Apr 30, 2019

adolfintel commented May 1, 2019

Halfeez92 commented May 2, 2019

adolfintel commented May 2, 2019

cnickle321 commented Jun 5, 2019

baroescoeg commented Jun 24, 2019

adolfintel commented Jun 25, 2019

baroescoeg commented Jun 25, 2019

adolfintel commented Jun 25, 2019

baroescoeg commented Jun 25, 2019

adolfintel commented Jun 25, 2019

baroescoeg commented Jun 25, 2019

adolfintel commented Jun 25, 2019

adolfintel commented Jun 30, 2019

adolfintel commented Jul 6, 2019

muellerjoel commented Aug 10, 2021 • edited Loading

adolfintel commented Aug 11, 2021

muellerjoel commented Aug 11, 2021

adolfintel commented Aug 11, 2021

muellerjoel commented Aug 11, 2021 via email

adolfintel commented Aug 11, 2021 • edited Loading

muellerjoel commented Aug 11, 2021 via email

ciroiriarte commented Sep 13, 2021 • edited Loading

adolfintel commented Sep 14, 2021

ciroiriarte commented Sep 14, 2021

adolfintel commented Oct 12, 2017 •

edited

Loading

adolfintel commented Apr 30, 2019 •

edited

Loading

muellerjoel commented Aug 10, 2021 •

edited

Loading

adolfintel commented Aug 11, 2021 •

edited

Loading

ciroiriarte commented Sep 13, 2021 •

edited

Loading