Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test against static servers #763

Closed
wants to merge 2 commits into from
Closed

Test against static servers #763

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
67985db
Added static option for static server list
Oct 30, 2020
5166310
Updated README and help string added example-server.json
Oct 30, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,12 +75,13 @@ Usage
::

$ speedtest-cli -h
usage: speedtest-cli [-h] [--no-download] [--no-upload] [--single] [--bytes]
[--share] [--simple] [--csv]
[--csv-delimiter CSV_DELIMITER] [--csv-header] [--json]
[--list] [--server SERVER] [--exclude EXCLUDE]
[--mini MINI] [--source SOURCE] [--timeout TIMEOUT]
[--secure] [--no-pre-allocate] [--version]
usage: speedtest.py [-h] [--no-download] [--no-upload] [--single] [--bytes]
[--share] [--simple] [--csv]
[--csv-delimiter CSV_DELIMITER] [--csv-header] [--json]
[--load-servers-from-json SERVERS_JSON] [--list]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe something shorter like --json would suffice.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--json is already being used for specifying json output

[--server SERVER] [--exclude EXCLUDE] [--mini MINI]
[--source SOURCE] [--timeout TIMEOUT] [--secure]
[--no-pre-allocate] [--version]

Command line interface for testing internet bandwidth using speedtest.net.
--------------------------------------------------------------------------
Expand Down Expand Up @@ -108,10 +109,12 @@ Usage
--json Suppress verbose output, only show basic information
in JSON format. Speeds listed in bit/s and not
affected by --bytes
--load-servers-from-json SERVERS_JSON
Load servers from json file for static testing
--list Display a list of speedtest.net servers sorted by
distance
--server SERVER Specify a server ID to test against. Can be supplied
multiple times
--server SERVER Specify a server ID from --list servers to test
against. Can be supplied multiple times
--exclude EXCLUDE Exclude a server from selection. Can be supplied
multiple times
--mini MINI URL of the Speedtest Mini server
Expand All @@ -126,6 +129,7 @@ Usage
--version Show the version number and exit



Python API
----------

Expand Down
16 changes: 16 additions & 0 deletions example-server.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"313.8447559412203": [
{
"url": "http://speedtest.glasfaser-ostbayern.de:8080/speedtest/upload.php",
"lat": "49.0167",
"lon": "12.0833",
"name": "Regensburg",
"country": "Germany",
"cc": "DE",
"sponsor": "R-KOM GmbH & Co. KG",
"id": "4404",
"host": "speedtest.glasfaser-ostbayern.de:8080",
"d": 313.8447559412203
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is distance, right? What is the source of that? Might be worth specifying in the readme. (I only know of https://c.speedtest.net/speedtest-servers-static.php which does not provide distance.)

}
]
}
206 changes: 108 additions & 98 deletions speedtest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

import os
import re
import json
import csv
import sys
import math
Expand Down Expand Up @@ -1224,132 +1225,139 @@ def get_config(self):

return self.config

def get_servers(self, servers=None, exclude=None):
def get_servers(self, servers=None, exclude=None, servers_json=None):
"""Retrieve a the list of speedtest.net servers, optionally filtered
to servers matching those specified in the ``servers`` argument
"""
if servers is None:
servers = []

if exclude is None:
exclude = []
if servers_json is None:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idk if I would nest the whole get_servers() function under this conditional. The code looks right at first glance (and functioned well enough when testing the added json functionality) but that doesn't mean it won't throw something else off.

It may be preferable to still get a list of servers and insert the static ones via JSON, then be able to list them with --list and select them with --server. This would also prevent nesting all the below under this if statement.

if servers is None:
servers = []

self.servers.clear()
if exclude is None:
exclude = []

for server_list in (servers, exclude):
for i, s in enumerate(server_list):
try:
server_list[i] = int(s)
except ValueError:
raise InvalidServerIDType(
'%s is an invalid server type, must be int' % s
)
self.servers.clear()

urls = [
'://www.speedtest.net/speedtest-servers-static.php',
'http://c.speedtest.net/speedtest-servers-static.php',
'://www.speedtest.net/speedtest-servers.php',
'http://c.speedtest.net/speedtest-servers.php',
]
for server_list in (servers, exclude):
for i, s in enumerate(server_list):
try:
server_list[i] = int(s)
except ValueError:
raise InvalidServerIDType(
'%s is an invalid server type, must be int' % s
)

headers = {}
if gzip:
headers['Accept-Encoding'] = 'gzip'
urls = [
'://www.speedtest.net/speedtest-servers-static.php',
'http://c.speedtest.net/speedtest-servers-static.php',
'://www.speedtest.net/speedtest-servers.php',
'http://c.speedtest.net/speedtest-servers.php',
]

errors = []
for url in urls:
try:
request = build_request(
'%s?threads=%s' % (url,
self.config['threads']['download']),
headers=headers,
secure=self._secure
)
uh, e = catch_request(request, opener=self._opener)
if e:
errors.append('%s' % e)
raise ServersRetrievalError()
headers = {}
if gzip:
headers['Accept-Encoding'] = 'gzip'

stream = get_response_stream(uh)
errors = []
for url in urls:
try:
request = build_request(
'%s?threads=%s' % (url,
self.config['threads']['download']),
headers=headers,
secure=self._secure
)
uh, e = catch_request(request, opener=self._opener)
if e:
errors.append('%s' % e)
raise ServersRetrievalError()

serversxml_list = []
while 1:
try:
serversxml_list.append(stream.read(1024))
except (OSError, EOFError):
raise ServersRetrievalError(get_exception())
if len(serversxml_list[-1]) == 0:
break
stream = get_response_stream(uh)

serversxml_list = []
while 1:
try:
serversxml_list.append(stream.read(1024))
except (OSError, EOFError):
raise ServersRetrievalError(get_exception())
if len(serversxml_list[-1]) == 0:
break

stream.close()
uh.close()
stream.close()
uh.close()

if int(uh.code) != 200:
raise ServersRetrievalError()
if int(uh.code) != 200:
raise ServersRetrievalError()

serversxml = ''.encode().join(serversxml_list)
serversxml = ''.encode().join(serversxml_list)

printer('Servers XML:\n%s' % serversxml, debug=True)
printer('Servers XML:\n%s' % serversxml, debug=True)

try:
try:
try:
root = ET.fromstring(serversxml)
except ET.ParseError:
e = get_exception()
raise SpeedtestServersError(
'Malformed speedtest.net server list: %s' % e
)
elements = etree_iter(root, 'server')
except AttributeError:
try:
root = ET.fromstring(serversxml)
except ET.ParseError:
e = get_exception()
raise SpeedtestServersError(
'Malformed speedtest.net server list: %s' % e
)
elements = etree_iter(root, 'server')
except AttributeError:
try:
root = DOM.parseString(serversxml)
except ExpatError:
e = get_exception()
raise SpeedtestServersError(
'Malformed speedtest.net server list: %s' % e
)
elements = root.getElementsByTagName('server')
except (SyntaxError, xml.parsers.expat.ExpatError):
raise ServersRetrievalError()

for server in elements:
try:
root = DOM.parseString(serversxml)
except ExpatError:
e = get_exception()
raise SpeedtestServersError(
'Malformed speedtest.net server list: %s' % e
)
elements = root.getElementsByTagName('server')
except (SyntaxError, xml.parsers.expat.ExpatError):
raise ServersRetrievalError()

for server in elements:
try:
attrib = server.attrib
except AttributeError:
attrib = dict(list(server.attributes.items()))
attrib = server.attrib
except AttributeError:
attrib = dict(list(server.attributes.items()))

if servers and int(attrib.get('id')) not in servers:
continue
if servers and int(attrib.get('id')) not in servers:
continue

if (int(attrib.get('id')) in self.config['ignore_servers']
or int(attrib.get('id')) in exclude):
continue
if (int(attrib.get('id')) in self.config['ignore_servers']
or int(attrib.get('id')) in exclude):
continue

try:
d = distance(self.lat_lon,
(float(attrib.get('lat')),
float(attrib.get('lon'))))
except Exception:
continue
try:
d = distance(self.lat_lon,
(float(attrib.get('lat')),
float(attrib.get('lon'))))
except Exception:
continue

attrib['d'] = d
attrib['d'] = d

try:
self.servers[d].append(attrib)
except KeyError:
self.servers[d] = [attrib]
try:
self.servers[d].append(attrib)
except KeyError:
self.servers[d] = [attrib]

break
break

except ServersRetrievalError:
continue
except ServersRetrievalError:
continue

if (servers or exclude) and not self.servers:
raise NoMatchedServers()
if (servers or exclude) and not self.servers:
raise NoMatchedServers()
else:
printer('Loading Servers from:\n%s' % servers_json, debug=True)
with open(servers_json) as json_file:
self.servers = json.load(json_file)

return self.servers


def set_mini_server(self, server):
"""Instead of querying for a list of servers, set a link to a
speedtest mini server
Expand Down Expand Up @@ -1744,11 +1752,13 @@ def parse_args():
help='Suppress verbose output, only show basic '
'information in JSON format. Speeds listed in '
'bit/s and not affected by --bytes')
parser.add_argument('--load-servers-from-json', dest='servers_json', type=PARSER_TYPE_STR,
help='Load servers from json file for static testing')
parser.add_argument('--list', action='store_true',
help='Display a list of speedtest.net servers '
'sorted by distance')
parser.add_argument('--server', type=PARSER_TYPE_INT, action='append',
help='Specify a server ID to test against. Can be '
help='Specify a server ID from --list servers to test against. Can be '
'supplied multiple times')
parser.add_argument('--exclude', type=PARSER_TYPE_INT, action='append',
help='Exclude a server from selection. Can be '
Expand Down Expand Up @@ -1903,7 +1913,7 @@ def shell():
if not args.mini:
printer('Retrieving speedtest.net server list...', quiet)
try:
speedtest.get_servers(servers=args.server, exclude=args.exclude)
speedtest.get_servers(servers=args.server, exclude=args.exclude, servers_json=args.servers_json)
except NoMatchedServers:
raise SpeedtestCLIError(
'No matched servers: %s' %
Expand Down