diff --git a/README.md b/README.md index a7a86c0..9d7dfba 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,47 @@ -My own version of the Beta project. +#About + +This repository houses the server and client code of a distributed network performance testing framework. Clients receive a daily testing schedule from the central “governor” server, and run NDT tests accordingly. Test data becomes part of MLab’s open data set. + +More about MLab: +http://www.measurementlab.net/ + +More about accessing MLab data: +https://github.com/m-lab/mlab-wikis/blob/master/HowToAccessMLabData.md + +##Original Use + +This framework was originally developed to measure the network performance within Alexandria City Public Schools. This project focused on mapping quantitative data, created by odroid devices serving as clients and placed inside the classrooms, to qualitative descriptions of network conditions based on teacher reports. Analysis of this coupled data was then used to draw conclusions about how connectivity impacts effective use of technology in the classroom. + +##Intended Use + +This testing framework is most effective in a situation that meets the following criteria: +Users are interested measuring end-to-end network performance +There is sufficient bandwidth that the tests run will not affect the results +Users are interested in testing several different locations + +#Quickstart + +##Requirements + +####Server + +* MongoDB +* Static IP or hostname accessible to clients + +####Clients + +* Internet connection + +##Process +To quickly begin creating performance data for your own network follow the steps below: + +1. Clone this repository. +2. Copy “client” folder onto your devices acting as clients. Follow instructions in client/SETUP.md to properly configure the client. +3. Copy “governor” folder onto the server. Follow instructions in governor/SETUP.md to properly configure the governor server. + +##Accessing Data +Clients generate random 16 byte test ID’s associated with each test. These test ID’s are stored in the database on the server mapped to the client ID of the specific client that ran the test. These test ID’s can be used as special flags for querying MLab data. + + + diff --git a/SETUP.md b/SETUP.md deleted file mode 100644 index f8d0df7..0000000 --- a/SETUP.md +++ /dev/null @@ -1,48 +0,0 @@ -## Setup client - -The client is a python script (client.py) intended to run on a small device, receiving its schedule from the governor server. - -### client.cfg - -Client.cfg contains the basic configuration information that client.py will use to request and receive a test schedule from the governor server. It also contains three IDs that define the device privately to the server. -``` -[Addresses] - -partnerID: 1 -groupID: 3 -deviceID: 7 -server_address: 52.91.156.102 -``` - -*partnerID* is an indicator intended to distinguish different users of the governor/client system. - -*groupID* distinguishes different groups of devices, for example at different locations. - -*deviceID* is a unique number for each device deployed by a given partner. - -*server_address* is the world routable IP address where your governor server is running. - -### Setting static IPs - -Put this stanza in /etc/network/interfaces - replacing the IPs with your network info: - -``` -auto eth0 -iface eth0 inet static -address 10.10.1.171 -netmask 255.255.0.0 -gateway 10.10.254.254 -broadcast 10.10.255.255 -dns-nameservers 10.5.0.10 10.5.0.20 -``` - -### Adding script as a startup service - -1) move 'acps-service' to /etc/init.d/ -2) enable it as a startup service: -$ sudo update-rc.d acps-service defaults - -### Connect to the OTI governor server - -ssh -i "oti-measuring-broadband.pem" ubuntu@52.91.156.102 - diff --git a/client.cfg b/client.cfg deleted file mode 100755 index e8703fd..0000000 --- a/client.cfg +++ /dev/null @@ -1,6 +0,0 @@ -[Addresses] - -partnerID: 1 -groupID: 3 -deviceID: 8 -server_address: 52.91.156.102 \ No newline at end of file diff --git a/client/SETUP.md b/client/SETUP.md new file mode 100644 index 0000000..cfa935b --- /dev/null +++ b/client/SETUP.md @@ -0,0 +1,86 @@ +#Setup client + +The client is a python script (client.py) intended to run on a small device, receiving its testing schedule from the governor server. + +##client.cfg + +Client.cfg contains the basic configuration information that client.py will use to request and receive a test schedule from the governor server. It also contains three IDs that define the device privately to the server. + +``` +[Addresses] + +partnerID: 1 +groupID: 3 +deviceID: 7 +server_address: 52.91.156.102 +client_path: /home/odroid/ACPS/ +``` + +*partnerID* is an indicator intended to distinguish different users of the governor/client system. +*groupID* distinguishes different groups of devices, for example at different locations. +*deviceID* is a unique number for each device deployed by a given partner. +*server_address* is the world routable IP address where your governor server is running. +*client_path* is the absolute path to the client code + +##Setting Variables in the Configuration File + +The client script depends on absolute paths. This path needs to be defined in the configuration file. If the absolute path to the client script (client.py) is /home/odroid/ACPS/client/client.py, then the client_path variable should be set to /home/odroid/ACPS. + +Make sure the group ID and device ID in the client configuration file (client.cfg) are what you intended for this device. The governor authenticates clients by checking that their client ID (the concatenation of its partner ID, group ID, and device ID) is in its database before assigning it a schedule. + +##Running NDT + +To be able to run NDT, you need to make sure your version of libjansson4 is updated. + +``` +$ sudo apt-get update +$ sudo apt-get install libjansson4 +``` + +##Timezone + +Governor server schedules in relation to the local time. To be sure that the scheduled tests are being run when you expect. Explicitly set the timezone on your client devices to your local time zone. + +``` +$ sudo dpkg-reconfigure tzdata +``` + +##Setting static IPs + +Put this stanza in /etc/network/interfaces - replacing the IPs with your network info: + +``` +auto eth0 +iface eth0 inet static +address 10.10.1.171 +netmask 255.255.0.0 +gateway 10.10.254.254 +broadcast 10.10.255.255 +dns-nameservers 10.5.0.10 10.5.0.20 +``` + +##Adding script as a startup service +1. Make sure the script is executable. + ``` + $ sudo chmod +x 'acps-service' + ``` +2. Move 'acps-service' to /etc/init.d/. While in the client directory: + ``` + $ sudo move 'acps-service' to /etc/init.d/ + ``` +3. Enable it as a startup service. + ``` + $ sudo update-rc.d acps-service defaults + ``` + +##Connect to the OTI governor server +``` +ssh -i "oti-measuring-broadband.pem" ubuntu@52.91.156.102 +``` + + + + + + + diff --git a/acps-service b/client/acps-service similarity index 100% rename from acps-service rename to client/acps-service diff --git a/client/client.cfg b/client/client.cfg new file mode 100755 index 0000000..55471a8 --- /dev/null +++ b/client/client.cfg @@ -0,0 +1,7 @@ +[Addresses] + +partnerID: 1 +groupID: 3 +deviceID: 8 +server_address: 52.91.156.102 +client_path: /home/odroid/ACPS/ \ No newline at end of file diff --git a/client.py b/client/client.py similarity index 98% rename from client.py rename to client/client.py index c054818..666a1f6 100755 --- a/client.py +++ b/client/client.py @@ -5,8 +5,9 @@ mySchedule= [] governor = sched.scheduler(time.time, time.sleep) config = ConfigParser.RawConfigParser() -clientPath= "/home/odroid/ACPS/" -configFile= clientPath+"client.cfg" +configFile= "client.cfg" + + ############################################ Helpers ############################################################# @@ -200,7 +201,8 @@ def send_testIDs(): talk_to_server(new_message) -config.read(configFile) +config.read(configFile) +clientPath= config.get("Addresses", "client_path") server_address=config.get("Addresses", "server_address") deviceID= config.get("Addresses", "deviceID") #clientID= "partnerID/groupID/deviceID" diff --git a/web100clt b/client/web100clt similarity index 100% rename from web100clt rename to client/web100clt diff --git a/server/SETUP.md b/server/SETUP.md new file mode 100644 index 0000000..e9bb8ce --- /dev/null +++ b/server/SETUP.md @@ -0,0 +1,68 @@ +#Setup governor +The governor is a python server (governor.py) intended to schedule tests run on small, remote devices. + +##governor.cfg + +Governor.cfg contains the basic configuration information that the governor server needs to assign schedules. + +``` +[Timing] +runs_per_day: 96 +database_name: ACPS +``` + +*runs_per_day* is the mean of an exponential distribution that is used to schedule about *runs_per_day* tests per day. +*database_name* is the name for the database you specified when setting up mongodb + +##Adding script as a startup service + +The server requires two things to be run on startup. +* Mongodb server in the background +* governor.py + +Instructions for how to run mongodb in the background can be found here: +https://docs.mongodb.org/manual/tutorial/manage-mongodb-processes/ + +To make the governor.py script run on start up: + +1. Write a simple script calling the governor script and printing the output into a log file in the “server” folder. + +``` +$ sudo vi governor_startup_script /etc/init.d/ +``` + +Within the file: + +```python +#!/bin/sh -e +python absolute/path/to/governor.py > absolute/path/to/governor.log +``` + +2. Enable it as a startup service. + +``` +$ sudo update-rc.d acps-service defaults +``` + +##Setting up the database + +By default the governor uses two collections: “devices” and “testIds”. Any device you want to be able to connect with the server should be put into the database. + +Device Schema + +``` +{ + deviceID : ‘1’, + groupID : ‘1’, + partnerID : ‘1’ +} +``` + +TestId Schema + +``` +{ + deviceID : ‘1/1/1’, + testId: ‘91312284cca5bcb38a353ae8c8ec9d4d’ +} +``` diff --git a/governor.cfg b/server/governor.cfg similarity index 75% rename from governor.cfg rename to server/governor.cfg index bbe3cfd..5440255 100755 --- a/governor.cfg +++ b/server/governor.cfg @@ -3,4 +3,5 @@ #rate of testing every 15 minutes runs_per_day: 96 +database_name: ACPS diff --git a/governor.py b/server/governor.py similarity index 99% rename from governor.py rename to server/governor.py index f13a352..c3e29e3 100755 --- a/governor.py +++ b/server/governor.py @@ -21,7 +21,7 @@ #################################### Database Code ##################################################################### client = MongoClient('localhost', 27017) -db = client.ACPS +db = client[config.get("Timing", "database_name")] device_collection = db.devices testID_collection= db.testIds