EmulaTor

EmulaTor is a script that generates Emulab configurations for performing experiments involving Tor.

EmulaTor aims to automate as many of the tedious details of Emulab configuration as possible while still allowing maximal customization.

EmulaTor is a project of the Georgetown University Security Lab.

Contact: aj499@georgetown.edu

Dependencies

EmulaTor has no dependencies other than Python 2.7.

Quickstart

• Clone the Git repo
• Run emulator.py to generate a configuration.
• Upload the generated tarball to /proj/YOUR_PROJECT on users.emulab.net and untar it.
• Upload the simulator.ns file generated by EmulaTor to the EmuLab website by clicking Modify Experiment on the page for your experiment.
• Swap your experiment in.
• Science!

Nodes and Topology

EmulaTor divides nodes into 5 categories:

• Authorities: These nodes run Tor instances configured as Directory Authorities. If you are using EmulaTor after June 2017 you will need to generate new key material for these nodes, as the key material provided with EmulaTor will have expired. See Useful Notes below for instructions.
• Servers: These nodes run nginx instances configured to serve files out of the location specified in the included nginx.conf file, /etc/nginx/sites/default. EmulaTor does not provide any files in this location by default; scripts (see Scripts below) should be configured to deploy files here if needed by your experiment.
• Exit Nodes: These nodes run Tor instances configured to run as exit relays. Their exit policy allows all traffic to all addresses on all ports.
• Non-exit Relay Nodes: These nodes run Tor instances configured to run as non-exit relays. (That is, their exit policy rejects all traffic.)
• Clients: These nodes run Tor instances configured to act as Tor clients. These nodes are also provided with cURL, as cURL can proxy requests through Tor (via --socks5-hostname localhost:SOCKS_PORT; see configuration.py for the value of SOCKS_PORT).

These nodes are all connected to a single LAN. Loss is implemented by placing an interposer node between the node and the LAN and setting loss on the link between the interposer and the node. See Invocation below for details on configuring loss.

See Setup Process below for details on how EmulaTor initializes these nodes.

Invocation

EmulaTor accepts the following command line arguments and flags. Note that while only -e and -y are mandatory, -j should be set in almost all circumstances. Some flags have both long and short versions; only the short versions of those flags are listed here. For more information run python emulator.py -h.

Arguments

• NAME - the name of the simulation; this is used to name the folder and tar-file created by EmulaTor.

Nodes

• -e EXITS - The number of exit nodes to create [MANDATORY]
• -y RELAYS - The number of non-exit relay nodes to create [MANDATORY]
• -a AUTHORITIES - The number of authority nodes to create
• -s SERVERS - The number of nginx server nodes to create
• -c CLIENTS - The number of client nodes to create

Loss

• -l NODE ... - A list of nodes and/or groups of nodes to make lossy. Nodes are named as follows, starting from 0: authorities: torthorityX, servers: serverX, exits: exitX, relays: relayX, clients: clientX. To make all nodes of a given type lossy, use the name of that type. (The type names are all, authorities, clients, exits, relays, non-exit-relays, and servers.) eg, to make the 2nd client node and all server nodes lossy, -l client1 servers.
• -r RATE ... - A list of loss rates to apply to the nodes and/or groups of nodes given to -l. Loss rates should be formatted with leading 0s. eg, for 10% loss, -r 1; for 1%, -r 01, etc. If one loss rate is given it will be applied to all the nodes listed with -l; otherwise a loss rate must be given for each node or group of nodes listed with -l. See below for examples of this behavior.

Misc Configuration

• -j PROJECT_NAME - The name of the EmuLab project under which the experiment will be run (ie the name of the folder in /proj you use)
• -i FILE ... - Copies the listed files or folders into the output folder for the experiment
• -v - Overwrite NAME and NAME.tar.gz if they already exist
• -o OS - Use a different OS for all nodes in the experiment. By default, Emulator loads all nodes with UBUNTU12-64-STD. See the EmuLab website for the full list of available OSs. Note that if you use -o to specify a different OS you must use -t to provide a version of Tor statically built for that OS, as the version of Tor packaged with Emulator is built against UBUNTU12-64-STD and is unlikely to work elsewhere.
• -t TOR - Use the specified Tor executable for all Tor nodes in the experiment. This option should be considered mandatory is -o is used. For instructions on creating a statically compiled version of Tor, see Useful Notes below.

Scripts

• -f SCRIPT-FILE - configure scripts as indicated in SCRIPT-FILE; see The Script File below for details
• --setup-script SETUP-SCRIPT - a setup script to run on all nodes
• --setup-script-start TIME - the time, in seconds, after simulation start to run SETUP-SCRIPT
• --setup-script-stop TIME - the time, in seconds, after simulation start to stop SETUP-SCRIPT
• --setup-script-client-only - run SETUP-SCRIPT on the client nodes only, instead of on all nodes The following options function analogously to the --setup-script options:
• --experimental-script EXPERIMENTAL-SCRIPT
• --experimental-script-start
• --experimental-script-stop
• --experimental-script-client-only
• --cleanup-script CLEANUP-SCRIPT
• --cleanup-script-start
• --cleanup-script-stop
• --cleanup-script-client-only

Scripts

Warnings

Due to EmuLab limitations, scripts should not use special (ie non-alphanumeric) characters in their filenames other than ., -, #, and _. Script filenames should also be relatively short, as long filenames can lead to EmuLab errors when loading the generated NS file.

The Script File

If you need more control over the execution of your scripts that is provided by the --<X>-script options, you may use a script file to specify scripts to be run by your experiment. For each script you wish to run you must include a block formatted as follows:

SCRIPT
/path/to/script.sh
TARGETS
START_TIME
STOP_TIME
END_SCRIPT

where TARGETS is a list of nodes and/or sets of nodes the script is to be run on, START_TIME is the start time of the script, and STOP_TIME is the stop time of the script, the latter two in seconds after simulation start. Acceptable values for TARGETS, which should be self-explanatory, are all, authorities, servers, exits, non-exit-relays, relays, and clients, as well as any node names (see discussion of node names under the -l option in Loss) eg

SCRIPT
/home/user/me/magic.sh
authorities client0 relays
900
7200
END_SCRIPT

Substitution with ##

When running scripts, it is sometimes useful to know the name of the node the script is executing on (eg for directing output to a separate log file for each machine). If a script contains ## in its filename a separate copy of the script will be made for each node the script is to be run on. In each copy of the script ## will be replaced in both the filename and body of the script with the name of the node that copy of the script will be run on.

Setup Process

EmulaTor-generated Emulab configurations bootstrap as follows:

1. At simulation start, the server nodes begin installing and configuring nginx and the authority nodes install and start Tor.
2. ~3 minutes after simulation start, the relays (exit and non-exit) install and start Tor.
3. ~5 minutes after simulation start, the clients install cURL and install and start Tor.
4. ~10-15 minutes after simulation start, the Tor network should have stabilized (Tor clients can take a surprisingly long time to bootstrap and create circuits) and be ready for use. Any scripts that involve Tor should be safe to use at this time.

configuration.py

configuration.py contains a variety of user-editable settings that you may use to change the behaviour of the experiments generated by Emulator. See the file itself for details.

Useful Notes

Hints and Warnings for Writing Scripts

• Emulab runs all scripts with /bin/csh, so any scripts that are not shell scripts should have appropriate shebang (#!) lines.
• Emulab scripts can use sudo freely.
• Redirecting output into a write-protected file with > will fail; pipe the output to tee run as root in order to avoid this issue. ie, echo test > /etc/test will fail, but echo test | sudo tee /etc/test will succeed.
• Emulab swallows ALL errors produced by running scripts; proofread your scripts carefully and insert logging (but see previous hint) as needed.

Statically Compiling Tor

These instructions adapted from those in the INSTALL file distributed with the Tor source code; these may become out of data as the Tor Project proceeds. Tor depends on libevent, openssl, and zlib. Begin by downloading the source code for each of these libraries, as well as the source code for Tor. (Emulator uses tor-0.2.6.9, libevent-2.0.22-stable, openssl-1.0.2c, and zlib-1.2.8; adjust the instructions below accordingly if you are using different versions.) These instructions assume that you have downloaded all the source tarballs to a folder called /static_tor. You must, of course, build Tor on a machine on the same OS you wish to run it on; allocating a single EmuLab node is useful for this purpose.

• cd /static_tor
• tar xzf tor-0.2.6.9.tar.gz
• tar xzf libevent-2.0.22-stable.tar.gz
• tar xzf openssl-1.0.2c.tar.gz
• tar xzf zlib-1.2.8.tar.gz
• cd libevent-2.0.22-stable
• ./configure --prefix=/static_tor/libevent --disable-shared --enable-static --with-pic
• make
• sudo make install
• cd ../openssl-1.0.2c
• ./config --prefix=/static_tor/openssl no-shared no-dso
• make
• sudo make install
• cd ../zlib-1.2.8
• ./configure
• make
• sudo make install prefix=/static_tor/zlib
• cd ../tor-0.2.6.9
• ./configure --enable-static-tor --with-libevent-dir=/static_tor/libevent --with-openssl-dir=/static_tor/openssl --with-zlib-dir=/static_tor/zlib
• make

The desired tor binary can now be found at /static_tor/tor-0.2.6.9/src/or/tor.

Regenerating Key Material for Directory Authorities

• Install Tor (if compiling from source, you will find tor-gencert in src/tools after compiling) on a machine on which you have NOT run Tor previously
• Run tor-gencert --create-identity-key to generate authority_certificate, authority_identity_key, authority_signing_key in the current directory
• Copy those files into /SUPPORT/auth/0/keys, replacing the expired versions already there
• Repeat the previous two steps for the remaining authorities in /SUPPORT/auth/ (you will need to delete the generated three files in order for rerunning tor-gencert to generate new key material)

Sample Invocations

To generate a simple configuration called test for the project TorPerf with two of each type of node and no loss:

python emulator.py -y 2 -e 2 -a 2 -s 2 -c 2 -j TorPerf test

To generate an identical configuration with several scripts included via both command line options and a script file:

python emulator.py -y 2 -e 2 -a 2 -s 2 -c 2 -j TorPerf -v --setup-script ./testA.sh --setup-script-start 600 --setup-script-stop 660 --experimental-script ./testB-##.sh --experimental-script-start 660 --experimental-script-stop 720 --experimental-script-client-only -f ./scripts test

To generate a larger configuration with 1% loss on the second server:

python emulator.py -y 4 -e 4 -s 3 -c 10 -l server1 -r 01 -j TorPerf test

To generate a configuration with 15% loss on all 3 clients:

python emulator.py -y 2 -e 2 -c 3 -l clients -r 15 -j TorPerf test

To generate a configuration with varied loss on several different nodes:

python emulator.py -y 2 -e 2 -c 3 -l client0 relay0 exit0 -r 15 10 05 -j TorPerf test