Note: This tool is in the alpha stage!
This tool can be used to conduct network RTT measurements, convert them into NetEm delay distribution tables, and load these tables into the network emulator.
Two methods for measuring the RTT are supported:
- Faked 3-way TCP handshake: Uses the delay between initial SYN and ACK packets in the handshake of a TCP connection.
- ICMP echo/request packets (as used by
ping
)
The gathered information can be used to configure the Linux network emulation queuing discipline (see tc-netem(8)).
It is possible to directly pass the results of the RTT probes to the Kernel by using a netlink socket and libnl. Therefore it allows to simulate an existing network connection on-the-fly. Alternatively, the measurements can be stored in a file and replayed later using Bash's IO redirection.
Run TCP SYN/ACK probes to measure round-trip-time (RTT):
./netem probe 8.8.8.8 53 > measurements.dat
The probe
sub-command returns the following fields per line on STDOUT:
current_rtt, mean, sigma
Collect measurements to build a tc-netem(8) delay distribution table
./netem dist generate < measurements.dat > google_dns.dist
Please note: you might have to change the scaling by adjusting the compile time constants in dist-maketable.h
!
./netem dist load < probing.dat
Please note: you might have to change the scaling by adjusting the compile time constants in dist-maketable.h
!
The output of this command and be stored in a file or directly passed to the emulate
subcommand:
./netem probe 8.8.8.8 53 | ./netem emulate
or
./netem emulate < measurements.dat
The emulate sub-command expects the following fields on STDIN seperated by whitespaces:
current_rtt, mean, sigma, gap, loss_prob, loss_corr, reorder_prob, reorder_corr, corruption_prob, corruption_corr, duplication_prob, duplication_corr;
At least the first three fields have to be given. The remaining ones are optional.
To apply the network emulation only to a limit stream of packets, you can use the mark
tool.
./netem -m 0xBABE dist load < measurements.dat
sudo LD_PRELOAD=${PWD}/mark.so MARK=0xBABE netcat google.de 80
This tool uses the dynamic linker to hook into the socket()
wrapper-function of libc (see mark.c
).
Usually, the hook will simply call the original socket(2)
syscall for non-AF_NET sockets.
But for AF_INET sockets, the hook will additionally call setsockopt(sd, SOL_SOCKET, SO_MARK, ...)
after the socket has been created.
Later on, the netem
tool will use combination of the classfull prio
qdisc and the fw
classifier to limit the network emulation only to the marked application (see use case 5, below).
Note: There are two pittfalls when using this approach:
- Make sure to specify the environmental variables after the sudo command! This is necessary, as
ping
is a SUID program. The dynamic linker strips certain enviromental variables (asLD_PRELOAD
) for security reasons when privileges are elevated. - Setting the packet mark requires CAP_NET_ADMIN privs. Therefore you must start the application as root. Unfortunately, some applications also drop those privs quite early (ping is an example which luckily has an
-m
option).
Alternatively you can set the mark using netfilter:
iptables -t mangle -I OUTPUT -d 8.8.8.8 --set-mark 0xBABE -j MARK
Or, the scripts/tc-cgroup.sh
script which uses a special priority for a certain cgroup:
./tc-cgroup ping google.de
./scripts/tc-dump.sh eth0
======= eth0: qdisc ========
qdisc prio 1: root refcnt 2 bands 4 priomap 2 3 3 3 2 3 1 1 2 2 2 2 2 2 2 2
Sent 17304 bytes 126 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc netem 2: parent 1:1 limit 1000 delay 3.3ms 9.1ms
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
======= eth0: filter ========
filter parent 1: protocol all pref 49152 fw
filter parent 1: protocol all pref 49152 fw handle 0xcd classid 1:1
======= eth0: class ========
class prio 1:1 parent 1: leaf 2:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:2 parent 1:
Sent 15126 bytes 115 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:3 parent 1:
Sent 3270 bytes 17 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:4 parent 1:
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
scripts/tc-reset.sh eth0
scripts/tc-dump.sh eth0
======= eth0: qdisc ========
qdisc pfifo_fast 8005: root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 57402 bytes 774 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
======= eth0: filter ========
======= eth0: class ========
Add more metrics to the probing system:
- loss
- duplication
- corruption
- reordering
There is experimental support for using Linux' HW / Kernelspace timestamping support (see ts.c
).
This allows to measure the RTT by using the arrival / departure times of packets in the NIC or in the Kernel, instead of relying on the inaccuarate user space.
Unfortunately, this hardware timestamping support requires special driver support. Therefore it's still disabled.
git submodule init git submodule update
cd libnl/ autreconf -i ./configure make make install
Just run make
.
- libnl3 (3.2.26)
- recent Linux Kernel version
For building libnl (make libnl)
- libtool
- bison
- flex
- autotools
- http://www.infradead.org/~tgr/libnl/
- https://www.kernel.org/doc/Documentation/networking/timestamping.txt
- https://github.com/shemminger/iproute2/tree/master/netem
- tc-netem(8)
Steffen Vogel post@steffenvogel.de
This tool was developed as a project for the Network Programming Laboratory of the Institute for Networked Systems, RWTH Aachen University, in summer 2015.
Copyright (C) 2015 Steffen Vogel
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.