Automatic collection of network measurement metadata.
Python 3.6 is required.
Using a Python virtual environment (venv) is strongly suggested:
# Install a python venv and required modules $ python3 -m venv xenv $ source xenv/bin/activate $ pip3 install -r requirements.txt
At present, there's no standard
setup.py distutils (or similar) script. But there will be, eventually.
To start SoMeta run the
metameasurement.py Python program. There are several
possible command-line options. See
metameasurement.py -h for a list. Some
additional detail is below, specifically regarding monitors and options.
-c option indicates the "external" measurement tool to start. By default,
sleep 5, which causes SoMeta simply to collect 5 seconds-worth of
metadata, given what ever monitors have been configured. You'll almost certainly
need to quote the command line for the external tool, and some escaping may be required
if there are embedded quotes needed for the tool (see the example with scamper, below).
-M option specifies a monitor to start. Standard available sources include cpu, mem, io, netstat, rtt (see the
To configure a monitor, parameters may be specified along with each monitor name, each separated by a colon (':'). Each parameter may be a single string, or a
key=value pair. The order of parameters doesn't matter.
Valid parameters for each standard monitor are:
-M cpu:interval=X: set the periodic sampling interval (default 1 sec)
-M io:interval=X: set the periodic sampling interval (default 1 sec)
-M mem:interval=X: set the periodic sampling interval (default 1 sec)
-M netstat:interval=X: set the periodic sampling interval.
Additional string arguments to the netstat monitor can specify interface names to monitor (all interfaces are included if none are specified). For example, to monitor en0's netstat counters every 5 seconds:
Monitor RTT along a path to destination
Dout of interface
IfaceNamewith probe rate
R. Probe interval is gamma distributed. The default destination is 220.127.116.11 and default probe rate is 1/sec.
ProbeTypecan either be
hoplimited(default is hoplimited)
MaxTTLis maximum ttl for hop-limited probes (pointless for ping probes). Default is maxttl = 1.
Protocolis (icmp | tcp | udp) (for hop-limited probes). Default is icmp.
allhops: probe all hops up to maxttl (for hop-limited probes)
constflow: manipulate packet contents to force first 4 bytes of transport header to be constant (to make probes follow a constant path). This parameter only has an affect on icmp; data are appended to force the checksum to be a constant value. Note: udp/tcp probes always have const first 4 bytes.
Here are some examples:
# Monitor only CPU performance while emitting 100 ICMP echo request (ping) probes to # www.google.com. $ python3 metameasurement.py -Mcpu -c "ping -c 100 www.google.com" # Monitor CPU performance and netstat counters (for all interfaces) for traceroute $ python3 metameasurement.py -Mcpu -Mnetstat -c "traceroute www.google.com" # Monitor CPU, IO and Netstat counters for ping # Set the metadata output file to start with "ping_google" $ python3 metameasurement.py -Mio -Mnetstat -c "ping www.google.com" -f ping_google # Monitor everything, including RTT for the first 3 hops of the network path toward # 18.104.22.168. As the external tool, use scamper to emit ICMP echo requests, dumping # its output to a warts file. $ python3 metameasurement.py -Mcpu -Mmem -Mio -Mnetstat:eth0 -Mrtt:interface=eth0:type=hoplimited:maxttl=3:dest=22.214.171.124 -f ping_metadata -l -c "scamper -c \"ping -P icmp-echo -c 60 -s 64\" -o ping.warts -O warts -i 126.96.36.199"
analyzemeta.py script performs some simple analysis on SoMeta metadata, printing results to the console.
plotmeta.py tool is designed to help plot various metrics collected through SoMeta monitors. To see what metrics may be plotted, you can run the following:
$ python3 plotmeta.py -l meta.json
meta.json is a SoMeta metadata file. The output of
plotmeta.py with the
-l option shows various items that can be plotted. Each item is organized into groups. You can either plot any number of individual items (
-i option), or plot each metric for an entire group (
-g option). If you want everything, use the
-a option. In addition,
-t option can be used to change the type of output plot. Use ecdf for empirical CDF or timeseries for simple scatter plot with timeline (which is default output of the plot tool). See
plotmeta.py -h for all options.
Here are some examples:
$ python3 plotmeta.py -t ecdf -i cpu:idle -i io:disk0_write_time meta.json $ python3 plotmeta.py -t timeseries -g cpu meta.json $ python3 plotmeta.py -a meta.json
Reproducing our results
Check SoMeta website (https://jsommers.github.io/metameasurement/) to reproduce our data (and results) at your end.
I gratefully acknowledge support from the National Science Foundation. The materials here are based upon work supported by the NSF under grant CNS-1054985 ("CAREER: Expanding the functionality of Internet routers").
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation.
Copyright 2017 SoMeta authors. All rights reserved.
The SoMeta software is distributed under terms of the GNU General Public License, version 3. See below for the standard GNU GPL v3 copying text.
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.