ana-cc / onionperf Public

Fixes #34216.

Now that <https://bugs.torproject.org/33258> is merged, we require these
two libraries.

If we don't do that, pip may fail because older versions don't ship with
the wheel package, which is needed to build some of our requirements.

To make this work, we also had to replace distutils with setuptools
because distutils does not understand the install_requires keyword.

Co-authored-by: Ana Custura <ana@netstat.org.uk>

As pointed out in #30586, requirements.txt is not in sync with what
onionperf actually requires.  This commit adds missing and removes
unnecessary dependencies.

We're now including the time that it takes for tor to establish a
circuit and attach a stream in the time to first/last byte.

Implements #34215.

Turns out that Networkx renamed the node attribute to nodes some time
between version 2.2.1 and 2.4. This workaround avoids accessing that
attribute to add another key/value pair to an existing node and
instead uses two different key/value pair sets when adding the node in
the first place. Not pretty, but works around the issue.

Fixes #34298.

These visualizations are not really useful to us, so we can as well
remove them and all remaining pylab code with it.

As part of this change, output files are being renamed from
tgen.onionperf.viz.yyyy-mm-dd.{csv,pdf} to just
onionperf.viz.yyyy-mm-dd.{csv,pdf}.

Implements #34214.

Still part of Implements #33434.

With the new partial downloads timestamps we can easily extract
time-to-first-50-KiB and time-to-first-1-MiB numbers from the 5 MiB
downloads. An analysis of past measurements showed that these numbers
are comparable.

By removing those smaller downloads we increase the total number of
actually useful measurements.

Total bandwidth consumption from this change increases from 14 to 140
kbps which still seems very reasonable.

Implements #34023.

The idea is to avoid overlapping measurements. With a new measurement
starting every 5:00 minutes, we time out after 4:30 minutes to make
"room" for the next measurement to start.

We also don't use the stallout feature anymore by setting the value to
0 seconds. The stallout would have failed a measurement after not
receiving new bytes for the given number of seconds. But that's also
not what we want to measure.

Implements #34024.

This commit touches similar places in the code as c8b0fc2 which added
v3 onion service support.

Implements #33434.

Previous OnionPerf visualizations were written using PyLab, which is a
relatively low-level visualization library based on matplotlib. PyLab
is the same thing as PyPlot but with some extra shortcuts. However,
using PyLab is discouraged now.

This commit switches all tgen plots to use seaborn, which is a
high-level data visualization library also based on matplotlib. This
commit also uses pandas for underlying data structures in order to
separate data tidying and data visualization.

As a side effect of using pandas, graphed data is now exported to a
.csv file using the same naming scheme as the produced tgen .pdf file.

Implements #33258.

This fixes <https://bugs.torproject.org/33435>.

Implements #34224.