Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration of MoonSniff #227

Closed
wants to merge 124 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
124 commits
Select commit Hold shift + click to select a range
1618af1
add moonsniff demo skeleton
AP-Frank Mar 11, 2018
7531259
extend moongen by moonsniff sources
AP-Frank Mar 12, 2018
b5cd5a0
add length control and a payload
AP-Frank Mar 13, 2018
adf1538
use uint16_t instead of uint8_t
AP-Frank Mar 13, 2018
b933996
implement basic ring_buffer
AP-Frank Mar 13, 2018
8f98e8d
add miss and hit counters
AP-Frank Mar 13, 2018
83823a6
setup new core implementation
AP-Frank Mar 14, 2018
1b6ffbd
simplify example and add storage for timestamps
AP-Frank Mar 14, 2018
980af33
add average latency computation
AP-Frank Mar 15, 2018
6e91e9d
wip debug latencies
AP-Frank Mar 15, 2018
a07f218
add mutexes to ensure thread safety
AP-Frank Mar 15, 2018
e9d17d1
WIP find a way to synchronize clocks
AP-Frank Mar 20, 2018
85c18d3
inserted multiple time synchronisation points
AP-Frank Mar 20, 2018
2f0a95b
add checks to validate timestamps and remove superfluous clock syncs
AP-Frank Mar 20, 2018
4d2c615
add scripts for moonsniff in target environment
AP-Frank Mar 21, 2018
e8719ef
correct stats, add additional syncing point
AP-Frank Mar 21, 2018
88b170a
correct parser args
AP-Frank Mar 21, 2018
7b6beaa
Merge branch 'moonsniff' of moongen:AP-Frank/MoonGen into moonsniff
AP-Frank Mar 21, 2018
e379c68
write latencies to file instead of dequeue
AP-Frank Mar 24, 2018
dc2f8b2
add barriers to prevent race conditions
AP-Frank Mar 24, 2018
fe43fef
rework average measurements to read from file
AP-Frank Mar 29, 2018
ff1cc38
update submodule
AP-Frank Apr 2, 2018
bcf5eef
more straight-forward access to statistics
AP-Frank Apr 2, 2018
45cb08c
general cleanup
AP-Frank Apr 2, 2018
e0d623f
remove mutexes alltogether
AP-Frank Apr 2, 2018
2e5c3ee
merge both timestamping methods
AP-Frank Apr 6, 2018
d5fb7e6
prepare for different usage modes
AP-Frank Apr 6, 2018
bfffebf
fix error where hit_list being to small
AP-Frank Apr 6, 2018
6e7aa28
change id length to effectivly 24 bits
AP-Frank Apr 7, 2018
1b90001
cleanup
AP-Frank Apr 7, 2018
a832e5b
WIP write binary and text based files
AP-Frank Apr 7, 2018
20006c8
WIP usage of the File_Writer/Reader classes
AP-Frank Apr 7, 2018
c3afb94
finish binary and text based file writing and reading
AP-Frank Apr 8, 2018
ba9866b
count misses which are caused by the hit_list not being filled yet
AP-Frank Apr 8, 2018
55358a8
rename writing modes to avoid naming conflicts
AP-Frank Apr 9, 2018
7f1a65f
add error handling
AP-Frank Apr 9, 2018
df69415
move post-processing into own script
AP-Frank Apr 9, 2018
8d5f8b7
add flags for binary mode and output selection
AP-Frank Apr 9, 2018
c6008e1
fix intendention
AP-Frank Apr 9, 2018
2942d9e
include counter to track overrides
AP-Frank Apr 11, 2018
d42d6cf
fix short identifiers sent in packets, performance tweaks
AP-Frank Apr 12, 2018
edff5c6
WIP add new writing mode and include debuging mode
AP-Frank Apr 13, 2018
1c22501
fix writing issue
AP-Frank Apr 13, 2018
c004628
add full offline capture mode
AP-Frank Apr 14, 2018
ae9e88b
fix typos
AP-Frank Apr 14, 2018
546c78e
fix flag related errors
AP-Frank Apr 14, 2018
bf9bde9
WIP post processing for mscap
AP-Frank Apr 14, 2018
e29423c
add basic matching
AP-Frank Apr 15, 2018
da2759b
include c++ histogram
AP-Frank Apr 16, 2018
7fbb92d
fix error for writing files and allow negative timestamps
AP-Frank Apr 16, 2018
6925a76
fix issue of counting invalid hits
AP-Frank Apr 16, 2018
c08dfe4
move timesync into master method
AP-Frank Apr 16, 2018
02b3b96
remove unnecessary C call
AP-Frank Apr 20, 2018
95c4b4b
make scripts more usable with cli options
AP-Frank Apr 23, 2018
c7bd492
fix missing arguments
AP-Frank Apr 25, 2018
e1ca44f
add option to specify the name for the output file
AP-Frank Apr 25, 2018
30c8e2b
avoid dpdk startup for postprocessing
AP-Frank May 1, 2018
13ad77d
re-add mutexes for live processing
AP-Frank May 6, 2018
c4f0922
variable bucket sizes for histogram
AP-Frank May 7, 2018
acd239d
add function to save complete packets
AP-Frank May 10, 2018
6c06556
WIP add post-processing of pcap files
AP-Frank May 14, 2018
9f300e9
get identifier form ipv4 chksm and id
AP-Frank May 15, 2018
944f7d5
WIP debug ipv4 hdr id parsing
AP-Frank May 16, 2018
de63097
fix counter showing values twice
AP-Frank May 22, 2018
7c0be08
add post-processing for pcap files
AP-Frank May 22, 2018
4bf6be4
WIP debug negative values in post-processing
AP-Frank May 25, 2018
6e0bd13
WIP debug negative timestamps
AP-Frank May 26, 2018
2318c79
WIP fix pre filling
AP-Frank May 26, 2018
c99b9e7
zero initialize map and fix size
AP-Frank May 28, 2018
45d145a
add debug function and switch to prefill function
AP-Frank May 28, 2018
0a40fec
fix errors and introduce debug option
AP-Frank May 29, 2018
89a971a
add moonsniff type to more accuratly detect moonsniff packets
AP-Frank May 29, 2018
5159714
Merge branch 'ms-experimental' of moongen:AP-Frank/MoonGen into ms-ex…
AP-Frank May 29, 2018
138a37f
write only to mscap if moonsniff type in pkt payload
AP-Frank May 29, 2018
2d829f9
read timestamp in ns accuracy from pcap
AP-Frank May 29, 2018
21de36e
WIP improve PCAP matching
AP-Frank Jun 2, 2018
57d899d
change set methods to getters
AP-Frank Jun 2, 2018
88d8bd7
WIP unify pcap and mscap matching
AP-Frank Jun 2, 2018
0943d33
update libmoon for hash functionality
AP-Frank Jun 10, 2018
3c0e91d
WIP incorporate hash based matching
AP-Frank Jun 10, 2018
92e9f03
add debug version for pcaps
AP-Frank Jun 10, 2018
a338191
add id extraction function
AP-Frank Jun 11, 2018
f8c6fe3
add tbb library
AP-Frank Jun 17, 2018
c7dc8a6
translate flowscope tbb approach to moongen
AP-Frank Jun 18, 2018
99cc431
add origin note an correct comments
AP-Frank Jun 18, 2018
89ef80b
add deque wrapper
AP-Frank Jun 18, 2018
4bc0181
WIP use tbb as packet matching back bone
AP-Frank Jun 18, 2018
48446df
add tbb core matching
AP-Frank Jun 18, 2018
df19994
fix error in pcap handling
AP-Frank Jun 20, 2018
63e6c3c
fix some casting issues
AP-Frank Jun 21, 2018
2849efa
add missing release, use two mempools
AP-Frank Jun 21, 2018
d2e3afb
make globals local whre possible
AP-Frank Jun 21, 2018
a15a546
fix debug output for short files
AP-Frank Jun 22, 2018
ebef62a
fix pointer dereferencing issues
AP-Frank Jun 22, 2018
2302e9c
enable size constraints for hashmap
AP-Frank Jun 22, 2018
3bc44b0
performance improvements
AP-Frank Jun 24, 2018
e5d81da
refactor tbb matching into own module
AP-Frank Jun 24, 2018
bcbc9ca
split two matching methods into different files
AP-Frank Jun 25, 2018
bf50ecc
new method to cleanup the table and performance upgrade
AP-Frank Jul 2, 2018
9643f4f
slight improvements to cleaning
AP-Frank Jul 2, 2018
ae060c9
cleanup moonsniff code
AP-Frank Jul 4, 2018
6a00bd6
new cleanup method to restrain table size
AP-Frank Jul 4, 2018
7d4a705
add mpps computation
AP-Frank Jul 9, 2018
ad865c5
add csv output
AP-Frank Jul 9, 2018
daea5b0
add support for different working directories
AP-Frank Jul 11, 2018
b4d2845
add documentation for post-processing and mscap matching
AP-Frank Jul 25, 2018
e5a0bbf
extend documentation to pkt-matcher and tbbmatch
AP-Frank Jul 26, 2018
16873c5
refactor array based matching
AP-Frank Sep 13, 2018
c75dfe0
add units to mean and variation in output
AP-Frank Sep 15, 2018
9330e37
remove old ms code
AP-Frank Sep 16, 2018
d7ad19e
cleanup for moonsniff.cpp
AP-Frank Sep 18, 2018
f51afda
fix post_count is nil error
AP-Frank Sep 20, 2018
ec4dacf
complete overhaul of live mode
AP-Frank Sep 21, 2018
ce5b6f4
add README
AP-Frank Sep 25, 2018
b03803d
fix array based matching for long intervals
AP-Frank Sep 30, 2018
1d0a09a
fix high rate measurements for live mode
AP-Frank Oct 1, 2018
ef5a336
rename time and rate flags
AP-Frank Oct 4, 2018
2a8231a
fix typo
AP-Frank Oct 4, 2018
50ddc4e
add hashmap support for different key and value sizes
AP-Frank Oct 12, 2018
53225d1
Merge branch 'master' into ms-experimental
AP-Frank Oct 12, 2018
881083f
comply with requested changes
AP-Frank Oct 14, 2018
9087835
fix whitespace issue
AP-Frank Oct 15, 2018
127981b
fix threshold check and use monotonic clock
AP-Frank Oct 15, 2018
d551aa4
make entry struct packed
AP-Frank Oct 15, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@
doc/html/
*.log
*.csv
*.mscap
6 changes: 6 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ include_directories(
${CMAKE_CURRENT_SOURCE_DIR}/lib
${CMAKE_CURRENT_SOURCE_DIR}/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/include
${CMAKE_CURRENT_SOURCE_DIR}/libmoon/deps/luajit/src
${CMAKE_CURRENT_SOURCE_DIR}/libmoon/deps/highwayhash/highwayhash
${CMAKE_CURRENT_SOURCE_DIR}/libmoon/deps/tbb/include
${CMAKE_CURRENT_SOURCE_DIR}/libmoon/lib
${CMAKE_CURRENT_SOURCE_DIR}/libmoon/src
)
Expand All @@ -26,6 +28,9 @@ set(files
src/software-timestamping
src/crc-rate-limiter
src/software-rate-limiter
src/moonsniff
src/histogram
src/hashmap
)

set(libraries
Expand All @@ -37,6 +42,7 @@ link_directories(
${CMAKE_CURRENT_BINARY_DIR}/libmoon
${CMAKE_CURRENT_SOURCE_DIR}/libmoon/deps/dpdk/x86_64-native-linuxapp-gcc/lib
${CMAKE_CURRENT_SOURCE_DIR}/libmoon/deps/luajit/usr/local/lib
${CMAKE_CURRENT_SOURCE_DIR}/libmoon/deps/highwayhash/lib
)

add_executable(MoonGen ${files})
Expand Down
124 changes: 124 additions & 0 deletions examples/moonsniff/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# MoonSniff: How-to

## What is MoonSniff for?

MoonSniff consists of several scripts and also extensions to the MoonGen core which allow measuring latencies of packets with an accuracy (trueness) of ±20 ns. This is possible with just commodity hardware. The test-setup we describe later on also makes sure that measurements have no impact on the system behavior in terms of jitter or additional latencies.

Basically, it can be seen as a cheaper alternative to buying professional DAQ packet capture cards.

## Which hardware is needed to run MoonSniff?
In order to run MoonSniff you need the following hardware:

- Traffic Generator: Server with NIC (MoonGen compatible for easiest use)
- Sniffer: Server with X552 Ethernet Controller with two 10 GbE ports

Note: The Intel Xeon Processor D-1500 family has onboard X552 NIC
- Splitter: 2x Passive optical fiber splitter
- Optical: 6x Optical fiber cable
- DUT: The device you want to test (switch, software forwarder, router, cable, etc.)


## How to setup MoonSniff?
One possible test setup is shown below:

Traffic Generator Device Under Test
|-----------------| |-----------------|
| | Splitter | |
| Out |----------------x---------------------| In |
| | | | |
| In |----------------]----x----------------| Out |
| | | | | |
|-----------------| | | |-----------------|
| |
|-----------------|
| Pre Post |
| |
| |
| |
| |
|-----------------|
Sniffer

The traffic generator will create packets which are sent towards the DUT. The packets will be processed by the DUT and then sent back. Due to the splitter, the sniffer will receive two copies of each packet. One on the pre interface (packet before it entered the DUT) and one on the post interface (packet after it traversed the DUT). The sniffer timestamps both packets and can compute the total time each packet has spent inside the DUT. It can then create histograms showing the distribution of latencies and mean/variance of the latencies.

## How to use MoonSniff?
MoonSniff provides scripts for the traffic generator, DUT, and sniffer. All scripts can be found in the ``examples/moonsniff/`` directory.

To see all options supported by a script, execute it with the ``-h`` or ``--help`` flag.


### Quick example
The following example describes how to use MoonSniff for the setup described above.

On the traffic generator:

./build/MoonGen examples/moonsniff/traffic-gen.lua 0 1

On the DUT:

./build/MoonGen examples/moonsniff/test-dev.lua 0 1

On the sniffer:

./build/MoonGen examples/moonsniff/traffic-gen.lua 0 1 --live


### MoonSniff Modes
MoonSniff runs in three different modes.

1. Live Mode

This mode is meant for fast average latency estimation and is also helpful as a first check if everything works as expected. Has comparably low precision.

**Important:** This mode requires all packets to feature an identifier. See the section about identifiers.

Execute the following for the Live Mode:

./build/MoonGen examples/moonsniff/traffic-gen.lua 0 1 --live

2. MSCAP Mode

This mode creates full histograms for longer time intervals. To achieve maximum precision this mode is split into a sniffing and a post-processing phase. The memory consumption is reduced by using MoonSniff's MSCAP format which instead of storing full packets, stores only identifiers as well as timestamps. The first phase generates two files, one pre-DUT and one post-DUT file. In the second phase packets from both files are matched together and the latencies are computed.

**Important:** This mode requires all packets to feature an identifier. See the section about identifiers.

Execute the following for the MSCAP Mode:

# MSCAP Mode is the default mode
# generates latencies-pre.mscap, latencies-post.mscap, and latencies-stats.csv
./build/MoonGen examples/moonsniff/traffic-gen.lua 0 1

# use generated files to compute latency-histogram
# generates hist.csv
./build/MoonGen examples/moonsniff/post-processing.lua -i latencies-pre.mscap -s latencies-post.mscap

3. PCAP Mode

This mode also creates full histograms. Contrary to the MSCAP mode, it does not require identifiers within packets. Packets are captured as a whole, and the user can provide a user defined function (UDF) which creates an identifier based on selected parts of the packet. The UDF is a Lua script which can make use of all features of MoonGen/libmoon, especially the packet API. The UDF can handle pre and post packets differently, hence, you can (with corresponding effort) compensate all deterministic changes made by the DUT to packets. E.g. a router changes IP-addresses, but if you know your routing table you can reverse this process and generate the same identifier. To change the UDF and to see a simple example, have a look at the [pkt-matcher.lua](pkt-matcher.lua) file.

Apart from this distinction, this mode operates the same way as the MSCAP mode.

**Important:** As whole packets are captured the resulting files are very large. An SSD is recommended for high data-rates.

Execute the following for the PCAP Mode:

# generates latencies-pre.pcap, latencies-post.pcap, and latencies-stats.csv
./build/MoonGen examples/moonsniff/traffic-gen.lua 0 1 --capture

# use generated files to compute latency-histogram
# generates hist.csv
./build/MoonGen examples/moonsniff/post-processing.lua -i latencies-pre.pcap -s latencies-post.pcap

### Identifiers
Identifiers are used by two modes to efficiently match corresponding pre and post packets. The way it is currently handled can be seen in the [traffic-gen.lua](traffic-gen.lua) file.

| UDP-IPv4 headers | identifier | MoonSniff type |
| ---------------- | ---------- | -------------- |
| x bytes | 4 bytes | 1 byte |

MoonSniff type is set to: ``0b01010101``.
The type is used to filter packets which do not belong to our generated traffic.


## Further Information
To assist further automation of MoonSniff we provide some example scripts that simplify measurement series in [this](https://github.com/AP-Frank/moonsniff-scripts) repository. It also provides a script to visualize the generated histograms.
Loading