Benchmarking

Benchmarking

From 1.6.1 the repository has a live benchmark that runs the same traffic against both capture backends, kernel (XDP and TC) and libpcap, and compares them. Use it to see how the two behave on your own hardware before choosing one. It needs a gateway running FLOD 1.6.1 or later and machines that can generate Normal, Flash Crowd, and attack traffic. The full write-up, with the figures and their limits, is in the repository's Backend Benchmark.

Every figure quoted here comes from the project's own simulated lab environment. Other networks, other hardware, and other traffic will give other numbers.

Running it

The script runs from a workstation and controls the gateway and the traffic machines over SSH.

cp scripts/benchmark_live.example.env my-benchmark.env
# edit it: gateway, generator hosts, SSH keys, targets, INGRESS_IFACE, EGRESS_IFACE
bash scripts/benchmark_live.sh my-benchmark.env

Run one copy at a time. The config file is shell that the script sources, so keep it readable only by you.

Variable	Default	Meaning
CAPTURE_MODES	kernel pcap	Backends to run, in order. Each gets the full phase set.
RUNS_PER_MODE	1	Repeats per backend. Two or more gives a run to run spread.
CALIBRATE	off	measure or apply runs scripts/calibrate.py during warm-up.
ATTACK_SWEEP_SECS	0	Seconds each attack type runs alone and with Normal traffic after the main phases.
NORMAL_VARIANTS, FLASHCROWD_VARIANTS, ATTACK_VARIANTS	empty	Named traffic variants that rotate between phases.
IDLE_INGRESS_MAX_PPS	300	The ingress interface must stay under this many packets a second with every generator stopped.

A run takes roughly one and a half to two hours for both backends with calibration on. It switches the sensor's backend through /etc/ddos_stage1/tuning.env, gives each run its own baseline that is deleted at the switch so warm-up starts from scratch, empties both ipsets and restarts Stage 2 before each run, and puts your original configuration back at the end, byte for byte.

Reading the report

Each session writes benchmark-live-results/session_<UTC time>/report.txt and results.json. Read these in order:

1. The calibration section of each run. Floors close to your Normal traffic (2 to 3 packets a second in the lab) and a few percent of windows flagged mean the run learned peacetime. Floors far above it, or most windows flagged, mean something else was sending and the run's detection figures are not usable.
2. The Normal phase. It should show almost no verdicts and no actions.
3. The per phase throughput, CPU, context switches and memory, and the Stage 2 latency: handoff from the sensor, inference, the enforcement call, and window close to rule applied.
4. The agreement table, which shows whether the backends reached the same verdicts and actions in each phase.

What the lab runs showed

In a clean pair of runs, one per backend:

	Kernel	libpcap
Sensor CPU, phases without an unpaced flood	0.8% to 5.9%	3.0% to 39.7%
Sensor CPU at the unpaced floods (116,000 to 138,000 packets a second)	72% to 79%	44% to 50%
Context switches a second	4 to 6	285 to 4,844
Sensor memory	7 MB	271 MB
Handoff to Stage 2, mean per phase	13 to 65 ms	2.6 to 34 ms

The backends agreed on the DDoS verdict in 16 of 17 phases. The kernel backend is far cheaper until a flood is unpaced, where the order reverses, and its handoff to Stage 2 is longer. Each backend has one clean run, so the spread between identical runs is unknown.

When a run looks wrong

See Troubleshooting. The usual causes are a generator left running by an interrupted session, two copies of the script running at once, and an egress interface that lost its address.