What
Run benchmark/latency/run_latency_test.sh against real Kafka burst traffic. Collect per-event latency for fixed (2s), fixed (500ms), and adaptive approaches.
Why
This is the paper's main result. Benchmark code exists but hasn't been executed with burst data flowing through Kafka → Spark.
Steps
- Start Kafka, create topic
adaptive-latency-test
- Run
generator/burst_generator.py — 300s, baseline=100, burst_mult=8
- Run
benchmark/latency/latency_benchmark.py for each mode
- Collect JSON from
/tmp/latency_results/
- Compute p50, p95, p99 per approach
- Adaptive should beat fixed during burst windows
Acceptance criteria
- Latency JSON for all 3 approaches on same workload
- Comparison table showing adaptive reduces burst-period latency
What
Run
benchmark/latency/run_latency_test.shagainst real Kafka burst traffic. Collect per-event latency for fixed (2s), fixed (500ms), and adaptive approaches.Why
This is the paper's main result. Benchmark code exists but hasn't been executed with burst data flowing through Kafka → Spark.
Steps
adaptive-latency-testgenerator/burst_generator.py— 300s, baseline=100, burst_mult=8benchmark/latency/latency_benchmark.pyfor each mode/tmp/latency_results/Acceptance criteria