Skip to content

Run end-to-end latency benchmark on burst workload #2

@AayushBarhate

Description

@AayushBarhate

What

Run benchmark/latency/run_latency_test.sh against real Kafka burst traffic. Collect per-event latency for fixed (2s), fixed (500ms), and adaptive approaches.

Why

This is the paper's main result. Benchmark code exists but hasn't been executed with burst data flowing through Kafka → Spark.

Steps

  1. Start Kafka, create topic adaptive-latency-test
  2. Run generator/burst_generator.py — 300s, baseline=100, burst_mult=8
  3. Run benchmark/latency/latency_benchmark.py for each mode
  4. Collect JSON from /tmp/latency_results/
  5. Compute p50, p95, p99 per approach
  6. Adaptive should beat fixed during burst windows

Acceptance criteria

  • Latency JSON for all 3 approaches on same workload
  • Comparison table showing adaptive reduces burst-period latency

Metadata

Metadata

Assignees

No one assigned

    Labels

    criticalMust have for paperevaluationRunning experiments

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions