Skip to content

JohnScheuer/dynamic-batch-size-controller

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

dynamic-batch-size-controller

Python License Status Simulation Models Last Commit Repo Size

Event-driven benchmark of dynamic batch size controllers for LLM serving, measuring the throughput vs TTFT SLO trade-off under variable load.

Author: Joao Felipe De Souza Year: 2026


Overview

GPU serving systems typically use a fixed batch size chosen at deployment time. Under variable load this is suboptimal at both extremes: a small batch size leaves the GPU underutilized during peak demand, while a large batch size inflates tail latency during quiet periods.

This benchmark asks:

  1. Can a dynamic batch size controller adapt in real time to balance GPU utilization and per-request TTFT?
  2. Which controller type is most effective in which workload regime?
  3. Is there a controller that dominates all others on the Pareto frontier?

Main Findings

1. The throughput vs SLO trade-off is real and measurable

No single policy dominates all metrics. The policies form a Pareto frontier with two clear extremes:

  • latency_target: lowest SLO violations, lowest throughput
  • bang_bang: highest throughput, moderate SLO

2. Bang-bang matches or exceeds queue-reactive throughput

In steady_heavy workloads, bang_bang achieves 3.10 req/s versus 1.65 req/s for fixed_16, matching queue_reactive while maintaining similar SLO rates.

In ramp_up workloads, bang_bang achieves 3.88 req/s versus 2.45 req/s for fixed_16, with lower p99 TTFT than queue_reactive.

3. Utilization-aware control collapses to minimum batch

Under the tested feedback dynamics, util_aware and hybrid reduce batch size aggressively and rarely recover it. Their average batch size stays near the configured minimum floor, behaving similarly to fixed_4. Utilization alone is an insufficient feedback signal.

4. Latency-targeting provides the best SLO compliance

In bursty workloads, latency_target achieves 3-4% SLO violation rate versus 65% for fixed_16. The cost is 0.63 vs 1.69 req/s throughput.

5. Fixed batch size is never globally optimal

fixed_16 is dominated in throughput by queue_reactive and bang_bang in every workload, and dominated in SLO by latency_target in most.


Controllers

fixed_4 / fixed_16 / fixed_64: Static baselines.

queue_reactive: Increases batch when queue grows, decreases when idle.

latency_target: Decreases batch when p99 TTFT exceeds target, increases when TTFT is well below target.

util_aware: Targets a GPU utilization band.

hybrid: Weighted combination of queue, latency, and utilization signals.

bang_bang: Aggressive scale-up under queue pressure or high TTFT, conservative scale-down only when both signals confirm easing. The asymmetry (fast up, slow down) is the key design choice.


Workloads

  • diurnal: sinusoidal 10x peak variation
  • bursty_spikes: sudden 20x arrival spikes
  • steady_heavy: constant high load near saturation
  • ramp_up: linear increase from light to heavy over 30 min
  • mixed_length: very short and very long requests mixed

Metrics

  • throughput_rps
  • p50_ttft_ms, p95_ttft_ms, p99_ttft_ms
  • slo_violation_rate
  • avg_gpu_util
  • avg_batch_size, min_batch_size, max_batch_size, batch_size_std
  • avg_queue_depth, p95_queue_depth

Running

From the project directory:

cd ~/dev/dynamic-batch-size-controller
source venv/bin/activate
python -u run.py

Output Files

results/summary_v13.csv
results/pairwise_v13.csv
plots/

Project Structure

dynamic-batch-size-controller/
|-- src/
|   |-- __init__.py
|   |-- config.py
|   |-- workload.py
|   |-- controller.py
|   |-- simulator.py
|   |-- bench.py
|   |-- analysis.py
|-- results/
|-- plots/
|-- run.py
|-- SUMMARY.txt
|-- DESIGN.md
|-- LICENSE
|-- README.md
|-- requirements.txt
|-- .gitignore

Conclusion

Dynamic batch size control resolves the throughput vs latency SLO trade-off that fixed batch sizes cannot address. A bang-bang controller with hysteresis is the best choice for throughput-optimized systems. A latency-targeting controller is best for systems with tight TTFT SLOs. Utilization-aware control collapses to minimum batch size under the tested feedback dynamics and requires adaptive recovery to be effective.


License

MIT License. See LICENSE.

About

Event-driven benchmark of dynamic batch size controllers for LLM serving, measuring the throughput vs TTFT SLO trade-off under variable load across five workload profiles.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages