Real PyTorch inference server comparing FCFS, EagerContBatch, and ChunkedPrefill on RTX 2070. Key finding: ChunkedPrefill is not a throughput optimizer but a fairness mechanism — it reduces worst-case decode stalls from 58.6ms to 34.1ms while EagerContBatch maximizes mean TTFT and throughput.

Measures actual GPU costs for LLM prefill and decode on RTX 2070 (8GB) to validate simulation parameters. Key findings: prefill converges to 30-70 us/token at long sequences; decode is memory-bandwidth-bound (constant with prefix length, 5300-10800 us/token single-request); simulation defaults are correct for server-level amortized batching.

Empirical study of speculative decoding with Qwen2-0.5B (draft) and Qwen2-1.5B (target) on a single RTX 2070. Measures token acceptance rate, speedup, KV sync overhead, and validates the Leviathan et al. (2023) analytical model. Shows why cost_ratio=1.18x (not 3.1x as expected) causes failure and what hardware conditions make it viable.

Measures real KV cache costs on GPU: formula validation (36 KB/token exact), decode memory-bound (3% variation), compaction bandwidth 19-67 GB/s, and counter-intuitive finding: compaction during active decode runs 30% faster due to CUDA stream overlap.

Profiles 6 attention variants on RTX 2070 (sm75): naive O(n²), SDPA auto, FlashAttention, math backend, mem_efficient, sliding window, and GQA. Key findings: sdpa_default is 7.64x faster than naive at seq=2048; memory grows 25x (naive) vs 2.4x (sdpa) for 32x longer sequences; kernel choice matters more than model architecture at long sequences.

Measures real speculative decoding speedup using the official HuggingFace assistant_model API across 4 model pairs and output lengths up to 512 tokens. Best result: distilgpt2->gpt2-medium achieves 1.747x speedup at 512 output tokens. Validates that cost_ratio and output_length are the key parameters.

Measures real prefix cache costs on GPU across 3 experimental versions. Key findings: 2.41x speedup with prefix=512; multi-turn speedup grows to 2.06x over 10 turns; batch sharing breakeven at n=2 (prefix=512) vs n=12 (prefix=128); LFU cache with Zipf alpha=2.0 achieves 82% hit rate with only 4 cache slots.