v0.2.0

What's New in 0.2.0

vLLM 0.19 Native Support

• All 12 plugins work natively on vLLM >= 0.19 via IOProcessor — no patches required
• Decoupled FactoryPooler protocol with zero vLLM imports

T5Gemma2 Encoder-Decoder Backbone

• Full encoder + decoder backbone for google/t5gemma-2-270m-270m
• Multimodal support: text + vision (SigLIP) with multimodal projector
• Full HuggingFace parity on raw logits (text: max_diff < 3e-4, decoder: < 9e-5)
• Two high-impact custom Triton kernels:
  • flash_t5gemma2_attention — tiled flash attention with softcapping, asymmetric sliding window, GQA, merged self+cross attention (+25% throughput)
  • fused_qk_norm_rope — GemmaRMSNorm + RoPE fused in single pass (+10% throughput)
• Combined: 1.44–1.51x speedup across all batch sizes. Peak: 854 samples/s at bs=64
• No pooler/IOProcessor yet — available as a building block for downstream tasks (ColPali, GLiNER, OCR, classification)
• Includes per-kernel benchmark and batch throughput benchmark with CUDA event timing
• Two-phase parity test (collect HF reference + test vLLM-factory) for both text and multimodal

Benchmark Results (12 plugins)

• Up to 12.6x throughput vs vanilla PyTorch at peak concurrency
• All 12/12 plugins pass parity validation
• Full sweep data and charts in bench/

Known Limitations

• 2 scoped monkey-patches remain (attention_mask forwarding, KV cache skip)
• GLiNER models show 10–30% throughput reduction at high concurrency (c≥32) due to vLLM V1 IPC overhead

Full Changelog: v0.1.2...v0.2.0

Full Changelog: v0.1.2...v0.2.0