v0.2.2

Fixed

• Preserve GLiNER shape-prefix metadata as float32 so bf16 pooling outputs do not round odd word counts above 256.
• Prevent token-level GLiNER post-processing reshape failures in DeBERTa/ModernBERT GLiNER poolers.

Validation

• python -m pytest /workspace/vllm-factory/test_shape_prefix.py -q
• python -m ruff check /workspace/vllm-factory/vllm_factory/pooling/shape_prefix.py /workspace/vllm-factory/plugins/modernbert_gliner_rerank/pooler.py /workspace/vllm-factory/poolers/gliner.py /workspace/vllm-factory/plugins/deberta_gliner_linker/pooler.py /workspace/vllm-factory/test_shape_prefix.py

What's Changed

• feat: add request affinity support for backend selection in multi-ins… by @srimon12 in #8
• feat: enhance observability logging and schema caching in GLiNER2 pro… by @srimon12 in #9
• fix(deberta_gliner2): support count_lstm_v2 / count_lstm_moe pooler variants by @MattThomas-fastino in #13
• feat: widen deberta_gliner2 schema contract for per-field thresholds by @MattThomas-fastino in #10
• feat: register DeBERTa backbones as SupportsLoRA targets by @MattThomas-fastino in #11
• feat: wire per-request LoRA adapter through deberta_gliner2 IOProcessor by @MattThomas-fastino in #12
• feat(dispatcher): log backend selection, affinity state, and schema size for GLiNER2 requests by @srimon12 in #14
• Feat/gliner2 modernbert plugin by @Reterno12 in #16

New Contributors

• @MattThomas-fastino made their first contribution in #13
• @Reterno12 made their first contribution in #16

Full Changelog: v0.2.1...v0.2.2

v0.2.1

What's Changed

Native GLiNER2 schema extraction with labels backward compat (#4)

• Support both schema (priority) and labels (normalized to {"entities": labels}) input contracts
• Six task types: entities, relations, classification, json_structures, entity_groups, mixed schemas
• New request parameters: threshold, include_confidence, include_spans

Request contract tightening — overflow handling, threshold, model prep (#5)

• Token overflow handling: max_model_len enforcement with optional truncate_overflow_text
• Threshold filtering applied to classification results (raw logits)
• Robust error handling in model preparation with actionable messages
• Request contract documentation with JSON payload examples

Request-side preprocessing caching (#6)

• LRU caching for tokenization, special token IDs, and schema preprocessing
• ~33% latency reduction on repeated requests with identical schemas

Full Changelog: v0.2.0...v0.2.1

What's Changed

• Add native GLiNER2 schema extraction support to vLLM pooling by @srimon12 in #4
• Tighten GLiNER2 request contract by @srimon12 in #5
• Adds request-side caching to the deberta_gliner2 by @srimon12 in #6

New Contributors

• @srimon12 made their first contribution in #4

Full Changelog: v0.2.0...v0.2.1

v0.2.0

What's New in 0.2.0

vLLM 0.19 Native Support

• All 12 plugins work natively on vLLM >= 0.19 via IOProcessor — no patches required
• Decoupled FactoryPooler protocol with zero vLLM imports

T5Gemma2 Encoder-Decoder Backbone

• Full encoder + decoder backbone for google/t5gemma-2-270m-270m
• Multimodal support: text + vision (SigLIP) with multimodal projector
• Full HuggingFace parity on raw logits (text: max_diff < 3e-4, decoder: < 9e-5)
• Two high-impact custom Triton kernels:
  • flash_t5gemma2_attention — tiled flash attention with softcapping, asymmetric sliding window, GQA, merged self+cross attention (+25% throughput)
  • fused_qk_norm_rope — GemmaRMSNorm + RoPE fused in single pass (+10% throughput)
• Combined: 1.44–1.51x speedup across all batch sizes. Peak: 854 samples/s at bs=64
• No pooler/IOProcessor yet — available as a building block for downstream tasks (ColPali, GLiNER, OCR, classification)
• Includes per-kernel benchmark and batch throughput benchmark with CUDA event timing
• Two-phase parity test (collect HF reference + test vLLM-factory) for both text and multimodal

Benchmark Results (12 plugins)

• Up to 12.6x throughput vs vanilla PyTorch at peak concurrency
• All 12/12 plugins pass parity validation
• Full sweep data and charts in bench/

Known Limitations

• 2 scoped monkey-patches remain (attention_mask forwarding, KV cache skip)
• GLiNER models show 10–30% throughput reduction at high concurrency (c≥32) due to vLLM V1 IPC overhead

Full Changelog: v0.1.2...v0.2.0

Full Changelog: v0.1.2...v0.2.0

v0.1.2

Full Changelog: v0.1.1...v0.1.2

v0.1.1

Initial production release of vllm-factory.

12 plugins for vLLM pooling models with IO processor support:
embeddinggemma, mmbert_gliner, deberta_gliner, deberta_gliner2, mt5_gliner,
deberta_gliner_linker, modernbert_gliner_rerank, moderncolbert, lfm2_colbert,
collfm2, colqwen3, nemotron_colembed.

Full Changelog: https://github.com/ddickmann/vllm-factory/commits/v0.1.1