v0.2.30

GLiNER v0.2.30 — FP8 inference, confidence calibration, and unified serving API

This release adds FP8 inference, tools for calibrating prediction confidence, and a unified API for entity and relation extraction. It also improves compiled serving and fixes issues across token alignment, batching, training, and evaluation.

New capabilities

• FP8 inference: Quantize encoder linear layers with TorchAO and enable FP8 QK attention with FlashDeBERTa using quantize="fp8" or --quantization fp8 when serving. Prediction heads and LoRA adapters retain their existing precision. [#402](#402)
• Confidence calibration: Fit a global temperature and optional bias on held-out data without updating backbone weights. The new calibration script saves the fitted parameters and reports calibration and prediction metrics before and after fitting. [57ac9bc](57ac9bc)
• Learned calibration head: Fit a small calibration head, adapted from GLiClass, that uses contextual embeddings and score statistics to adjust confidence for individual span–label or word–label candidates. The backbone remains frozen, and the head is saved with the model. [8358e11](8358e11)
• Additional training objectives: Add ce and ce_calibrated losses for multi-label learning against a fixed “no entity” logit, with an optional BCE calibration term. Both retain GLiNER’s sigmoid-threshold decoding. [2d1e37a](2d1e37a)
• Unified serving API: Submit named entity and relation-extraction tasks through /gliner/v1/process and /gliner/v1/process/batch, with per-task labels and thresholds, structured JSON input, source-field offsets, automatic windowing for long inputs, and Python client helpers. Includes model discovery, capabilities, health, and adapter-cache endpoints. [#402](#402)

Fixes and improvements

• Improve compiled serving with reusable batch sizes, consistent forward inputs, and support for compilable FlashDeBERTa kernels. Add batch statistics and improve PolyLoRA handling. [#402](#402)
• Prevent padded words and entity classes from generating spurious token-derived spans in mixed batches. [#402](#402)
• Preserve original word positions when tokenization drops input words. [#394](#394)
• Fix relation-extraction batches containing empty items. [#400](#400)
• Fix classification evaluation with preloaded datasets. [#398](#398)
• Preserve explicit warmup_steps when initializing training arguments with Transformers 5. [#396](#396)
• Expand the supported Transformers dependency range to >=4.51.3,<5.23.0. [520e5de](520e5de)

Upgrade notes

• FP8 inference requires torchao and a CUDA GPU with compute capability 8.9 or newer. Use compilation for the intended optimized execution path.
• Calibration requires fitting on representative held-out data; upgrading alone does not calibrate existing checkpoints.
• The learned calibration head supports native PyTorch inference. It is not applied during training, with ONNX/OpenVINO runtimes, or with streaming models.
• Global temperature-only calibration preserves decisions at threshold 0.5; adding a bias can change those decisions.
• The existing /gliner endpoint remains available. The new API supports span tasks and, with compatible checkpoints, relation tasks.
• Use /v1/process and the Python client’s process() methods; the serving documentation in this tag still contains references to the earlier analyze naming.

Full changelog: [v0.2.29...v0.2.30](v0.2.29...v0.2.30)