ExLlamaSharp 1.2.1-beta

Pre-release: zero-gaps on top of continuous batching. This is not the GitHub Latest download; stable remains 1.1.1.

Install

Download ExLlamaSharp-Setup-win-x64.exe from this pre-release (not the one-liner, which still installs 1.1.1). Run it (UAC), then open http://127.0.0.1:14563

• Admin UI: admin / changeme (change this in Setup)
• API key: sk-exllamasharp-dev

Changes

• A/B, tenants, and LoRA adapter HTTP APIs are real (CRUD + OpenAI X-Ab-Test-Id / X-Adapter-Id)
• EXL3 worker: LoRA load/unload, tools/JSON-schema prompt hints, multi-GPU CUDA_VISIBLE_DEVICES, speculative draft settings
• Embeddings via ONNX when model.onnx is present (local fallback otherwise)
• Quantize/import jobs call real Python convert / folder registration
• SignalR metrics + Charts / A/B UI; tenant filtering when multi-tenancy is enabled
• Worker-only production generate; OpenAI images/audio stay 501 (Media version later)

Notes

Treat this as a beta. Keep a 1.1.1 install if you need the last stable build.