Third public alpha — a substantial agent-runtime and inference release.
Agent compatibility
- Add OpenAI Responses API and Anthropic Messages API support alongside Chat Completions.
- Stream tool-call events with strict schema-constrained arguments.
- Validate compatibility with Codex, Claude Code, Cline, OpenCode, Clu, and Ornith request shapes.
- Publish the agent-torture conformance and cross-runtime comparison tooling.
Runtime and performance
- Add continuous batched decoding and persistent/forkable KV prefixes.
- Share CUDA model weights across parallel slots and add optional Q8 KV cache.
- Fix CPU SIMD build flags and raise the default thread count for materially better out-of-box CPU throughput.
- Add optimized CUDA kernels while retaining token-identity gates.
Models and correctness
- Add native Ornith/Qwen3.5, Apertus, Gemma 4, SmolLM, Mistral, and Phi-3 tokenizer/template behavior.
- Strengthen strict JSON Schema support, including integer bounds and additional parser/schema OOM handling.
- Harden HTTP framing, graceful lifecycle handling, cross-process VRAM reservations, model swapping, and Python stream cancellation.
Release artifacts include Linux x86_64, macOS arm64, and Windows x86_64 binaries, packaged archives with provenance, and SHA-256 checksums.
Full diff: v0.1.1-alpha...v0.1.2-alpha