midge v0.2.0 — engine for everyone, agents welcome

Everything since v0.1.0, all CI-validated:

Run any compatible model — and know before you download

Engine generalized beyond gpt-oss: Mixtral-style and Qwen3-MoE-style
families (plain SwiGLU, QK-norm, norm_topk_prob routers), validated
per-position against the NumPy reference on both engines
midge check <hf-repo|path>: fetches only config.json, judges
architecture compatibility with named reasons, benchmarks your kernel,
disk and RAM, and prints a verdict with honest tok/s estimates

OpenAI-compatible server (midge serve): streaming, session prefix
cache (usage reports standard cached_tokens), reasoning controls,
per-request seeds, clean context-overflow handling with engine recovery
Tool/function calling: tools rendered into harmony, tool invocations
returned as message.tool_calls (streamed as deltas), tool-role history
replays correctly — the full agent loop for OpenClaw/Hermes-style agents
Hybrid mode: --upstream URL --route auto relays permitted requests at
cloud speed with transparent local fallback, per-request privacy pinning,
circuit breaker so a dead upstream taxes exactly one request. Zero network
calls unless you configure an upstream

AVX2+FMA kernels, runtime-dispatched: 8–9× (0.48→3.98 GB/s/core;
1.8→12.2 tok/s warm end-to-end on one core)
Batched prefill (expert-major): 3.3× cold prompt processing under
memory pressure — the regime bigger-than-RAM models live in
GPU via the MLX backend (Apple Silicon; CUDA via mlx[cuda])

./midge ui: instant web interface — pick a model (or paste any HF
repo), watch download/convert live, engine builds at the end, chat panel
on the same API agents use
./install.sh + requirements.txt; friendly errors on machines missing
anything; Python 3.9 restored; macOS builds without OpenMP instead of failing
Clean-room audited: every documented command verified on a bare venv

Still honest: real-weight runs on gpt-oss checkpoints and physical NVIDIA
hardware remain community-verifiable territory — reports welcome.