midge v0.2.1 — first-run reliability fixes

Maintenance release. If you hit an install or load problem on v0.2.0, update to this release — it fixes several first-run issues.

Fixes

• Model load no longer hangs silently. The engine now prints load progress (# loading, # mapped experts (N GB)) and the client waits for readiness with a bounded, correct timeout — a stalled or misconfigured load fails with a clear message instead of hanging forever.
• midge search no longer crashes. It previously raised a TypeError against newer huggingface_hub; it now adapts to the installed version and degrades gracefully offline.
• Friendly errors on a fresh machine. midge --help and every command now print a clear "missing package — run pip install -r requirements.txt" message on a dependency-free Python instead of a traceback.
• Optional-dependency test suites skip cleanly (make test-mlx, make test-server) when mlx/openai aren't installed, instead of erroring.
• Added SUPPORT.md and an issue template with load-hang triage and the exact startup lines to expect.

Verified

AddressSanitizer + UBSan clean; clang + strict gcc builds pass; fresh-clone install.sh in a bare venv; C, MLX, and UI suites green in CI; the convert→load→chat flow shows progress with no hang. Regression tests added for the search crash and the bare-Python guard.

See SUPPORT.md if you hit trouble.

midge v0.2.0

midge v0.2.0 — engine for everyone, agents welcome

Everything since v0.1.0, all CI-validated:

Run any compatible model — and know before you download

• Engine generalized beyond gpt-oss: Mixtral-style and Qwen3-MoE-style
  families (plain SwiGLU, QK-norm, norm_topk_prob routers), validated
  per-position against the NumPy reference on both engines
• midge check <hf-repo|path>: fetches only config.json, judges
  architecture compatibility with named reasons, benchmarks your kernel,
  disk and RAM, and prints a verdict with honest tok/s estimates

Agents

• OpenAI-compatible server (midge serve): streaming, session prefix
  cache (usage reports standard cached_tokens), reasoning controls,
  per-request seeds, clean context-overflow handling with engine recovery
• Tool/function calling: tools rendered into harmony, tool invocations
  returned as message.tool_calls (streamed as deltas), tool-role history
  replays correctly — the full agent loop for OpenClaw/Hermes-style agents
• Hybrid mode: --upstream URL --route auto relays permitted requests at
  cloud speed with transparent local fallback, per-request privacy pinning,
  circuit breaker so a dead upstream taxes exactly one request. Zero network
  calls unless you configure an upstream

Speed (measured, docs/SPEED.md)

• AVX2+FMA kernels, runtime-dispatched: 8–9× (0.48→3.98 GB/s/core;
  1.8→12.2 tok/s warm end-to-end on one core)
• Batched prefill (expert-major): 3.3× cold prompt processing under
  memory pressure — the regime bigger-than-RAM models live in
• GPU via the MLX backend (Apple Silicon; CUDA via mlx[cuda])

Zero-to-chat

• ./midge ui: instant web interface — pick a model (or paste any HF
  repo), watch download/convert live, engine builds at the end, chat panel
  on the same API agents use
• ./install.sh + requirements.txt; friendly errors on machines missing
  anything; Python 3.9 restored; macOS builds without OpenMP instead of failing
• Clean-room audited: every documented command verified on a bare venv

Still honest: real-weight runs on gpt-oss checkpoints and physical NVIDIA
hardware remain community-verifiable territory — reports welcome.

midge v0.1.0 — tiny engine, immense models

First release.

midge runs 100B+ mixture-of-experts LLMs on ordinary machines by keeping the dense trunk resident and streaming routed experts from disk. First supported models: openai/gpt-oss-120b and gpt-oss-20b (a 75 GB disk and 8 GB of RAM are enough for the 120B model).

Highlights

• C engine (Linux): mmap'd experts with the OS page cache as the expert cache — bounded memory by construction, graceful under pressure. YaRN RoPE, attention sinks, sliding-window ring KV, MXFP4/int4/int8 kernels, usage-aware --preload-gb warmup.
• MLX engine (Apple Silicon): the same converted model directory, experts streamed through a byte-budgeted LRU in unified memory, container blobs consumed directly by MLX's quantized kernels (affine int4 and native mode="mxfp4").
• Converter: torch-free, shard-by-shard streaming (peak disk ≈ container + one shard), resumable after interruption, lossless MXFP4 transcode.
• Validation: both engines checked per-position against an independent NumPy reference across all dtypes in CI; MXFP4 decoding verified bit-exactly against transformers' official dequantizer and cross-checked against OpenAI's reference implementation.

Quickstart

git clone https://github.com/drmihirbrahme/midge && cd midge
make && pip install numpy tokenizers huggingface_hub
./midge convert openai/gpt-oss-20b models/gpt-oss-20b
./midge chat models/gpt-oss-20b

See the README for the Apple Silicon (./midge-mlx) usage, capacity planning (./midge plan), and troubleshooting.