v0.4.18

What's Changed

Added

• POST /v1/systemone answers a fixed set of questions about a state with a
  served DiffusionGemma model, in the Jev decision API shapes.
• think: "auto" on /v1/systemone writes a thought only when an answer is
  unsure.
• gmlx systemone sends a decision request to a server or answers it
  offline, and server.systemone holds the route's settings.
• The docs are published at https://asher.github.io/gmlx/ with navigation
  and search, rebuilt from each release.
• --no-gpu-keepwarm and server.gpu_keepwarm: false turn GPU keep-warm
  off. The flag could only turn it on, which it already was by default.
• The server logs the block and exact prompt-cache tiers at load, as it
  already did for the checkpoint tier.
• gmlx doctor prints the macOS version and warns when it is older than
  26.2, the version that mlx-kquant's kernels need.

Changed

• A server request that sets no output cap runs until the model stops or its
  context fills, as gmlx run does, instead of stopping at 2048 tokens.
  gmlx serve --max-tokens and a profile's max_tokens still set a cap.
• The package and Homebrew license metadata declare Apache-2.0 beside
  BUSL-1.1 and MIT, since gmlx ships Apache-2.0 files from omlx, misaki and
  vLLM.
• gmlx distill train refuses a view aligned before this release when the
  student is a gemma-4 or gemma-1.1 GGUF, since its stop ids changed. Run
  gmlx distill align again.
• Requires mlx-kquant 0.4.14. On GPUs with NAX (M5), its kernels speed up
  speculative decoding on Q4_0, Q8_0, Q2_K to Q6_K, PQ2_0 and
  PTQ1_0 targets, and batched decode on every codec.
• DFlash 2 decoding runs faster on targets with a float output head, such as
  Q8_K_XL Qwen3.8-27B. GMLX_DRAFT_HEAD=f16 restores the previous head.
• gmlx run, chat and bench with MTP or a DFlash 2 drafter decode
  faster, on the command-buffer settings gmlx serve already decoded with.
• A single gmlx serve request with MTP or a DFlash 2 drafter spends less
  time between speculative rounds.
• Speculative decoding on Qwen3.x targets verifies faster, most of all with
  DFlash 2 drafters and on models with 2 KV heads such as Qwen3.6-35B-A3B.
• MTP on Qwen3.5-9B, and other models with 16 query heads and 4 KV heads,
  runs faster at long context when it verifies 3 or 4 tokens.
• Speculative decoding on Gemma-4 with its assistant drafter runs faster at
  long context, by 19% on E2B and 5% on 31B with a 30k-token prompt.
• Gemma-4 E2B and 12B decode faster at long context, by 2% on E2B with a
  30k-token prompt.
• gmlx init writes only the settings you chose and the model entries,
  under a header that links the configuration reference, instead of every
  option as a commented example.
• Only a pinned model is held for the life of the server. The default or
  sole model of a config still loads at start, but it now unloads after its
  idle timeout or under memory pressure like any other model.
• Every pinned model loads at start, in file order, instead of only the
  first one.
• The server quantizes the KV cache from the first token by default, as
  gmlx run and gmlx chat do. A single request kept an fp16 cache up to
  5000 tokens unless quantized_kv_start was set.
• gmlx chat with --base-url, --host, --port, --api-key or
  --no-start runs as a server client, as --server does, instead of
  refusing the flag.

Fixed

• With --mmproj and --kv-quant-scheme kvarn, run and chat keep the
  KV cache in fp16, as their warning says. An explicit --kv-bits built an
  affine cache instead.
• gmlx init -i pre-fills the wizard from every flag it is given, such as
  --with-stt, --disk-cache and --idle-ttl. It used only the output
  file, the port and the first models folder.
• A server started without a config file ignores SIGHUP with a log line.
  The signal stopped it.
• gmlx doctor checks the MCP servers in assistant.mcp, which
  gmlx chat --assistant always uses. It skipped them unless talk or an
  alias used them.
• gmlx launch open-webui keeps an AUDIO_TTS_VOICE that you export
  instead of replacing it with the Kokoro default.
• The default prompt-cache disk folder follows XDG_CACHE_HOME, like the
  other gmlx caches.
• cache.disk.namespace now names the disk-tier namespace. The model path
  always took its place, so the key had no effect.
• The server applies stream_fast_disk and gmlx serve --stream-fast-disk
  to the decode feeder. Only run and chat applied them.
• A chat theme with a new name under themes no longer crashes the markdown
  renderer, and a theme that shadows a built-in renders with its own colors.
• The ptk_toolbar theme key now colors the chat toolbar. It was never
  applied.
• gmlx serve, gmlx run and gmlx chat serve the models that a config's
  discover block finds. Only gmlx list and gmlx rm saw them, so a
  request for a discovered id failed.
• The install docs state macOS 26.2 as the minimum for every install route.
  They described a build from source for older macOS, which the Metal
  kernels do not support.
• The --draft-block-size help and docs called N the drafted tokens. A round
  drafts N-1, so llama-server's --spec-draft-n-max M matches N = M+1.
• DiffusionGemma prompts on gmlx run and on the server started with two BOS
  tokens, one from the chat template and one from the tokenizer. They now
  start with one.
• gemma-4 GGUFs did not stop on <|tool_response> after a tool call, and
  some did not stop on <eos>, so a reply could run past its end.
• gemma-1.1 GGUFs that type <end_of_turn> as text did not stop on it, so a
  reply could show the tag or run on past the turn.
• Concurrent requests on a gemma-4 model with a drafter could decode wrong
  tokens, where the same request sent alone decoded right.
• With a drafter, a repeated gemma-4 prompt shorter than the sliding window
  prefilled in full unless its length fell on a block boundary.
• gmlx distill gen with --serve-arg=--adapter exited because the server
  lists the adapted model and <id>-base. It now sends its requests to the
  adapted model.
• A model entry with ttl_s: null took the server's idle timeout. It now
  never unloads, as null already meant under server.defaults.

Full Changelog: v0.4.17...v0.4.18