Skip to content

Project Zero v0.2.0

Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 22 Oct 22:25
· 27 commits to master since this release
a49b9a5

Notice (2026-10-23): security advisories. This release fixes two security issues present in v0.1.0: prompt injection through chat messages (GHSA-6r45-5hm9-j4qc, medium) and an escape from the /agent command sandbox (GHSA-j6vf-qvf9-fc89, low). The --static-dir path-traversal fix listed under Security below was never present in a published release, so it has no advisory.


Changed

  • Breaking: Rename the binary to pz (was adaptive_ai_engine); pz --version prints Project Zero <version>.
  • Breaking: Rename release downloads to project-zero-<version>-x86_64-unknown-linux-gnu.tar.gz, unpacking into a directory of the same name (was adaptive_ai_engine-<version>-x86_64-linux.tar.gz).
  • Publish each GitHub release's notes from this changelog instead of an auto-generated commit list.
  • Cut startup from 20–70 s to under 1–8 s by caching the DRAM bandwidth probe per host (--calibrate or PZ_BW_REPROBE=1 re-measures), zeroing large buffers lazily and indexing tokenizer lookups.
  • Reduce resident memory by releasing the file pages of converted tensors and committing KV cache rows only as far as a generation can reach.
  • Quantize the INT8/INT4 classifier (LM head) block-wise instead of with one scale per row.
  • Speed up long-context decode with --kv-type i4 by about 27% with AVX2/AVX-512 int4 KV kernels: SmolLM2-1.7B Q4_K_M, about 2,200-token context, 4 threads: 8.32 → 10.58 tok/s on a 4-vCPU AVX-512 Xeon.
  • Speed up mixture-of-experts decode about 1.7–1.8× with a fused expert FFN, 8-bit-activation integer kernels and dynamic expert scheduling: DeepSeek-V2-Lite-Chat Q4_K_S at 4 threads goes from 6.3–7.3 to 11.7–12.6 tok/s (llama.cpp: 9.4–10.5) on a 4-vCPU AVX-512 VNNI Xeon.
  • Keep Q4_K/Q6_K output layers and Q5_0/Q5_1/Q8_0 projections in their quantized format instead of widening them, reducing DeepSeek-V2-Lite resident memory from 12.0 to 10.6 GB.

Deprecated

  • Deprecate the adaptive_ai_engine name: make and the release archive still provide it as a symlink to pz that prints a notice; replaced by pz and removed in v0.3.0.

Added

  • Add Qwen 3.5/3.6 hybrid-attention models (e.g. Ternary-Bonsai-27B) and the Q2_0 quant.
  • Add Qwen3-MoE GGUF models.
  • Add Mamba-1 state-space models.
  • Add every remaining mainline GGUF quant type: Q4_1, Q8_1, Q8_K, BF16 and the IQ1–IQ4 families.
  • Add --perplexity <file> (with --ppl-ctx, --ppl-chunks) and --raw-prompt for accuracy checks against llama.cpp.
  • Add speculative decoding with --draft-model and --spec-length.
  • Add hot-swappable LoRA adapters (--lora).
  • Add grammar-constrained JSON mode.
  • Add a configurable sampler chain (--top-k, --min-p, --repeat-penalty, --presence-penalty, --frequency-penalty, --repeat-last-n) and stop sequences (--stop).
  • Add toggleable cryptographic watermarking (--watermark*, POST /v1/watermark/detect).
  • Add best-of-N selection and step-level tree search with an optional process reward model (--mcts*, --prm*, request field mcts).
  • Add chat-template overrides from a file (--chat-template-file) or per request (chat_template; disable with --no-request-template).
  • Add context sizing flags --ctx-size and --kv-ram-fraction.
  • Add int8 and int4 KV cache storage (--kv-type f32|i8|i4).
  • Add disk-backed KV cache (--kv-offload <dir>).
  • Add KV prefix reuse across API requests and a multi-entry prompt snapshot cache (--prompt-cache, request field cache_prompt).
  • Add opt-in CORS (--cors, --cors-origin) and API-key auth (--api-key), /metrics, OpenAPI docs at /docs and request cancellation.
  • Add GET /v1/config, publishing server defaults, valid ranges and feature flags.
  • Add a FIFO generation queue (--queue-size, --queue-timeout).
  • Add parallel serving (--parallel N, --parallel-batch N): up to N chat requests generate at once, batched into one forward pass per token for dense attention models, with per-slot KV memory taken from a shared budget; /v1/config, /metrics and Settings → Server defaults show and save it. With 4 concurrent clients aggregate throughput rises from 45 to 101 tok/s (SmolLM2-135M Q8_0) and from 12.7 to 29.8 tok/s (SmolLM2-1.7B Q4_K_M) on a 4-vCPU Xeon at 4 threads.
  • Add graceful shutdown on Ctrl+C/SIGTERM in --server mode.
  • Add distributed pipeline inference over TCP (--remote-layers, --rpc-worker).
  • Add notify-only update checks (--check-update, GET /v1/update, --no-update-check, --update-*).
  • Add CHANGELOG.md to the release archive.
  • Add release-notes and changelog links for the running build to GET /v1/update (release_notes_url, changelog_url) and to the web UI under Settings → Updates.
  • Add the web chat UI with image upload, settings rendered from /v1/config (including the chat-template editor and tree search), a watermark panel, a usage footer and an update banner.
  • Add colour, progress, live token statistics, markdown rendering and a startup banner to the CLI.
  • Add persistent settings: pz config (interactive setup), pz config list|get|set|add|unset|path, a config file at ~/.config/project-zero/config.toml (PZ_CONFIG overrides the location) and PZ_<KEY> environment variables. Precedence is command-line flag > environment > file > default; --no-config ignores the file and environment.
  • Add Settings → Server defaults to the web UI: see whether each server value comes from the config file, an environment variable or a command-line flag, and save changed settings (and the web UI title) as server defaults for the next start (GET /v1/config config, POST /v1/config/defaults; disable with --no-config-write).
  • Add install.sh: a one-line, checksum-verified installer for Linux x86-64 that puts pz in ~/.local/bin without root, finds the newest release, offers pz config, and uninstalls with --uninstall; it ships with each release and is listed in SHA256SUMS.
  • Add a guided pz config setup for non-technical users: each step explains in plain words what it means, shows example choices and their effect, and says how to change it later; it can download a checksum-verified starter model sized to the computer's free memory (list replaceable with PZ_MODEL_CATALOG) and open the chat page when done. The installer hands over to it.
  • Add faster kernels: K-6 5-trit LUT ternary (AVX-512BW), plain NEON ternary, AVX-512 VNNI Q2_0, llama.cpp-compatible Q4_K x8 repack, fused zero-copy Q6_K and batched speculative-verify kernels.

Fixed

  • Fix chat replies running past the end of the answer when a model ends its turn with a control token its metadata does not declare (e.g. Falcon3-1.58bit's >>UNUSED_1<<): in chat mode any control token the chat template never writes now ends the turn, so no --stop workaround is needed.
  • Fix BitNet I2_S GGUF files (the official BitNet b1.58 2B-4T GGUF, Falcon3-1.58bit) producing incoherent text: weights are now read in bitnet.cpp's interleaved 128-weight layout with the per-tensor scale stored after the packed data.
  • Fix pz config set repeat-last-n -1 being rejected although --repeat-last-n -1 is valid.
  • Fix JSON mode (--json, response_format: json_object) sometimes starting with a bare string or padding with whitespace at high temperature: output now always starts with { or [ and ends at the matching bracket.
  • Fix a nonexistent or unreadable --model path being reported only after the first-run hardware calibration, which can take minutes.
  • Fix a SIGILL crash on CPUs that advertise AVX-512 VBMI in CPUID but cannot execute it.
  • Fix the macOS build, broken by a Linux-only call in the disk-backed KV cache (--kv-offload).
  • Fix oversized buffer requests on macOS and Windows getting the process killed instead of failing with an out-of-memory error.
  • Fix AVX-512 VNNI kernels being disabled on every CPU by a startup self-test that expected the wrong result.
  • Fix mixture-of-experts routing ignoring the GGUF gating function, weight normalization and weight scale (affected Qwen3-MoE and DeepSeek-V2).
  • Fix generation not stopping at a GGUF model's end-of-turn tokens.
  • Fix the CLI silently generating nothing when a prompt cannot be tokenized; it now prints an error (the API already returned 400).
  • Fix --lora being ignored when combined with --draft-model.
  • Fix the INT8/INT4/BF16 classifier option having no effect on Q2_0-native models.
  • Fix wrong output from IQ4_NL weights and mojibake from byte-level BPE detokenizing.
  • Fix JSON mode accepting numbers with leading zeros.
  • Fix a data race in thread pool shutdown.
  • Fix a saved calibration overriding the automatically chosen SIMD tier.
  • Fix the DRAM bandwidth probe under-reporting about 3× and hardware ceilings using fixed BitNet-2B sizes for every model.
  • Fix release builds silently linking objects compiled for a different build variant.
  • Fix a streaming API reply continuing to generate after its client disconnected, holding the generation slot until max_tokens.
  • Fix stopping the server (Ctrl+C/SIGTERM) while a request was generating: the generation now stops before the model is unloaded instead of running on freed memory.
  • Fix pz config set best-of-max accepting values up to 1024 that --best-of-max then rejects at startup (the limit is 64 in both).

Security

  • Fix prompt injection through chat messages: control tokens such as <|im_start|> or <|eot_id|> typed or pasted into a message (or returned by a tool) are now read as plain text, so they can no longer end a turn or start a forged system turn; prompts containing unknown <|...|> text are no longer rejected.
  • Fix path traversal out of --static-dir when serving web assets.
  • Harden the agent tool sandbox with an argument-level command policy and fix an exit-code race, a symlink escape and PATH hijacking.

Full changes: v0.1.0...v0.2.0