Skip to content

vMLX v1.3.31

Choose a tag to compare

View all tags
@jjang-ai jjang-ai released this 09 Apr 00:53
· 3824 commits to main since this release
v1.3.31
eb193bd

4-agent audit 2026-04-07 — Phase 5 polish (22 fixes)

Cross-agent audit delivering cross-session cache sharing, Aho-Corasick stop matching, SSM companion cache extraction, Gemma 4 native shim, and the mlx-lm 0.31.2 bump.

263/263 cache tests PASS — 27/27 live cells across Qwen3 / Gemma 4 / Nemotron Cascade 2 / Mistral 4.

Agent 1 — Cache (Coordinator)

  • LRU+Trie cross-session prefix sharing on MemoryAwarePrefixCache (production default) — system → user → assistant priority eviction
  • cache_type LRU + per-type bucket eviction on BlockAwarePrefixCache (paged / hybrid auto-switch)
  • Byte-budget eviction + --prefix-cache-max-bytes CLI flag
  • compute_model_cache_key() content-derived cache identity (survives JIT reload, prevents cross-config pollution)
  • Disk SQLite: cache_type column + priority eviction + last-fetch type round-trip in L1 backfill
  • Scheduler dispatcher now fires segment boundaries on memory-aware + paged (not just legacy)
  • SSM companion cache stats surfaced to CachePanel UI (nbytes_mb)

Agent 2 — State Machine / API Gateway / Reasoning / Tool Parsers

  • state_machine.py — vendored Aho-Corasick SequenceStateMachine from mlx-lm 0.31.2
  • engine/batched.py F11 — _compute_segment_boundaries helper + 2 wire sites
  • engine/simple.py — non-streaming prompt_tokens fallback + strip reserved keys from chat_template_kwargs
  • mllm_batch_generator.py — narrow _ssm_state_cache.clear() blast radius
  • scheduler.deep_reset — wires reset_factory_cache() on shutdown
  • server.py — public size/max_entries on /v1/cache stats

Agent 3 — SSM / Hybrid / Batched Mamba

  • BatchMambaCache.lengths/advance/prepare/finalize backport from mlx-lm 0.31.2
  • SSMCompanionCache extracted to vmlx_engine/utils/ssm_companion_cache.py with is_complete per-entry flag
  • model_key mixed into SHA-256 (defends against cross-config pollution)
  • is_hybrid_ssm_cache/model/config polymorphic helpers
  • mamba_cache.advance() in-place -= (~40 alloc/step saved on Nemotron Cascade)
  • BatchGenerator empty-cache crash fix (ISSUE-A3-003)
  • +16 new tests in test_ssm_companion_cache.py (37 total)

Agent 4 — Loaders / JANG / Smelt / Tokenizers

  • is_mla_model centralized helper (walks top → text_config → language_config → llm_config)
  • Gemma 4 native text MoE local-port shim (forward-compat defers to upstream)
  • utils/tokenizer.py — trust_remote_code=True default (silences HF warning, unblocks custom tokenizers)
  • api/utils.py — is_mllm_model() result cache (kills INFO log spam)
  • utils/smelt_loader.py — defensive safetensors offset-vs-EOF guard
  • utils/jang_loader.py — §6.3 VLM remap + §6.4 Gemma 4 PLE guards
  • jang_tools bundling fix (ISSUE-A4-001)
  • ImportError chaining in models/llm.py (ISSUE-A4-003)

Core

  • R-001: mlx-lm 0.30.2 → 0.31.2 bump (quad-verified across 4 sync surfaces)
  • Removed orphan disk-streaming artifacts (ssd_generate, streaming_wrapper, weight_index) superseded by memory_cache + disk_cache architecture
Assets 2
Loading