Skip to content

rMLX 0.2.3

Choose a tag to compare

@Pushkinist Pushkinist released this 18 Jun 08:13
· 89 commits to main since this release
a0a2f74

Multi-model registry hardening. Two --registry serving bugs fixed: the
multimodal encoder-output cache no longer leaks vision/audio features across
models, and eager model preload now respects --max-loaded-models. No breaking
changes.

Fixed

  • Multimodal encoder-output cache cross-model leak. In --registry
    multi-model mode the vision/audio encoder cache was keyed on the
    post-preprocess content hash only, so a cached image encoding produced for one
    model (projected to its hidden_size) was returned to a different model for
    the same image — a vision-feature shape mismatch (HTTP 503) when the hidden
    sizes differed. The cache key now folds in a stable per-model signature, so
    entries are never shared across models; same-model repeats still hit. (#132)
  • Registry eager-preload ignored --max-loaded-models. rmlx serve --registry preloaded every model at startup even with a smaller resident cap,
    paying the full load cost for models that were immediately evicted (a
    ~5-minute boot for a 13-model registry). Preload is now bounded to at most
    --max-loaded-models entries (the alphabetically-first ids, since the
    registry is id-sorted); the rest load on demand. (#133)

Changed

  • README.md refreshed to 0.2.3 with an accurate "What works" summary, and
    docs/CLI.md documents that the multimodal cache key now includes model
    identity (no cross-model sharing) and that registry preload is bounded to the
    resident cap.