rMLX 0.2.3
Multi-model registry hardening. Two --registry serving bugs fixed: the
multimodal encoder-output cache no longer leaks vision/audio features across
models, and eager model preload now respects --max-loaded-models. No breaking
changes.
Fixed
- Multimodal encoder-output cache cross-model leak. In
--registry
multi-model mode the vision/audio encoder cache was keyed on the
post-preprocess content hash only, so a cached image encoding produced for one
model (projected to itshidden_size) was returned to a different model for
the same image — a vision-feature shape mismatch (HTTP 503) when the hidden
sizes differed. The cache key now folds in a stable per-model signature, so
entries are never shared across models; same-model repeats still hit. (#132) - Registry eager-preload ignored
--max-loaded-models.rmlx serve --registrypreloaded every model at startup even with a smaller resident cap,
paying the full load cost for models that were immediately evicted (a
~5-minute boot for a 13-model registry). Preload is now bounded to at most
--max-loaded-modelsentries (the alphabetically-first ids, since the
registry is id-sorted); the rest load on demand. (#133)
Changed
README.mdrefreshed to 0.2.3 with an accurate "What works" summary, and
docs/CLI.mddocuments that the multimodal cache key now includes model
identity (no cross-model sharing) and that registry preload is bounded to the
resident cap.