Skip to content

v0.28.0.dev20260907121714

v0.28.0.dev20260907121714: Bump MLX to 0.32.1 and update the mlx-lm pin (#723)
@FENP FENP tagged this 07 Sep 12:11 
## Summary

- Bump the exact MLX runtime pin from 0.32.0 to 0.32.1.
- Advance the MLX-LM pin from `254d153f` to `9e6acca6`.
- Update contiguous-cache handling for MLX-LM's `ArraysCache.state` to
`ArraysCache.cache` rename.
- Adapt GGUF, LoRA, TurboQuant, pooling, and XLM-RoBERTa code to the
updated dependency APIs and type information.

## Motivation

This repository-wide dependency and compatibility update was split from
#607 so that the Ling-3.0 MLA/KDA implementation can remain
model-scoped. PR #607 will be rebased on top of this change.

The pinned MLX-LM revision requires MLX 0.32.1 and contains APIs not
present in the latest MLX-LM release. No other open PR was found for the
same MLX 0.32.1 or `ArraysCache` compatibility update.

## Validation

Environment: Apple M3 Pro with 18 GB unified memory, macOS 14.4.1,
Python 3.12.12, MLX 0.32.1, and MLX-LM at
`9e6acca691e64d6d8bb808c328fcdea459099cca`.

```bash
VLLM_METAL_BUILD_FROM_SOURCE=1 \
  .venv/bin/python -m pytest -p no:cacheprovider \
  -m "not slow" tests -q

.venv/bin/python -m ruff check .
.venv/bin/python -m ruff format --check .
.venv/bin/python -m mypy vllm_metal
git diff --check origin/main...HEAD
```

Results:

- Pytest: `1997 passed, 18 skipped, 40 deselected`.
- Ruff: all checks passed; 291 files already formatted.
- Mypy: no issues in 137 source files.
- Git whitespace check: passed.

## AI assistance

AI assistance was used to inspect the dependency changes, separate the
compatibility update from #607, review the diff, and run the reported
tests. The submitter reviewed the changed code and remains responsible
for the contribution.

Signed-off-by: FENP <yuanyongjie.yyj@antgroup.com>
Assets 2
Loading