Repository navigation
SmolVLA on Apple Silicon: an MLX runtime using mlx-vlm’s SmolVLM as an architecture reference #2165
daniiarabdiev
started this conversation in
Show and tell
Replies: 1 comment
|
@daniiarabdiev Very cool! Have you used it to actually train and control an arm or other hardware? |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I built mlx-smolvla, an MLX-native runtime for SmolVLA checkpoint inference, LeRobot async-inference-protocol serving, and preview LoRA fine-tuning. mlx-vlm’s SmolVLM implementation was a useful architecture reference while porting the backbone; the base runtime does not depend on mlx-vlm, Torch, Transformers, or LeRobot.
The repository includes PyTorch comparison gates and separates strict CPU compatibility checks from the Metal execution path. Training remains a research preview. Serving is loopback-tested, and sustained robot task completion remains unvalidated.
Prior art: tokimoa/smolvla-mlx (Hugging Face, 2026-07-29) is an earlier independent inference-only port; tokimoa also published pi0-mlx and groot-n1.7-mlx. mlx-smolvla is not a fork of that port or mlx-vlm.
Sharing the result here to acknowledge the reference implementation and invite feedback from developers working with SmolVLM-based models in MLX, especially on checkpoint loading and numerical comparison methodology.
Repo: https://github.com/daniiarabdiev/mlx-smolvla
— Daniiar (ML infra engineer; SO-101 + M5 Pro on the bench)
All reactions