Repository navigation
v0.1.0
First release of North Micro Vision Studio: a local Streamlit app for running Cohere's North Micro Vision — a 2.4B-parameter, Apache-2.0 vision-language model that reads images at their native resolution rather than squashing them to a fixed square — on Apple Silicon via MLX. Documents, charts, OCR and visual grounding, with nothing leaving the machine.
It runs the bf16 conversion only (mlx-community/North-Micro-Vision-Instruct-bf16, ~5 GB), so results stay comparable to Cohere's published numbers.
Two pages
Chat — multi-turn document Q&A, OCR, chart reading and captioning. Each image stays bound to the turn it arrived on, so a follow-up question can refer back to a page uploaded several turns earlier. Responses stream token by token.
Grounding — single-shot detection with a bounding-box overlay, a table of parsed coordinates and a PNG download. Boxes come back on a 0–1000 grid rather than in pixels, so the overlay is drawn on your full-resolution upload even though the encoder saw a downscaled copy.
Worth knowing
- Token cost scales with pixel area. The encoder spends one token per 32×32 px block, so
tokens = pixels / 1024. The image budget slider is the only lever on prefill latency and KV-cache memory; its ceiling is 3,868,706 px — A4 at 200 dpi, the largest resolution Cohere trained on. - Every MLX call runs on one pinned worker thread. mlx-vlm binds its generation stream to whichever thread first imports it, and Streamlit runs each rerun on a fresh thread, so a naive app works exactly once and then fails. The single worker also serialises GPU work across browser tabs, which matters because MLX generation is not reentrant.
- No system prompt is ever sent. Cohere states the model was not trained to follow them, so task instructions go in the user turn.
Requirements
An Apple Silicon Mac (16 GB unified memory is enough; peak is ~6.2 GB), Python 3.11–3.13, and uv. The first launch pulls ~5 GB of weights into the Hugging Face cache.
uv sync
uv run streamlit run streamlit_app.pyMeasured on an M2 Max (32 GB)
Weight load 3.4 s · prefill ~590 tok/s · decode 72–88 tok/s · peak 6.17 GB
What's changed
- Initial commit (f647374)
- Add Streamlit studio for North Micro Vision on MLX (c7a5d08)
- Fix defects found in code review (103cc2b)
- Enforce the module-scope MLX import ban with a hook (06222a9)
- Ignore .claude/settings.local.json (1cae663)
- Add CI workflow and draft releases on version bump (f29e4e7)
- Pin setup-uv to an exact release (f26c3c9)
Full Changelog: https://github.com/darylalim/north-micro-vision-studio/commits/v0.1.0