Skip to content

v0.102.2

Choose a tag to compare

@basnijholt basnijholt released this 25 Jul 20:16
2212f0c

Bug fixes

  • MLX Whisper: stop unbounded memory growth under --ttl 0 (#620). The long-lived transcription subprocess never released MLX's reusable Metal buffer cache, so it ratcheted up to the largest working set ever seen and eventually got pushed to swap — one deployment reached a 17.1 GB physical footprint for a 2.9 GB model, with a 5.2s clip degrading from ~2.0s to ~5.2s. The backend now calls mx.clear_cache() after each request, including on failure. Only the reusable cache is freed, not live arrays, so model weights stay resident and inference stays hot; measured cost is ~7.6 ms per request (~0.3%).

  • Require mlx>=0.24 for the mlx-whisper extra (#620). The top-level mx.clear_cache() does not exist before mlx 0.24 — 0.23.2 and older only expose the deprecated mx.metal.clear_cache() — and mlx-whisper itself only requires mlx>=0.11. Without the floor, an environment resolving an older mlx would raise AttributeError on every transcription.

Internal

  • Pin uv in the uv-lock and sync-requirements pre-commit hooks so the generated agent_cli/_requirements/*.txt files depend on the config rather than on how fresh each contributor's hook cache is (#620).
  • Render the shared consoles without colour during tests (#625).
  • Drop the broken Homebrew tap dispatch from the release workflow (#623).