Skip to content

0.2.0

Choose a tag to compare

@mattt mattt released this 21 Sep 16:53
Immutable release. Only release title and notes can be modified.
1760ed0

Changes

  • MLXDecisionModel no longer caches the shared prompt prefix by default. mlx-swift 0.31.6 and earlier include an MLX version whose Metal attention kernel applies the causal mask incorrectly when a prompt is prefilled in two parts, which can change answers. MLX 0.31.2 fixes this (ml-explore/mlx#3271). Pass prefixCaching: true to opt back in after checking cached results for your model (#7)

Additions

  • DecisionSession.Diagnostics reports duration, the wall-clock time for each local answer, excluding model loading and prefix filling (#5)
  • MLXDecisionModel has an opt-in closedThinkFallback that appends a closed thinking block for reasoning models whose chat template ignores enable_thinking: false. The approach follows Daisuke Majima's coreai-kit#13 (#6)

Under the hood

  • MLX CI builds no longer fail on Metal compiler paths restored from another runner's cache (#4)