You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Changes
MLXDecisionModel no longer caches the shared prompt prefix by default. mlx-swift 0.31.6 and earlier include an MLX version whose Metal attention kernel applies the causal mask incorrectly when a prompt is prefilled in two parts, which can change answers. MLX 0.31.2 fixes this (ml-explore/mlx#3271). Pass prefixCaching: true to opt back in after checking cached results for your model (#7)
Additions
DecisionSession.Diagnostics reports duration, the wall-clock time for each local answer, excluding model loading and prefix filling (#5)
MLXDecisionModel has an opt-in closedThinkFallback that appends a closed thinking block for reasoning models whose chat template ignores enable_thinking: false. The approach follows Daisuke Majima's coreai-kit#13 (#6)
Under the hood
MLX CI builds no longer fail on Metal compiler paths restored from another runner's cache (#4)