Releases · ml-explore/mlx-swift-lm

Release list

3.31.4 Latest

Latest

davidkoski released this 30 Jun 16:31

3.31.4

bd4b743

New release with the last couple months of new features and fixes.

Upcoming changes (potential instability on main):

https://github.com/ml-explore/mlx-swift -- when we pick up a new release of this, e.g. for ml-explore/mlx-swift#429 it will require a newer swift-tools-version (newer Xcode, though the SDK requirements remain the same)
#334 -- a big PR that adds two new libraries

Use this tag if you want to isolate.

What's Changed

Guard embedder inputs against maxPositionEmbeddings by @alankessler in #163
Doc corrections for v3 by @DePasqualeOrg in #216
add fallback links to the readme -- docs are not yet built by @davidkoski in #218
Fix Qwen35 VLM crash on text-only inference by @dirvine in #149
Add coherence integration tests by @DePasqualeOrg in #235
fix: flatten prompt in TokenRing.loadPrompt to handle 2D inputs by @spokvulcan in #170
Pass tools schema to ToolCallProcessor for type-aware parsing by @alankessler in #167
Forward tools and additionalContext in GlmOcr and SmolVLM2 processors by @alankessler in #174
fix: avoid shared CIContext in media processing by @Meapri in #268
feat: add ParoQuant (pairwise rotation quantization) support by @spokvulcan in #164
fix ropeOffset use in Gemma3NText by @davidkoski in #280
Handle stringified JSON tool call arguments by @kuosuko in #172
fix build by @davidkoski in #281
fix segsum dtype promotion -- 2x memory waste on hybrid SSM models by @TheTom in #229
Remove TokenLoopHandler Sendable requirement by @thisray in #284
feat: expose speculative decoding in ChatSession (#181) by @VDurocher in #193
Fix EmbeddingGemma init-order crash + dense head hidden size by @0xweb3r in #223
fix: disable CIContext intermediate caching to prevent IOSurface exha… by @Satchitananda in #226
fix Gemma 4 MoE router -- softmax order + fuse norm dispatches by @TheTom in #228
add FusedGateUpSwitchGLU -- single fused gate_up_proj for MoE models by @TheTom in #227
Fix UserInput init not setting self.images / self.videos (#182) by @john-rocky in #253
pipeline prefill chunks with asyncEval -- 10x on GDN models by @TheTom in #225
fix gated delta state precision -- fp32 state to match Python by @TheTom in #224
Fix(Docs) non-compiling README quick-start example by @programVeins in #265
combine docs check with build to reuse a macos runner by @davidkoski in #286
Remove SDK build version command from workflow by @madrob in #306
models should not mutate state during eval by @davidkoski in #283
Fix: tolerate 1D token inputs in Gemma 4 VLM forward pass by @brdcastro in #241
fix Idefics3 prepare() to honor windowSize -- prompt prefill is never chunked by @Adi2K in #297
Fix Gemma 4 vision pooler kernel derivation by @joshuaejackson in #290
Add LLM generation benchmark helper by @john-rocky in #248
Fix Gemma4 attention crash when vProj is nil (attentionKeqV) by @jfranknichols in #247
Adopt GemmaFunctionParser to accomodate Gemma4 tool calls. by @viktike in #183
Optimize Gemma 4 text decode: compile-fused RMSNorm-add and gelu-mul (+23.8%) by @john-rocky in #249
fix Idefics3 vision encoder dtype -- SigLIP runs in float32 instead of bf16 by @Adi2K in #296
Share the fused gated-delta kernel between MLXLLM and MLXVLM by @john-rocky in #257
Fix Qwen2-VL chat template: emit vision tokens before text by @NivDvir in #242
Fix consistent float32 dtype for gated delta SSM operations in Qwen3.5 models by @somegeekintn in #317
Fix vision-encoder attention mask being ignored in Qwen2.5-VL by @NivDvir in #238
LoRA: runtime toggle and PEFT adapter loader by @smdesai in #316
Fix Qwen35 VLM sanitize for bare 'model.*' weight keys (#143) by @john-rocky in #254
Fix SmolVLM2 tiles() upscaling small images, ~9x slowdown (#208) by @john-rocky in #255
Add structured ChatSession continuation by @aleroot in #313
Fix sporadic JSON tool-call detection and add parser hardening for mixed text/tool outputs by @aleroot in #205
Fix Gemma4Text quantized KV cache attention by @aleroot in #237
Fix MROPE, rope_deltas, and invFreq loading in Qwen2.5-VL by @NivDvir in #239
Moe refactoring -> reduces repeated small graph construction overhead by @aleroot in #324
Make Gemma 4 per_layer_model_projection quantizable by @davidkoski in #309
support for audio resources by @davidkoski in #298
Qwen2/2.5-VL: default to the model card's recommended image budget by @NivDvir in #243
Fix Gemma 4 cross-layer KV sharing in no-cache forward pass by @pavan-kkn in #333
Add Gemma 4 MTP speculative decoding by @angelsbrood in #308
fix Gemma4 prepare() to honor windowSize -- prompt prefill is never chunked by @joshuaejackson in #337
Align Falcon H1 with upstream MLX inference by @aleroot in #331
Fix LFM2-MoE sigmoid routing and expert-bias selection by @ykhrustalev in #332
remove references per author request by @davidkoski in #341
fix speculative decode tests by @davidkoski in #326
Improve Qwen3.5 recurrent cache handling by @aleroot in #323
Raise swift-syntax floor to 602.0.0 for prebuilt artifact resolution by @roryford in #340
Expose ModelTypeRegistry.contains(_:) for pre-load support checks by @jfranknichols in #349
Add Nemotron Labs Diffusion by @smdesai in #310
fix MLXVLM prepare(): honor windowSize for chunked prefill (FastVLM / Gemma3 / Qwen2VL / LFM2VL / Pixtral / Mistral3) by @john-rocky in #344
Fix speculative decoding test oracle by @aleroot in #354
Fix nomic-embed-text-v1.5 'Key embeddings.position_embeddings.weight not found' by @JamalK in #305
Add Gemma 4 12B unified by @adrgrondin in #327
Fix iOS build: replace homeDirectoryForCurrentUser with shared HF cache helper by @hehua2008 in #356
Add speculative decoding telemetry and memory gating by @aleroot in #314
Add Swift model conversion API by @aleroot in #318
Fix build on Xcode 16.3 / SDK < 26 by @pcuenca in #359
add kvScheme parameter for extensible KV cache compression by @TheTom in #230
Fix Gemma 4 QAT load: KV-shared layers carry no k_proj/v_proj/k_norm by @Flo5k5 in #342
Lfm2.5 bidirectional embedders by @ronaldmannak in #365
Fix Gemma4 QAT (E-series) load: KV-shared layers have no k_proj/v_proj/k_norm by @Flo5k5 in #330
Validate RoPE model configurations by @RNT56 in #371
Add runtime stop string handling by @RNT56 in #372
Falcon-H1R: correctness, cache, and preset fixes ...

Contributors

dirvine, GoodOlClint, and 37 other contributors

Assets 2

3.31.3

davidkoski released this 15 Apr 14:33

3.31.3

1c05248

First 3.x release.

See for information on what changed and how to upgrade

What's Changed

Decouple from tokenizer and downloader packages by @DePasqualeOrg in #118
Batched LLM inference part 1 - consolidating RoPE calls by @ronaldmannak in #178
Add speculative decoding by @petrukha-ivan in #173
Fix doc comments and verify in CI by @DePasqualeOrg in #176
Add more documentation for integrations to readme by @DePasqualeOrg in #201
Fix tool calling for Llama 3 by @aleroot in #145
Add IntegrationTesting Xcode project and additional integration test models by @atdrendel in #142
Fix links in readme by @DePasqualeOrg in #204
Fix Swift 6 Sendable error in Llama3ToolCallParser by @Lakr233 in #203
Add gemma 4 model (text, vision, MoE) by @adrgrondin in #180
Add Gemma 4 text model support (E2B and E4B) by @stefan-geens in #185
small v3 api fixes by @davidkoski in #190
v3 api embedder fixes by @davidkoski in #202
Fix prompt-cache round-trip support for ArraysCache, MambaCache, and CacheList by @ronaldmannak in #155
Prepare Gemma4Text for batched RoPE offsets by @ronaldmannak in #212
Fix Gemma 4 system message and modality order by @adrgrondin in #211
Add qwen3_next to tool call format inference by @alankessler in #166
add upgrade docs, how to use, how to develop. by @davidkoski in #206

New Contributors

@Lakr233 made their first contribution in #203
@stefan-geens made their first contribution in #185

Full Changelog: 2.31.3...3.31.3

Contributors

atdrendel, ronaldmannak, and 8 other contributors

Assets 2

2.31.3

davidkoski released this 01 Apr 20:50

2.31.3

25b00d4

In addition to the many changes and improvements in mlx-swift-lm, this also:

uses mlx-swift 0.31.3
switches Package.swift to use the 6.1 swift-tools-version -- this will help keep the code concurrency safe

Important

This will be the last tag for the 2.x releases. We will continue with some breaking API changes on main with 3.x

What's Changed

Enforce structured concurrency for MLXEmbedders by @CodebyCR in #111
Add wiredMemoryTicket to GenerateTokens by @ronaldmannak in #117
Adding Support for Qwen3.5 & Qwen3.5 MoE (Text-only) by @johnmai-dev in #97
Allow reading LFM2 models nested rope params by @adrgrondin in #122
Fix KVCache serialization by @petrukha-ivan in #121
Adding Support for Qwen3.5 & Qwen3.5 MoE (Vision) by @johnmai-dev in #120
Add JSON5 support by @ronaldmannak in #125
Pick up swift-transformers 1.1.9 by @davidkoski in #126
ensure models have links back to where they were ported from by @davidkoski in #105
add optional toolCall dispatch and tool output injection by @davidkoski in #114
Pass additionalContext to Qwen3VL by @adrgrondin in #127
Qwen3.5 performance optimization by @johnmai-dev in #129
Fix XMLFunctionParser regex to match newlines by @pixelsoccupied in #131
audit RoPE use across models by @davidkoski in #115
fix Sendable issues, unused code, deprecation warnings by @davidkoski in #113
Add qwen3_5_text model type support by @adhney in #135
Fix LFM2.5 VL tools by @viktike in #139
Fix tool calling for Mistral 3 by @atdrendel in #132
Fixed tool calling for qwen3.5 by @tpae in #133
Adding support for GLM-OCR model by @smdesai in #144
Add topK, minP and penalty parameters to GenerateParameters by @adrgrondin in #141
Add gemma 3 embedding model by @CodebyCR in #136
fix: @ModuleInfo for pooler + attention mask dtype in Bert/NomicBert by @jowharshamshiri in #153
Fix LFM2 tool calling with nested parentheses in arguments by @tpae in #152
fix unreliable tests by @davidkoski in #128
add missing context/toolcall parameters by @davidkoski in #140
perf: eliminate CPU←GPU sync in penalty processors, optimize TopPSampler by @spokvulcan in #147
Add copy() to KVCache protocol and all implementations by @alankessler in #158
Add KV cache initializers and cache access to ChatSession by @alankessler in #151
Add model-defined pooling fallback for embedding models by @sxy-trans-n in #156
Handle multiple tool calls in ChatSession by @alankessler in #162
switch to swift 6 -- prevent concurrency issues, fix concurrency issues by @davidkoski in #165

New Contributors

@johnmai-dev made their first contribution in #97
@pixelsoccupied made their first contribution in #131
@adhney made their first contribution in #135
@viktike made their first contribution in #139
@atdrendel made their first contribution in #132
@jowharshamshiri made their first contribution in #153
@spokvulcan made their first contribution in #147
@alankessler made their first contribution in #158
@sxy-trans-n made their first contribution in #156

Full Changelog: 2.30.6...2.31.3

Contributors

tpae, atdrendel, and 14 other contributors

Assets 2

2.30.6

davidkoski released this 18 Feb 18:29

2.30.6

7e19e09

Switch to mlx-swift 0.30.6: https://github.com/ml-explore/mlx-swift/releases/tag/0.30.6

Important:

ml-explore/mlx-swift-examples#462 -- incorrect detection of NAX hardware on iPhone 16 Pro
- via ml-explore/mlx#3083

What's Changed

Fix AfMoE and DeepSeek V3 by @DePasqualeOrg in #30
Add GLM 4.7 Flash models by @ronaldmannak in #68
Added minicpm support by @robertmsale in #71
Support for NemotronH by @fmarmori in #75
Added Qwen3-Next-80b by @robertmsale in #70
Feature/vlm external video frames support [Draft] by @vade in #64
Add configurable tool call parsing with support for multiple model formats by @tpae in #78
Add skill.md by @ronaldmannak in #92
LFM2.5 outputs [func(arg='value')] Pythonic syntax, not JSON by @tpae in #91
Update code snippet in README. by @zaneenders in #96
trim vocabulary padding in weight sanitization in Gemma3Text by @aleroot in #99
Rename MLXEmbedders sub library to match target name by @CodebyCR in #102
Added wired memory control by @robertmsale in #72
Fix MLXEmbedders README.md example by @CodebyCR in #103
Add documentation for specified embedding model by @CodebyCR in #104
Add raw token streaming (TokenGeneration) and generateTokens convenience APIs by @ronaldmannak in #88
Fix latest mistral 3.2 model loading by @aleroot in #108
Add tools parameter to ChatSession for function calling support by @louis-jan in #107
Add MiniMax and MiMo v2 Flash models by @ronaldmannak in #50
Update glm4_moe_lite To Store KV Latent in Cache by @ronaldmannak in #73
switch to current mlx-swift by @davidkoski in #100