Skip to content

3.31.4

Latest
Latest

Choose a tag to compare

View all tags
@davidkoski davidkoski released this 30 Jun 16:31
3.31.4
bd4b743
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

New release with the last couple months of new features and fixes.

Upcoming changes (potential instability on main):

Use this tag if you want to isolate.

What's Changed

  • Guard embedder inputs against maxPositionEmbeddings by @alankessler in #163
  • Doc corrections for v3 by @DePasqualeOrg in #216
  • add fallback links to the readme -- docs are not yet built by @davidkoski in #218
  • Fix Qwen35 VLM crash on text-only inference by @dirvine in #149
  • Add coherence integration tests by @DePasqualeOrg in #235
  • fix: flatten prompt in TokenRing.loadPrompt to handle 2D inputs by @spokvulcan in #170
  • Pass tools schema to ToolCallProcessor for type-aware parsing by @alankessler in #167
  • Forward tools and additionalContext in GlmOcr and SmolVLM2 processors by @alankessler in #174
  • fix: avoid shared CIContext in media processing by @Meapri in #268
  • feat: add ParoQuant (pairwise rotation quantization) support by @spokvulcan in #164
  • fix ropeOffset use in Gemma3NText by @davidkoski in #280
  • Handle stringified JSON tool call arguments by @kuosuko in #172
  • fix build by @davidkoski in #281
  • fix segsum dtype promotion -- 2x memory waste on hybrid SSM models by @TheTom in #229
  • Remove TokenLoopHandler Sendable requirement by @thisray in #284
  • feat: expose speculative decoding in ChatSession (#181) by @VDurocher in #193
  • Fix EmbeddingGemma init-order crash + dense head hidden size by @0xweb3r in #223
  • fix: disable CIContext intermediate caching to prevent IOSurface exha… by @Satchitananda in #226
  • fix Gemma 4 MoE router -- softmax order + fuse norm dispatches by @TheTom in #228
  • add FusedGateUpSwitchGLU -- single fused gate_up_proj for MoE models by @TheTom in #227
  • Fix UserInput init not setting self.images / self.videos (#182) by @john-rocky in #253
  • pipeline prefill chunks with asyncEval -- 10x on GDN models by @TheTom in #225
  • fix gated delta state precision -- fp32 state to match Python by @TheTom in #224
  • Fix(Docs) non-compiling README quick-start example by @programVeins in #265
  • combine docs check with build to reuse a macos runner by @davidkoski in #286
  • Remove SDK build version command from workflow by @madrob in #306
  • models should not mutate state during eval by @davidkoski in #283
  • Fix: tolerate 1D token inputs in Gemma 4 VLM forward pass by @brdcastro in #241
  • fix Idefics3 prepare() to honor windowSize -- prompt prefill is never chunked by @Adi2K in #297
  • Fix Gemma 4 vision pooler kernel derivation by @joshuaejackson in #290
  • Add LLM generation benchmark helper by @john-rocky in #248
  • Fix Gemma4 attention crash when vProj is nil (attentionKeqV) by @jfranknichols in #247
  • Adopt GemmaFunctionParser to accomodate Gemma4 tool calls. by @viktike in #183
  • Optimize Gemma 4 text decode: compile-fused RMSNorm-add and gelu-mul (+23.8%) by @john-rocky in #249
  • fix Idefics3 vision encoder dtype -- SigLIP runs in float32 instead of bf16 by @Adi2K in #296
  • Share the fused gated-delta kernel between MLXLLM and MLXVLM by @john-rocky in #257
  • Fix Qwen2-VL chat template: emit vision tokens before text by @NivDvir in #242
  • Fix consistent float32 dtype for gated delta SSM operations in Qwen3.5 models by @somegeekintn in #317
  • Fix vision-encoder attention mask being ignored in Qwen2.5-VL by @NivDvir in #238
  • LoRA: runtime toggle and PEFT adapter loader by @smdesai in #316
  • Fix Qwen35 VLM sanitize for bare 'model.*' weight keys (#143) by @john-rocky in #254
  • Fix SmolVLM2 tiles() upscaling small images, ~9x slowdown (#208) by @john-rocky in #255
  • Add structured ChatSession continuation by @aleroot in #313
  • Fix sporadic JSON tool-call detection and add parser hardening for mixed text/tool outputs by @aleroot in #205
  • Fix Gemma4Text quantized KV cache attention by @aleroot in #237
  • Fix MROPE, rope_deltas, and invFreq loading in Qwen2.5-VL by @NivDvir in #239
  • Moe refactoring -> reduces repeated small graph construction overhead by @aleroot in #324
  • Make Gemma 4 per_layer_model_projection quantizable by @davidkoski in #309
  • support for audio resources by @davidkoski in #298
  • Qwen2/2.5-VL: default to the model card's recommended image budget by @NivDvir in #243
  • Fix Gemma 4 cross-layer KV sharing in no-cache forward pass by @pavan-kkn in #333
  • Add Gemma 4 MTP speculative decoding by @angelsbrood in #308
  • fix Gemma4 prepare() to honor windowSize -- prompt prefill is never chunked by @joshuaejackson in #337
  • Align Falcon H1 with upstream MLX inference by @aleroot in #331
  • Fix LFM2-MoE sigmoid routing and expert-bias selection by @ykhrustalev in #332
  • remove references per author request by @davidkoski in #341
  • fix speculative decode tests by @davidkoski in #326
  • Improve Qwen3.5 recurrent cache handling by @aleroot in #323
  • Raise swift-syntax floor to 602.0.0 for prebuilt artifact resolution by @roryford in #340
  • Expose ModelTypeRegistry.contains(_:) for pre-load support checks by @jfranknichols in #349
  • Add Nemotron Labs Diffusion by @smdesai in #310
  • fix MLXVLM prepare(): honor windowSize for chunked prefill (FastVLM / Gemma3 / Qwen2VL / LFM2VL / Pixtral / Mistral3) by @john-rocky in #344
  • Fix speculative decoding test oracle by @aleroot in #354
  • Fix nomic-embed-text-v1.5 'Key embeddings.position_embeddings.weight not found' by @JamalK in #305
  • Add Gemma 4 12B unified by @adrgrondin in #327
  • Fix iOS build: replace homeDirectoryForCurrentUser with shared HF cache helper by @hehua2008 in #356
  • Add speculative decoding telemetry and memory gating by @aleroot in #314
  • Add Swift model conversion API by @aleroot in #318
  • Fix build on Xcode 16.3 / SDK < 26 by @pcuenca in #359
  • add kvScheme parameter for extensible KV cache compression by @TheTom in #230
  • Fix Gemma 4 QAT load: KV-shared layers carry no k_proj/v_proj/k_norm by @Flo5k5 in #342
  • Lfm2.5 bidirectional embedders by @ronaldmannak in #365
  • Fix Gemma4 QAT (E-series) load: KV-shared layers have no k_proj/v_proj/k_norm by @Flo5k5 in #330
  • Validate RoPE model configurations by @RNT56 in #371
  • Add runtime stop string handling by @RNT56 in #372
  • Falcon-H1R: correctness, cache, and preset fixes by @aleroot in #368
  • Fix Gemma4 VLM end token defaults by @RNT56 in #373
  • Add Mamba2 (pure SSM) model by @GoodOlClint in #380
  • Add Mixtral (sparse MoE) model by @GoodOlClint in #378
  • Gemma 4: skip vision_embedder weights in Gemma4Model.sanitize by @surawatthana in #366
  • Gemma4: drop gemma4_unified vision_embedder weights in sanitize by @neuromechanist in #363
  • Support correlated tool call transcripts by @aleroot in #360
  • Qwen2-VL: implement M-RoPE in the language model (mirror of #239) by @mnmly in #345
  • Add optional seed to GenerateParameters for reproducible sampling by @GoodOlClint in #377

New Contributors

Full Changelog: 3.31.3...3.31.4

Contributors

dirvine, GoodOlClint, and 37 other contributors
Assets 2
Loading