DeepSeek-V4 + NVFP4 Exporting

[danielhanchen]

Unsloth Studio can now export to NVFP4, FP8, imatrix GGUFs after training, can act as a llama-swap API system, includes Japanese and Brazilian language support, includes MLX, safetensors tool calling + healing support and much more! Unsloth core makes GRPO 1.3x faster, have HTTP fallback for stalled downloads, make MoE training 3 to 5x faster, and fixes many bugs! This release series uses unsloth>=2026.7.1.

Smarter OpenAI-Compatible API Serving

Run one local API endpoint across with safer automatic model swapping and better agent-tool recovery.

• API requests can opt into model auto switching, so a request for a different downloaded local GGUF can load and serve that model automatically.
• The model swap path is safe by default: unknown model names keep using the current model and do not trigger surprise downloads.
• /v1/models now returns clean model IDs and the local GGUF catalog, so API clients do not see local .gguf file paths.
• Idle auto-unload can free VRAM after inactivity while still reloading the last model on the next API request.
• API clients can control tool-call healing per request and opt into the extra nudge retry when a model tries to call a tool but returns malformed markup.

Export Improvements

Exports are more flexible and avoid more unnecessary downloads.

• Unsloth now lets you select multiple export formats at once.
• Unsloth now supports portable FP8/INT8 exports, GGUF LoRA, and source-matched exports.
• save_pretrained_merged now supports compressed FP8/FP4 exports.
• The Unsloth export UI now includes imatrix GGUF and compressed export options.
• Exporting multiple checkpoints now avoids more repeated base-model downloads.
• FP8, INT8, and GGUF-LoRA exports now respect trust_remote_code.
• GGUF export now handles models with missing quantization settings more reliably.

RAG and File Chat

Chat with files should be more useful on real documents.

• RAG attachments can now use whole-document context.
• File chat now reads more PDFs and Word documents correctly, including right-to-left text, Indic text, and DOCX tables.
• Local RAG checks are less likely to fail because of proxy settings.
• The embedding model is now customizable, with Hugging Face search and a reorganized settings tab.

Unsloth Polish and Reliability

Unsloth is smoother and more reliable during everyday use.

• Training and chat progress are less likely to freeze silently during long runs.
• Compare mode starts more reliably.
• Switching or cancelling models during loading is less likely to get stuck.
• Hub browsing now recovers better from invalid Hugging Face tokens and shows clearer local file paths.
• Hub Discover now shows all models by default, with a cleaner model picker and a fits-on-device filter.
• Project export menus are clearer, with combined project exports and individual chat exports grouped separately.
• Unsloth training runs now show project names.
• Guided tours, reasoning timers, thinking stops, settings, file dialogs, and update screens now feel cleaner and more consistent.
• Japanese and Portuguese (Brazil) are now available in the Unsloth UI.
• Installer and package checks were tightened to catch more unsafe or unexpected dependency changes.

Installer, Hardware and Platform Fixes

Unsloth installs and runs more reliably across different machines, GPUs, and network setups.

• Installer pins were updated to unsloth>=2026.6.9.
• macOS installs no longer require CMake or Homebrew when a prebuilt llama.cpp is available.
• Apple Silicon installs now handle paths with spaces more reliably and size GGUF context for unified memory.
• macOS installs work better behind corporate TLS-inspection proxies.
• Linux and macOS path handling is more reliable when folders contain spaces.
• Windows ROCm RAG embedding avoids a torchao import crash.
• Windows Unsloth startup handles UTF-8 text more reliably.
• ROCm-on-WSL now supports discrete Radeon RDNA 3/4 GPUs, not just Strix Halo.
• Blackwell sm_100 and sm_103 data-center GPUs now use the correct llama.cpp prebuilt selection.
• GGUF fit checks now reserve memory for CUDA context and vision/model overhead more accurately.
• Tensor parallelism fixed for vision and mmproj GGUF models.
• Existing local llama.cpp builds can now be reused with --with-llama-cpp-dir.

Training, Models and Kernels

Training and model loading are more reliable across more model types, GPUs, and fine-tuning setups.

• GRPO training now supports sequence packing by default for old/ref log-probability calculations.
• GRPO now avoids repeating the same shared prompt across grouped completions.
• GRPO logit scaling now works correctly with DDP-wrapped models.
• Full fine-tuning now uses the correct precision on V100 and other GPUs without bf16 support.
• RL trainers no longer reject valid full fine-tuning precision settings.
• Unsloth gradient checkpointing is no longer silently disabled by TrainingArguments.
• DDP training no longer crashes from a CPU-resident RoPE buffer.
• MoE LoRA detection now targets the correct expert MLP layers.
• Grouped MoE can now turn on automatically for loaded and PEFT models.
• FP8 quantization now handles odd tensor shapes and scale formats more reliably.
• FP8 dequant kernels now use safer offset handling.
• Llama 3 RoPE scaling is preserved with Transformers v5.
• LoRA reloads continue to work with PEFT 0.19.
• fast_generate now shows a clearer error when vLLM-style inputs are used without fast inference.

Updating Unsloth

To update Unsloth or install a fresh Unsloth Unsloth, use the commands below:
macOS, Linux, WSL:

curl -fsSL https://unsloth.ai/install.sh | sh

Windows:

irm https://unsloth.ai/install.ps1 | iex

What's Changed in Unsloth

• Bump install.sh / install.ps1 pin to unsloth>=2026.6.9 by @danielhanchen in Bump install.sh / install.ps1 pin to unsloth>=2026.6.9 #6580
• Studio: default Hub Discover scope to all models by @shimmyshimmer in Studio: default Hub Discover scope to all models #6593
• Studio: model picker search placeholder, Search Hub tooltip, list polish by @shimmyshimmer in Studio: model picker search placeholder, Search Hub tooltip, list polish #6592
• Studio: refresh chat guided tour for the redesigned model picker by @shimmyshimmer in Studio: refresh chat guided tour for the redesigned model picker #6597
• CI: fix import-hoist false positive, vision-cache test cwd, and llama.cpp CLI smoke by @danielhanchen in CI: fix import-hoist false positive, vision-cache test cwd, and llama.cpp CLI smoke #6598
• fix(studio): handle multimodal list content in inference text paths ([Bug] Unsloth Studio "Error: 'list object' has no attribute 'replace'" #4383) by @danielhanchen in fix(studio): handle multimodal list content in inference text paths (#4383) #6480
• studio/setup.sh: guard empty CUDA arch detection in the source build (studio/setup.sh: validate source-built llama.cpp and guard empty CUDA arch detection #5854) by @danielhanchen in studio/setup.sh: guard empty CUDA arch detection in the source build (#5854) #6481
• fix: use EMPTY_LOGITS on the fused-CE not-return_dict path (UnboundLocalError: cannot access local variable 'logits' where it is not associated with a value #2068) by @danielhanchen in fix: use EMPTY_LOGITS on the fused-CE not-return_dict path (#2068) #6482
• Fix GRPOTrainer evaluate() crash without prior training by @danielhanchen in Fix GRPOTrainer evaluate() crash without prior training #6523
• Fix misleading 'only for image models' error for Qwen3-VL when torchvision is missing by @danielhanchen in Fix misleading 'only for image models' error for Qwen3-VL when torchvision is missing #6525
• Fix FlashAttention fp32 crash with DoRA (use_dora=True) by @danielhanchen in Fix FlashAttention fp32 crash with DoRA (use_dora=True) #6526
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in [pre-commit.ci] pre-commit autoupdate #6587
• Withhold HF_TOKEN from pull_request CI runs by @danielhanchen in Withhold HF_TOKEN from pull_request CI runs #6600
• Fix Studio export checkpoint ordering by @Lyxot in Fix Studio export checkpoint ordering #6602
• Studio: treat data-center Blackwell (sm_100/sm_103) as Blackwell in llama.cpp prebuilt selection by @danielhanchen in Studio: treat data-center Blackwell (sm_100/sm_103) as Blackwell in llama.cpp prebuilt selection #6584
• Studio: fall back to anonymous HF browsing on a malformed token by @danielhanchen in Studio: fall back to anonymous HF browsing on a malformed token #6605
• Chat: match reasoning thinking icon to the composer bulb by @shimmyshimmer in Chat: match reasoning thinking icon to the composer bulb #6607
• Hub: restore Unsloth owner avatar to HF profile picture by @shimmyshimmer in Hub: restore Unsloth owner avatar to HF profile picture #6606
• Studio: scope "Remember settings next time" per GGUF quant and apply on every load path by @oobabooga in Studio: scope "Remember settings next time" per GGUF quant and apply on every load path #6594
• Fix sentence-transformers RAG embedder on Windows ROCm (torchao import crash) by @danielhanchen in Fix sentence-transformers RAG embedder on Windows ROCm (torchao import crash) #6608
• Remove git blame ignore revs by @wasimysaid in Remove git blame ignore revs #6582
• Studio: allow --secure with --api-only (headless secure API server) and add --api-only to unsloth studio run by @danielhanchen in Studio: allow --secure with --api-only (headless secure API server) and add --api-only to unsloth studio run #6591
• studio: tidy Windows-ROCm torchao comments (follow-up to Fix sentence-transformers RAG embedder on Windows ROCm (torchao import crash) #6608) by @danielhanchen in studio: tidy Windows-ROCm torchao comments (follow-up to #6608) #6610
• Studio: correct anyio<4.14 comments to the real [Bug] Streaming chat completion crashes with RuntimeError (Python 3.13 / anyio cancel scope) #6483 cause (4.14/Py3.13 streaming cancel-scope bug) by @danielhanchen in Studio: correct anyio<4.14 comments to the real #6483 cause (4.14/Py3.13 streaming cancel-scope bug) #6581
• studio: report the true reasoning duration and fix Stop for thinking models by @danielhanchen in studio: report the true reasoning duration and fix Stop for thinking models #6521
• Add HTTPS hint to Studio launch message by @Imagineer99 in Add HTTPS hint to Studio launch message #6583
• Fix Gemma 4 GGUF OpenAI API streams by @wasimysaid in Fix Gemma 4 GGUF OpenAI API streams #6476
• Studio: lazy-import matplotlib so the server starts when the wheel is blocked by @LeoBorcherding in Studio: lazy-import matplotlib so the server starts when the wheel is blocked #6596
• Studio: add sidebar update button with installed-version display by @dylanschroers in Studio: add sidebar update button with installed-version display #6545
• Thread finetune_audio_layers through get_peft_model (Gemma 4 / Gemma 3N audio LoRA) by @danielhanchen in Thread finetune_audio_layers through get_peft_model (Gemma 4 / Gemma 3N audio LoRA) #6558
• Fix CPOTrainer crash with multimodal processors (Gemma 4) by @danielhanchen in Fix CPOTrainer crash with multimodal processors (Gemma 4) #6522
• Fix Qwen3 NaN loss: delegate pad_token repair to shared unsloth_zoo.pad_token by @danielhanchen in Fix Qwen3 NaN loss: delegate pad_token repair to shared unsloth_zoo.pad_token #6524
• Harden MLX self-heal install against supply-chain code execution by @danielhanchen in Harden MLX self-heal install against supply-chain code execution #6599
• Update safe Studio Tauri cargo dependencies by @wasimysaid in Update safe Studio Tauri cargo dependencies #6612
• Studio: remove OpenEnv and other unused packages by @oobabooga in Studio: remove OpenEnv and other unused packages #6585
• Bump vite from 8.0.10 to 8.0.16 in /studio/frontend in the npm-frontend-security group across 1 directory by @dependabot[bot] in Bump vite from 8.0.10 to 8.0.16 in /studio/frontend in the npm-frontend-security group across 1 directory #6354
• Fix SyntheticDataKit.chunk_data dropping single-chunk documents by @vineethsaivs in Fix SyntheticDataKit.chunk_data dropping single-chunk documents #6595
• Match IGNORED_TOKENIZER_NAMES case-insensitively by @vineethsaivs in Match IGNORED_TOKENIZER_NAMES case-insensitively #6620
• Installer: don't require cmake/Homebrew on macOS (prebuilt llama.cpp) by @oobabooga in Installer: don't require cmake/Homebrew on macOS (prebuilt llama.cpp) #6617
• Fix construct_chat_template leaking {INPUT}/{OUTPUT} sentinel into the chat template by @vineethsaivs in Fix construct_chat_template leaking {INPUT}/{OUTPUT} sentinel into the chat template #6531
• Studio: slim the GLM-5.2 thinking menu width by @shimmyshimmer in Studio: slim the GLM-5.2 thinking menu width #6631
• Studio: confirm saved Hugging Face token with a tick by @shimmyshimmer in Studio: confirm saved Hugging Face token with a tick #6630
• Fix _SameTaskStreamingResponse disconnect test bypassing init by @danielhanchen in Fix _SameTaskStreamingResponse disconnect test bypassing __init__ #6627
• Clarify Studio --secure hint exposes a public Cloudflare tunnel by @danielhanchen in Clarify Studio --secure hint exposes a public Cloudflare tunnel #6615
• Tidy verbose Studio launch messages by @danielhanchen in Tidy verbose Studio launch messages #6628
• Make _uv_safe_path space-safe on macOS/Linux (install.sh --local / Studio setup fails with error: File not found: <truncated path> when the repo is cloned into a directory path containing a space #6503) by @GodlyDonuts in Make _uv_safe_path space-safe on macOS/Linux (#6503) #6534
• Clarify in README that studio --secure creates a public tunnel by @danielhanchen in Clarify in README that studio --secure creates a public tunnel #6632
• Studio: add Unsloth Docs to the MCP server presets by @shimmyshimmer in Studio: add Unsloth Docs to the MCP server presets #6633
• Studio: cap GGUF context to unified memory on Apple Silicon by @oobabooga in Studio: cap GGUF context to unified memory on Apple Silicon #6622
• Studio: group the project export menu by Combined and Per chat by @shimmyshimmer in Studio: group the project export menu by Combined and Per chat #6637
• Studio: tighten the sidebar footer spacing and update-card fade by @shimmyshimmer in Studio: tighten the sidebar footer spacing and update-card fade #6641
• Studio: keep the sidebar bottom fade in sync when groups collapse by @shimmyshimmer in Studio: keep the sidebar bottom fade in sync when groups collapse #6640
• Pin isolated Node.js installer to committed sha256 digests by @danielhanchen in Pin isolated Node.js installer to committed sha256 digests #6625
• Studio: shareable per-checkpoint preview links by @NilayYadav in Studio: shareable per-checkpoint preview links #6486
• Verify DiffusionGemma visual-server binary against approved checksums by @danielhanchen in Verify DiffusionGemma visual-server binary against approved checksums #6635
• Studio: honor stream=false on the GGUF agentic tool path ([Bug] OpenAI-compatible /v1/chat/completions ignores "stream": false and returns an SSE stream #6570) by @oobabooga in Studio: honor stream=false on the GGUF agentic tool path (#6570) #6618
• Fix Studio GGUF variant expansion crash by @Lyxot in Fix Studio GGUF variant expansion crash #6636
• Installer: make UV_OVERRIDE space-safe on Apple Silicon (install.sh --local / Studio setup fails with error: File not found: <truncated path> when the repo is cloned into a directory path containing a space #6503) by @danielhanchen in Installer: make UV_OVERRIDE space-safe on Apple Silicon (#6503) #6639
• Polish Studio desktop chrome by @wasimysaid in Polish Studio desktop chrome #6332
• Fix Linux AppImage packaging by @wasimysaid in Fix Linux AppImage packaging #6657
• Studio: clean up empty leftover quant folders so they can be deleted by @danielhanchen in Studio: clean up empty leftover quant folders so they can be deleted #6616
• Studio: fix misleading max_seq_length advice for train-on-completions row drops by @danielhanchen in Studio: fix misleading max_seq_length advice for train-on-completions row drops #6664
• Studio: UNSLOTH_NPM_REGISTRY opt-in for corporate npm mirrors (Advanced install should be compatible with corporate firewalls #6491) by @danielhanchen in Studio: UNSLOTH_NPM_REGISTRY opt-in for corporate npm mirrors (#6491) #6663
• Studio: keep the live progress stream alive during pre-first-step preparation by @danielhanchen in Studio: keep the live progress stream alive during pre-first-step preparation #6665
• Keep pad-named pad_tokens (e.g. <|vision_pad|>); fix Qwen3-Base load crash by @danielhanchen in Keep pad-named pad_tokens (e.g. <|vision_pad|>); fix Qwen3-Base load crash #6652
• Studio: keep the training event pump alive so progress can't silently freeze by @danielhanchen in Studio: keep the training event pump alive so progress can't silently freeze #6643
• Studio: stop leaking the auth token through HTML canvas preview frames by @danielhanchen in Studio: stop leaking the auth token through HTML canvas preview frames #6634
• feat: add GPU-aware model filtering and For You section- Add fit filt… by @zrva in feat: add GPU-aware model filtering and For You section- Add fit filt… #6645
• Studio: prompt variables into prompt editor by @Imagineer99 in Studio: prompt variables into prompt editor #6434
• Verify linuxdeploy AppImage digest before use in desktop release by @danielhanchen in Verify linuxdeploy AppImage digest before use in desktop release #6673
• Studio: require signed capability tokens for /p preview links by @danielhanchen in Studio: require signed capability tokens for /p preview links #6666
• Fix offline checkpoint load/export: "tokenizer is weirdly not loaded" by @danielhanchen in Fix offline checkpoint load/export: "tokenizer is weirdly not loaded" #6554
• Studio Colab: opt-in shareable Cloudflare tunnel link by @LeoBorcherding in Studio Colab: opt-in shareable Cloudflare tunnel link #6684
• Studio: auto-shut-down an exposed first-run instance if the admin password is never changed by @danielhanchen in Studio: auto-shut-down an exposed first-run instance if the admin password is never changed #6651
• Fix DDP crash from CPU-resident rotary inv_freq buffer by @Abdul-Moiz31 in Fix DDP crash from CPU-resident rotary inv_freq buffer #6662
• Patch FalconH1RMSNorm to fix float64 compilation crash on Intel Arc DG2 by @anmolxlight in Patch FalconH1RMSNorm to fix float64 compilation crash on Intel Arc DG2 #6691
• feat: improve Unsloth Studio chat title generation quality by @mvanhorn in feat: improve Unsloth Studio chat title generation quality #6697
• Studio: harden background consumer loops and streaming paths against silent UI freezes by @danielhanchen in Studio: harden background consumer loops and streaming paths against silent UI freezes #6653
• fix: wrap unprotected evaluate() calls with robust_evaluate() to handle navigation context loss by @jimdawdy-hub in fix: wrap unprotected evaluate() calls with robust_evaluate() to handle navigation context loss #6677
• Add GGUF flash attention and TP CLI options by @OnePunchMonk in Add GGUF flash attention and TP CLI options #6561
• studio: return a clean model id from the OpenAI API instead of the local .gguf path by @danielhanchen in studio: return a clean model id from the OpenAI API instead of the local .gguf path #6518
• fix(install): enable UV_NATIVE_TLS on macOS for corporate TLS-inspection proxies by @Cesarsk in fix(install): enable UV_NATIVE_TLS on macOS for corporate TLS-inspection proxies #6671
• Stop marking non-8bit Unsloth models as 8-bit by @Lyxot in Stop marking non-8bit Unsloth models as 8-bit #6708
• studio: list the full local model catalog from /v1/models by @danielhanchen in studio: list the full local model catalog from /v1/models #6519
• Silence torchao _C*.so load-failure WARNING on torch >= 2.11 by @danielhanchen in Silence torchao _C*.so load-failure WARNING on torch >= 2.11 #6712
• Harden flaky Studio CI: retry VS-hide rename and tolerate same-URL nav interrupt by @danielhanchen in Harden flaky Studio CI: retry VS-hide rename and tolerate same-URL nav interrupt #6713
• Fix fast_inference crash on ABI-broken vLLM: probe compiled extensions, not just import vllm by @oobabooga in Fix fast_inference crash on ABI-broken vLLM: probe compiled extensions, not just import vllm #6621
• Studio: restore tensor parallelism for vision/mmproj GGUFs by @danielhanchen in Studio: restore tensor parallelism for vision/mmproj GGUFs #6659
• Studio: stop handing CI/user secrets to downloaded llama.cpp binaries by @danielhanchen in Studio: stop handing CI/user secrets to downloaded llama.cpp binaries #6696
• Revert "feat: add GPU-aware model filtering and For You section- Add fit filt…" by @danielhanchen in Revert "feat: add GPU-aware model filtering and For You section- Add fit filt…" #6722
• Remove unused FalconH1RMSNormGated import by @danielhanchen in Remove unused FalconH1RMSNormGated import #6728
• Studio: cascade user message deletion to include assistant reply by @NilayYadav in Studio: cascade user message deletion to include assistant reply #6720
• Fix custom chat templates with a {system_message} placeholder (dead code in _change_system_message) by @vineethsaivs in Fix custom chat templates with a {system_message} placeholder (dead code in _change_system_message) #6735
• Fix gpt-oss detection in save: config.architectures is a list, not a string by @vineethsaivs in Fix gpt-oss detection in save: config.architectures is a list, not a string #6711
• fix(studio): show local file path tooltip for Hub-tab local models by @mvanhorn in fix(studio): show local file path tooltip for Hub-tab local models #6715
• Studio: fix compare adapter selection by @Imagineer99 in Studio: fix compare adapter selection #6411
• perf(dataprep): cache regex and field lists, fix typos by @Muhammad-Ikhwan-Fathulloh in perf(dataprep): cache regex and field lists, fix typos #6714
• Speed up Studio desktop startup by @wasimysaid in Speed up Studio desktop startup #6742
• Fix on-device locations dialog layout by @Imagineer99 in Fix on-device locations dialog layout #6743
• Fix settings dialog overlay stacking above chat header by @Imagineer99 in Fix settings dialog overlay stacking above chat header #6745
• (feat) Add project names to studio training runs by @shimmyshimmer in (feat) Add project names to studio training runs #6512
• Studio: name the missing extractor when a Recipes file upload fails by @shimmyshimmer in Studio: name the missing extractor when a Recipes file upload fails #6642
• fix: remove sidebar update dev override by @Imagineer99 in fix: remove sidebar update dev override #6746
• fix: upgrade hono to 4.12.25 (CVE-2026-54290) by @orbisai0security in fix: upgrade hono to 4.12.25 (CVE-2026-54290) #6736
• Fix llama.cpp CMake build detection in save.py by @ashzak in Fix llama.cpp CMake build detection in save.py #5957
• CI: add PyPI extra-index to CPU torch installs (fix sympy ResolutionImpossible) by @danielhanchen in CI: add PyPI extra-index to CPU torch installs (fix sympy ResolutionImpossible) #6660
• Fix full finetuning precision on V100 / no-bf16 GPUs by @danielhanchen in Fix full finetuning precision on V100 / no-bf16 GPUs #5880
• Fix triton-xpu 3.7.1 sha256 hashes for Intel XPU install by @Oscilloscope98 in Fix triton-xpu 3.7.1 sha256 hashes for Intel XPU install #6629
• Fix stale xformers and flash-attn wheel URLs by @danielhanchen in Fix stale xformers and flash-attn wheel URLs #4213
• Studio: keep "Fine-tuned" compare label clear of the floating top right controls by @NilayYadav in Studio: keep "Fine-tuned" compare label clear of the floating top right controls #6755
• Add FP8/FP4 compressed export to save_pretrained_merged by @danielhanchen in Add FP8/FP4 compressed export to save_pretrained_merged #6706
• Studio: imatrix GGUF option and FP8/NVFP4 compressed export in the export UI by @danielhanchen in Studio: imatrix GGUF option and FP8/NVFP4 compressed export in the export UI #6729
• Whole-document context for RAG chat attachments by @danielhanchen in Whole-document context for RAG chat attachments #6693
• Studio: quick eject from the model selector by @shimmyshimmer in Studio: quick eject from the model selector #6654
• studio: explicit Cloudflare tunnel notice and public-exposure warning at startup by @danielhanchen in studio: explicit Cloudflare tunnel notice and public-exposure warning at startup #6515
• fix(studio/llama_cpp): disable trust_env on the loopback health probe (mac os health check uses default trust_env #6750) by @Anai-Guo in fix(studio/llama_cpp): disable trust_env on the loopback health probe (#6750) #6752
• feat(i18n): add Japanese locale support for the Studio UI catalog by @DovahkiinYuzuko in feat(i18n): add Japanese locale support for the Studio UI catalog #6705
• fix: keep LoRA reloads working with PEFT 0.19 by @rodboev in fix: keep LoRA reloads working with PEFT 0.19 #6748
• studio: allow updating HF models through UI by @Anish9901 in studio: allow updating HF models through UI #5388
• Keep the JS-bundle scan check size-agnostic so the baseline does not drift by @danielhanchen in Keep the JS-bundle scan check size-agnostic so the baseline does not drift #6770
• Reduce comments across recent fixes by @danielhanchen in Reduce comments across recent fixes #6776
• MLX CI: find llama-cli where save_pretrained_gguf actually installs it by @danielhanchen in MLX CI: find llama-cli where save_pretrained_gguf actually installs it #6777
• Studio RAG: disable trust_env on loopback llama-server httpx clients by @danielhanchen in Studio RAG: disable trust_env on loopback llama-server httpx clients #6775
• scan_packages: key baseline on matched-code hash so payloads in baselined files are not auto-suppressed by @danielhanchen in scan_packages: key baseline on matched-code hash so payloads in baselined files are not auto-suppressed #6552
• Pin llm-compressor auto-install to a vetted version range by @danielhanchen in Pin llm-compressor auto-install to a vetted version range #6778
• Fix Windows Studio UTF-8 startup handling by @Imagineer99 in Fix Windows Studio UTF-8 startup handling #6614
• Studio: opt-in OpenAI /v1 model auto-switch and idle keep-warm by @danielhanchen in Studio: opt-in OpenAI /v1 model auto-switch and idle keep-warm #6392
• Fix GRPO logit scaling when model is wrapped by DDP by @ftrajkov-amd in Fix GRPO logit scaling when model is wrapped by DDP #5955
• Fix gpt-oss offload_embedding and generate() kwargs, and guard offload_embedding on tied/vLLM models by @danielhanchen in Fix gpt-oss offload_embedding and generate() kwargs, and guard offload_embedding on tied/vLLM models #6774
• Add a fits-on-device filter to the model selects by @shimmyshimmer in Add a fits-on-device filter to the model selects #6802
• Studio RAG: fix RTL/Indic PDF corruption and dropped DOCX tables by @danielhanchen in Studio RAG: fix RTL/Indic PDF corruption and dropped DOCX tables #6780
• Studio: customizable RAG embedding model with HF search, settings tab reorganization by @shimmyshimmer in Studio: customizable RAG embedding model with HF search, settings tab reorganization #6800
• Studio: self-heal a pre-[Bug] Streaming chat completion crashes with RuntimeError (Python 3.13 / anyio cancel scope) #6483-fix anyio>=4.14 stuck in existing installs by @Abdul-Moiz31 in Studio: self-heal a pre-#6483-fix anyio>=4.14 stuck in existing installs #6805
• feat: Implementation of the Portuguese (Brazil) language and VRAM/RAM monitor by @Dspofu in feat: Implementation of the Portuguese (Brazil) language and VRAM/RAM monitor #6509
• Fix: skip fp16/bf16 validation for full finetuning in RL trainers by @InfoSage05 in Fix: skip fp16/bf16 validation for full finetuning in RL trainers #6813
• studio/frontend: drop developer-only /grid-test route by @danielhanchen in studio/frontend: drop developer-only /grid-test route #5662
• [Studio] Add --with-llama-cpp-dir installer flag to reuse a local llama.cpp by @LeoBorcherding in [Studio] Add --with-llama-cpp-dir installer flag to reuse a local llama.cpp #6472
• Add MLX-aware public Unsloth trainer API by @Lyxot in Add MLX-aware public Unsloth trainer API #6462
• report a complete load once llama-server is healthy by @hakanbaysal in report a complete load once llama-server is healthy #6790
• fast_generate: clear error for vLLM-style inputs when fast_inference=False by @danielhanchen in fast_generate: clear error for vLLM-style inputs when fast_inference=False #6786
• CLI: Rename unsloth connect to unsloth start by @NilayYadav in CLI: Rename unsloth connect to unsloth start #6613
• Tool-call healing (default on) and opt-in nudging for the client-tool passthrough by @danielhanchen in Tool-call healing (default on) and opt-in nudging for the client-tool passthrough #6801
• Studio: multi-select export formats, portable FP8/INT8, GGUF LoRA, and source parity by @danielhanchen in Studio: multi-select export formats, portable FP8/INT8, GGUF LoRA, and source parity #6767
• Studio: flush passthrough stream headers before upstream prefill stalls by @rodboev in Studio: flush passthrough stream headers before upstream prefill stalls #6835
• Fix TrainingArguments silently disabling unsloth gradient checkpointing by @oobabooga in Fix TrainingArguments silently disabling unsloth gradient checkpointing #6829
• Studio: reserve CUDA context and mmproj/MTP soft overhead in the GGUF fit budget by @danielhanchen in Studio: reserve CUDA context and mmproj/MTP soft overhead in the GGUF fit budget #6718
• Fix Studio custom folders on Linux external drives by @ramisworld in Fix Studio custom folders on Linux external drives #6799
• Guard Windows ROCm torchao override skip by @InfoSage05 in Guard Windows ROCm torchao override skip #6837
• Fix export-time trust_remote_code bypass in FP8/INT8/GGUF-LoRA export by @danielhanchen in Fix export-time trust_remote_code bypass in FP8/INT8/GGUF-LoRA export #6869
• Harden Windows Pester install against missing PSGallery by @danielhanchen in Harden Windows Pester install against missing PSGallery #6892
• Auto Xet to HTTP download fallback in from_pretrained; share Studio's fallback via unsloth_zoo by @danielhanchen in Auto Xet to HTTP download fallback in from_pretrained; share Studio's fallback via unsloth_zoo #6638
• GRPO: sequence packing for the no-grad old/ref logp path (default-on) by @danielhanchen in GRPO: sequence packing for the no-grad old/ref logp path (default-on) #6738
• Add PrefixGrouper for GRPO: dedup the shared prompt across a group's completions by @danielhanchen in Add PrefixGrouper for GRPO: dedup the shared prompt across a group's completions #6871
• Handle odd shapes and non-float scales in FP8BlockQuantLinear by @danielhanchen in Handle odd shapes and non-float scales in FP8BlockQuantLinear #6848
• Scope MoE expert LoRA detection to actual MLP projection targets by @danielhanchen in Scope MoE expert LoRA detection to actual MLP projection targets #6849
• Honor an explicit sdpa or flex_attention request when flash is disabled by @danielhanchen in Honor an explicit sdpa or flex_attention request when flash is disabled #6847
• Auto-enable grouped MoE on loaded / PEFT'd models via loader hook by @danielhanchen in Auto-enable grouped MoE on loaded / PEFT'd models via loader hook #6727
• Respect requested 4bit/8bit for local '-bf16' directories by @danielhanchen in Respect requested 4bit/8bit for local '-bf16' directories #6726
• Sync FORCE_FLOAT32 fallback with unsloth-zoo (gemma4, glm4_moe, qwen3_moe) by @danielhanchen in Sync FORCE_FLOAT32 fallback with unsloth-zoo (gemma4, glm4_moe, qwen3_moe) #6865
• Note bundled flash-linear-attention kernels for gated-deltanet models by @danielhanchen in Note bundled flash-linear-attention kernels for gated-deltanet models #6850
• CI: pin lockfile-audit actions to commit SHAs by @anxkhn in CI: pin lockfile-audit actions to commit SHAs #6902
• fix(fp8): use int64 offsets in weight_dequant_kernel by @anxkhn in fix(fp8): use int64 offsets in weight_dequant_kernel #6884
• fix: map None quant method to q8_0 before lowercasing in GGUF export by @anxkhn in fix: map None quant method to q8_0 before lowercasing in GGUF export #6889
• fix: correct class name in SyntheticDataKit.chunk_data guard message by @anxkhn in fix: correct class name in SyntheticDataKit.chunk_data guard message #6901
• studio: label RAM and VRAM readouts as GiB not GB by @danielhanchen in studio: label RAM and VRAM readouts as GiB not GB #6895
• Fix llama3 RoPE scaling dropped on transformers v5 by @danielhanchen in Fix llama3 RoPE scaling dropped on transformers v5 #6907
• Fix ruff-format spacing drift in loader.py by @danielhanchen in Fix ruff-format spacing drift in loader.py #6910
• studio: tool calling + healing parity for Llama-3, Mistral, Gemma 4 on safetensors + MLX by @danielhanchen in studio: tool calling + healing parity for Llama-3, Mistral, Gemma 4 on safetensors + MLX #5620
• Studio: stop chat generation on the assistant-turn-end token (fixes Qwen3.5 loop) by @danielhanchen in Studio: stop chat generation on the assistant-turn-end token (fixes Qwen3.5 loop) #6804
• studio: deterministic backend tool-calling wiring test by @danielhanchen in studio: deterministic backend tool-calling wiring test #6836
• Studio: coerce tool_call arguments to dict before chat templating (fixes MLX tool follow-up error) by @danielhanchen in Studio: coerce tool_call arguments to dict before chat templating (fixes MLX tool follow-up error) #6807
• Studio: Gemma tool-call streaming follow-ups + nested-XML escape fix (Fix Gemma 4 GGUF OpenAI API streams #6476) by @danielhanchen in Studio: Gemma tool-call streaming follow-ups + nested-XML escape fix (#6476) #6611
• studio: tool calling for DeepSeek (R1/V3/V3.1), GLM 4.x, Kimi K2 on safetensors + MLX by @danielhanchen in studio: tool calling for DeepSeek (R1/V3/V3.1), GLM 4.x, Kimi K2 on safetensors + MLX #5624
• scan_packages: baseline transitive-dep drift in the supply-chain scan by @danielhanchen in scan_packages: baseline transitive-dep drift in the supply-chain scan #6917
• Studio: parse Mistral [TOOL_CALLS] and rehearsal tool-call shapes by @danielhanchen in Studio: parse Mistral [TOOL_CALLS] and rehearsal tool-call shapes #5704
• Studio: exclude mlx-lm 0.31.3 (broke gemma4/qwen3_5 QK-norm load on Apple Silicon) by @danielhanchen in Studio: exclude mlx-lm 0.31.3 (broke gemma4/qwen3_5 QK-norm load on Apple Silicon) #6803
• Studio chat: tool-call nudging on by default (API stays opt-in) by @danielhanchen in Studio chat: tool-call nudging on by default (API stays opt-in) #6883
• Studio: close switch/cancel races during model load by @danielhanchen in Studio: close switch/cancel races during model load #6918
• Studio: client-tool passthrough healing for safetensors and MLX by @danielhanchen in Studio: client-tool passthrough healing for safetensors and MLX #6870
• Studio: keep the nudge wiring test collectable without the unsloth stack by @danielhanchen in Studio: keep the nudge wiring test collectable without the unsloth stack #6924
• Studio: serialize the compare-mode dispatcher lifecycle to fix a start race by @danielhanchen in Studio: serialize the compare-mode dispatcher lifecycle to fix a start race #6922
• Studio: apply presence_penalty on the safetensors and MLX inference paths by @danielhanchen in Studio: apply presence_penalty on the safetensors and MLX inference paths #6923
• Fix repeated base model downloads across checkpoint exports by @shimmyshimmer in Fix repeated base model downloads across checkpoint exports #6896
• Studio: fix torch CUDA undefined-symbol errors from a conflicting LD_LIBRARY_PATH by @danielhanchen in Studio: fix torch CUDA undefined-symbol errors from a conflicting LD_LIBRARY_PATH #6905
• Clear stale yolo approval state on no-launch reruns by @danielhanchen in Clear stale yolo approval state on no-launch reruns #6868
• ROCm-on-WSL: support discrete Radeon (RDNA 3/4) in WSL, not just Strix Halo by @LeoBorcherding in ROCm-on-WSL: support discrete Radeon (RDNA 3/4) in WSL, not just Strix Halo #6915
• Guard RoPE scaling against the transformers v5 buffer blank; honor extended RoPE factor by @danielhanchen in Guard RoPE scaling against the transformers v5 buffer blank; honor extended RoPE factor #6925
• Run the malware gate on the RAG embedding model before it loads by @danielhanchen in Run the malware gate on the RAG embedding model before it loads #6887
• Add RDNA 2/3/4 ROCm routing tests via a CPU-only torch spoof by @danielhanchen in Add RDNA 2/3/4 ROCm routing tests via a CPU-only torch spoof #6935
• GRPO: default router_aux_loss_coef to 0 on TRL >= 1.7.0 by @danielhanchen in GRPO: default router_aux_loss_coef to 0 on TRL >= 1.7.0 #6938
• Add DeepSeek-V4-Flash-GGUF to Studio with none/high/max reasoning by @danielhanchen in Add DeepSeek-V4-Flash-GGUF to Studio with none/high/max reasoning #6908

New Contributors

• @vineethsaivs made their first contribution in Fix SyntheticDataKit.chunk_data dropping single-chunk documents #6595
• @zrva made their first contribution in feat: add GPU-aware model filtering and For You section- Add fit filt… #6645
• @Abdul-Moiz31 made their first contribution in Fix DDP crash from CPU-resident rotary inv_freq buffer #6662
• @Cesarsk made their first contribution in fix(install): enable UV_NATIVE_TLS on macOS for corporate TLS-inspection proxies #6671
• @Muhammad-Ikhwan-Fathulloh made their first contribution in perf(dataprep): cache regex and field lists, fix typos #6714
• @Oscilloscope98 made their first contribution in Fix triton-xpu 3.7.1 sha256 hashes for Intel XPU install #6629
• @DovahkiinYuzuko made their first contribution in feat(i18n): add Japanese locale support for the Studio UI catalog #6705
• @rodboev made their first contribution in fix: keep LoRA reloads working with PEFT 0.19 #6748
• @ftrajkov-amd made their first contribution in Fix GRPO logit scaling when model is wrapped by DDP #5955
• @Dspofu made their first contribution in feat: Implementation of the Portuguese (Brazil) language and VRAM/RAM monitor #6509
• @InfoSage05 made their first contribution in Fix: skip fp16/bf16 validation for full finetuning in RL trainers #6813
• @hakanbaysal made their first contribution in report a complete load once llama-server is healthy #6790
• @ramisworld made their first contribution in Fix Studio custom folders on Linux external drives #6799
• @anxkhn made their first contribution in CI: pin lockfile-audit actions to commit SHAs #6902

Full Changelog: v0.1.471-beta...v0.1.48-beta

What's Changed in Unsloth Zoo

• Fix Gemma3 compiled SFT crash (ArgsMismatchError on q_norm/k_norm) by @danielhanchen in Fix Gemma3 compiled SFT crash (ArgsMismatchError on q_norm/k_norm) unsloth-zoo#804
• training_utils: respect PEFT bias training in prepare_model_for_training ([Bug] Bias Training Does Not Work #2343) by @danielhanchen in training_utils: respect PEFT bias training in prepare_model_for_training (#2343) unsloth-zoo#795
• tests: verify the MoE fused experts + attention merge (qwen3_5_moe build + model-space check, gemma4) by @danielhanchen in tests: verify the MoE fused experts + attention merge (qwen3_5_moe build + model-space check, gemma4) unsloth-zoo#792
• Fix LoRA scaling count mismatch on merge for Qwen2.5-VL exports ([Bug] Assertion error when exporting Qwen2.5 VL with no extra information #2966) by @danielhanchen in Fix LoRA scaling count mismatch on merge for Qwen2.5-VL exports (#2966) unsloth-zoo#806
• Fix Llama/Mixtral MoE LoRA merge dropping unfused expert deltas ([Bug] Llama 3.1 Moe Training , Wrong dimenion of MOE tensors after training / Unsloth override issue #5403) by @danielhanchen in Fix Llama/Mixtral MoE LoRA merge dropping unfused expert deltas (#5403) unsloth-zoo#812
• Generalize pad_token repair into a shared module by @danielhanchen in Generalize pad_token repair into a shared module unsloth-zoo#816
• Fix GRPO eval crash: keep gradient checkpointing buffers out of inference mode by @danielhanchen in Fix GRPO eval crash: keep gradient checkpointing buffers out of inference mode unsloth-zoo#805
• peft_utils: add finetune_audio_layers for Gemma 4 / Gemma 3N audio + embedder LoRA by @danielhanchen in peft_utils: add finetune_audio_layers for Gemma 4 / Gemma 3N audio + embedder LoRA unsloth-zoo#814
• Allow pad-named tokens (e.g. <|vision_pad|>) as valid pad_tokens by @danielhanchen in Allow pad-named tokens (e.g. <|vision_pad|>) as valid pad_tokens unsloth-zoo#831
• Apply extra_special_tokens coercion on the MLX load path by @danielhanchen in Apply extra_special_tokens coercion on the MLX load path unsloth-zoo#835
• Add imatrix option to quantize_gguf for IQ low-bit GGUF quants by @danielhanchen in Add imatrix option to quantize_gguf for IQ low-bit GGUF quants unsloth-zoo#839
• train_on_responses_only: faster filtering, safer collator and num_proc, clearer errors by @danielhanchen in train_on_responses_only: faster filtering, safer collator and num_proc, clearer errors unsloth-zoo#836
• Auto-detect instruction/response parts in train_on_responses_only by @danielhanchen in Auto-detect instruction/response parts in train_on_responses_only unsloth-zoo#834
• Feat(mlx): Add NEFTune (neftune_noise_alpha) for text models to MLXTrainer by @BardiaKoopah in Feat(mlx): Add NEFTune (neftune_noise_alpha) for text models to MLXTrainer unsloth-zoo#824
• feat(mlx): runtime bnb dequant -> MLX affine 4-bit on Apple Silicon by @BardiaKoopah in feat(mlx): runtime bnb dequant -> MLX affine 4-bit on Apple Silicon unsloth-zoo#815
• Feat(mlx): Add early stopping + load_best_model_at_end to MLXTrainer by @BardiaKoopah in Feat(mlx): Add early stopping + load_best_model_at_end to MLXTrainer unsloth-zoo#822
• Feat(mlx): Wire report_to wandb/tensorboard for standalone MLXTrainer by @BardiaKoopah in Feat(mlx): Wire report_to wandb/tensorboard for standalone MLXTrainer unsloth-zoo#821
• Align MLX trainer internals for public UnslothTrainer by @Lyxot in Align MLX trainer internals for public UnslothTrainer unsloth-zoo#790
• GRPO: sequence packing for the gradient logp path (default-on) by @danielhanchen in GRPO: sequence packing for the gradient logp path (default-on) unsloth-zoo#840
• Fix Gemma3 processor batched tokenization (ragged padding) by @danielhanchen in Fix Gemma3 processor batched tokenization (ragged padding) unsloth-zoo#843
• Fix MLX trainer tests stale after load_best_model_at_end + trainer internals by @danielhanchen in Fix MLX trainer tests stale after load_best_model_at_end + trainer internals unsloth-zoo#853
• Keep CUDA graphs with LoRA on vLLM >= 0.19 by @danielhanchen in Keep CUDA graphs with LoRA on vLLM >= 0.19 unsloth-zoo#867
• Add shared Xet to HTTP stall fallback (hf_xet_fallback, hf_cache_state) by @danielhanchen in Add shared Xet to HTTP stall fallback (hf_xet_fallback, hf_cache_state) unsloth-zoo#829
• Dequantize FP8 dense models on merged_16bit save (fixes unsloth#4919) by @danielhanchen in Dequantize FP8 dense models on merged_16bit save (fixes unsloth#4919) unsloth-zoo#803
• Support DoRA (use_dora=True) in the safetensors LoRA merge by @danielhanchen in Support DoRA (use_dora=True) in the safetensors LoRA merge unsloth-zoo#828
• Make mxfp4 converter import resilient to a missing _cpu variant by @danielhanchen in Make mxfp4 converter import resilient to a missing _cpu variant unsloth-zoo#826
• GRPO PrefixGrouper: shared-prefix grad path in grpo_accumulated_loss by @danielhanchen in GRPO PrefixGrouper: shared-prefix grad path in grpo_accumulated_loss unsloth-zoo#868
• Keep DeepSeek-V4 hyper-connection mixers eager to stop backward inf by @danielhanchen in Keep DeepSeek-V4 hyper-connection mixers eager to stop backward inf unsloth-zoo#859
• Cap the fused cross entropy chunk target so chunking stays active on large GPUs by @danielhanchen in Cap the fused cross entropy chunk target so chunking stays active on large GPUs unsloth-zoo#857
• Batch the fused MoE expert LoRA merge into one matmul on GPU by @danielhanchen in Batch the fused MoE expert LoRA merge into one matmul on GPU unsloth-zoo#827
• Load prequantized bnb 4-bit MoE expert checkpoints under transformers v5 by @danielhanchen in Load prequantized bnb 4-bit MoE expert checkpoints under transformers v5 unsloth-zoo#856
• Optional recompute-in-backward for the v5 grouped MoE path by @danielhanchen in Optional recompute-in-backward for the v5 grouped MoE path unsloth-zoo#838
• Add a grouped bnb 4-bit training forward for gpt-oss experts by @danielhanchen in Add a grouped bnb 4-bit training forward for gpt-oss experts unsloth-zoo#862
• Optional gate gradient identity for the grouped MoE combine by @danielhanchen in Optional gate gradient identity for the grouped MoE combine unsloth-zoo#858
• Vendor flash-linear-attention gated-delta kernels (no pip install needed) by @danielhanchen in Vendor flash-linear-attention gated-delta kernels (no pip install needed) unsloth-zoo#865
• Grouped-GEMM MoE forward for transformers<5 ModuleList experts by @danielhanchen in Grouped-GEMM MoE forward for transformers<5 ModuleList experts unsloth-zoo#837
• Make float16 training work for gemma4, qwen3_moe and glm4_moe by @danielhanchen in Make float16 training work for gemma4, qwen3_moe and glm4_moe unsloth-zoo#866
• Route gemma-4 sliding-window layers through FlashAttention-2 by @danielhanchen in Route gemma-4 sliding-window layers through FlashAttention-2 unsloth-zoo#860
• Fix MLX notebook generate input/output compatibility by @Lyxot in Fix MLX notebook generate input/output compatibility unsloth-zoo#855
• Refuse DoRA on mxfp4 packed MoE before dequant; refresh bnb4bit patch-count canary by @danielhanchen in Refuse DoRA on mxfp4 packed MoE before dequant; refresh bnb4bit patch-count canary unsloth-zoo#871
• Skip bnb4bit per-expert MoE tests against the transformers<5 core_model_loading stub by @danielhanchen in Skip bnb4bit per-expert MoE tests against the transformers<5 core_model_loading stub unsloth-zoo#876
• saving utils check both model.base_model and base_model.model by @toranb in saving utils check both model.base_model and base_model.model unsloth-zoo#95

---

This discussion was created from the release DeepSeek-V4 + NVFP4 Exporting.