Skip to content

Kimi K3 + Deep Research + Parallel Chat

Latest
Latest

Choose a tag to compare

View all tags
@shimmyshimmer shimmyshimmer released this 29 Jul 15:35
v0.1.51-beta
ae4cb71
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Kimi K3 + Parallel Chat + Deep Research

Hey everyone! Kimi K3 can now run locally with Unsloth Dynamic GGUFs, Unsloth can keep multiple chats generating in parallel, and the new Deep Research mode plans, reads and cites sources using your local model.

This release also brings better AMD and Intel GPU support, DoRA training, and many installer, MLX, export and inference fixes.

Kimi K3

Moonshot AI’s Kimi K3 is a 2.8T-parameter MoE model with 104B active parameters, native vision support and a 1M context window. Kimi K3 is thinking-only and supports low, high and max reasoning effort.

You can run our Kimi K3 Dynamic GGUFs through Unsloth or llama.cpp. Unsloth automatically detects multi-GPU setups and can offload model layers to system memory.

Kimi K3 is a very large model, so plan your hardware accordingly:

  • UD-IQ1_S is 595GB in disk space.
  • UD-Q4_K_XL is 1.51TB in disk space.
  • For lossless inference, use UD-Q8_K_XL, which is 1.56TB in disk space.

Read our full Kimi K3 guide and learn more about Unsloth Dynamic 2.0 GGUFs.

Parallel Chat

Unsloth can now run multiple conversations at once. Starting a New Chat leaves the previous answer generating, and each active conversation gets its own progress indicator and Stop control.

  • 4 llama-server slots by default, adjustable in the web UI.
  • Tools, uploads, self-healing and agents stay isolated between chats.
  • Stop one chat without interrupting others or restarting the server.
  • Unsloth reduces the slot count when memory is limited.
  • Reloading the model still stops active chats after confirmation.

Deep Research

Deep Research turns a local model into a complete research workflow. Give it a question and it will create a plan, search the web, organize the evidence and produce a cited report.

  • Review and edit the plan before research begins.
  • Follow its progress and collected sources while it works.
  • Resume or cancel without losing completed research.
  • Allow or block websites to control source selection.
  • Sources and citations stay saved with each run.
  • Before writing, Unsloth checks for unsupported claims, contradictions and unresolved gaps. Recommendations without direct evidence are marked as testable inferences.

One run can be active per chat. Deep Research currently works with local models. Optional full-page grounding can read top search results into temporary RAG before synthesis; enable it with UNSLOTH_RESEARCH_AUTO_SCRAPE=1. It remains off by default because it adds scraping time and needs additional context.

Improved AMD Support

This update builds on our initial AMD release and setup guide with support for more GPUs and more reliable ROCm inference and training.

  • Improved detection for RDNA2, Radeon, Ryzen, Strix Halo and workstation GPUs.
  • MI50 and Radeon VII support 16-bit LoRA and full fine-tuning on Linux.
  • Fixed 4-bit NaNs, library conflicts and long RDNA startup stalls.
  • Unsupported Windows HIP GPUs can fall back to Vulkan.
  • Vulkan devices show their real names and can be selected individually.
  • Clean Windows installs no longer require Winget or developer tools.

Training, Models and Platform Updates

  • Intel XPU enables local chat and training on Intel Arc and Data Center GPUs.
  • DoRA is available alongside LoRA and full fine-tuning.
  • Large exports can use all visible GPUs to avoid GPU 0 memory limits.
  • MLX adds streaming datasets, continued pretraining, callbacks and better VLM and LoRA support.
  • Fixed incorrect flash_attention_2 model output.
  • Unsloth and saving utilities now work without bitsandbytes.
  • Fixed .json datasets and Qwen3.5/3.6 MoE and GRPO notebook setup.
  • Hub download folders are easier to find and open.
  • Release notes are available in the Unsloth update popup.
  • unsloth start --as-subagent lets Codex, Claude Code and Pi delegate tasks to a local Unsloth model.

Updating / installing Unsloth

macOS, Linux, WSL:

curl -fsSL https://unsloth.ai/install.sh | sh

Windows:

irm https://unsloth.ai/install.ps1 | iex

What's Changed

New Contributors

Full Changelog: v0.1.50-beta...v0.1.510-beta

What's Changed in Unsloth-Zoo

Contributors

pwasiewi, hakanbaysal, and 27 other contributors
Assets 2
Loading
0 Join discussion