Kimi K3 + Deep Research + Parallel Chat #7620
danielhanchen
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Kimi K3 + Parallel Chat + Deep Research
Hey everyone! Kimi K3 can now run locally with Unsloth Dynamic GGUFs, Unsloth can keep multiple chats generating in parallel, and the new Deep Research mode plans, reads and cites sources using your local model.
This release also brings better AMD and Intel GPU support, DoRA training, and many installer, MLX, export and inference fixes.
Kimi K3
Moonshot AI’s Kimi K3 is a 2.8T-parameter MoE model with 104B active parameters, native vision support and a 1M context window. Kimi K3 is thinking-only and supports low, high and max reasoning effort.
You can run our Kimi K3 Dynamic GGUFs through Unsloth or
llama.cpp. Unsloth automatically detects multi-GPU setups and can offload model layers to system memory.Kimi K3 is a very large model, so plan your hardware accordingly:
UD-IQ1_Sis 595GB in disk space.UD-Q4_K_XLis 1.51TB in disk space.UD-Q8_K_XL, which is 1.56TB in disk space.Read our full Kimi K3 guide and learn more about Unsloth Dynamic 2.0 GGUFs.
Parallel Chat
Unsloth can now run multiple conversations at once. Starting a New Chat leaves the previous answer generating, and each active conversation gets its own progress indicator and Stop control.
Deep Research
Deep Research turns a local model into a complete research workflow. Give it a question and it will create a plan, search the web, organize the evidence and produce a cited report.
One run can be active per chat. Deep Research currently works with local models. Optional full-page grounding can read top search results into temporary RAG before synthesis; enable it with
UNSLOTH_RESEARCH_AUTO_SCRAPE=1. It remains off by default because it adds scraping time and needs additional context.Improved AMD Support
This update builds on our initial AMD release and setup guide with support for more GPUs and more reliable ROCm inference and training.
Training, Models and Platform Updates
flash_attention_2model output..jsondatasets and Qwen3.5/3.6 MoE and GRPO notebook setup.unsloth start --as-subagentlets Codex, Claude Code and Pi delegate tasks to a local Unsloth model.Updating / installing Unsloth
macOS, Linux, WSL:
curl -fsSL https://unsloth.ai/install.sh | shWindows:
What's Changed
import unslothworking when bitsandbytes is absent by @danielhanchen in Keepimport unslothworking when bitsandbytes is absent #7502New Contributors
Full Changelog: v0.1.50-beta...v0.1.510-beta
What's Changed in Unsloth-Zoo
This discussion was created from the release Kimi K3 + Deep Research + Parallel Chat.
All reactions