model: MTP support for Qwen3-Next (#25589)
-
mtp for qwen3nex
-
fix for python type-check
-
Fix to compute num_mtp from directly mtp layer
-
define opt_num_mtp_layers in _QwenMtpMixin and fix some comments
-
Fix for python type check
-
Update gguf-py/gguf/constants.py
Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co
-
rebase and add load_mtp flags
-
Update src/models/qwen3next.cpp
Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co
- Update src/models/qwen3next.cpp
Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co
Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co
Website:
macOS/iOS:
- macOS Apple Silicon (arm64)
- macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED
- macOS Intel (x64)
- iOS XCFramework
Linux:
- Ubuntu x64 (CPU)
- Ubuntu arm64 (CPU)
- Ubuntu s390x (CPU)
- Ubuntu x64 (Vulkan)
- Ubuntu arm64 (Vulkan)
- Ubuntu x64 (ROCm 7.2)
- Ubuntu x64 (OpenVINO)
- Ubuntu x64 (SYCL FP32)
- Ubuntu x64 (SYCL FP16)
Android:
Windows:
- Windows x64 (CPU)
- Windows arm64 (CPU)
- Windows arm64 (OpenCL Adreno)
- Windows x64 (CUDA 12) - CUDA 12.4 DLLs
- Windows x64 (CUDA 13) - CUDA 13.3 DLLs
- Windows x64 (Vulkan)
- Windows x64 (OpenVINO)
- Windows x64 (SYCL)
- Windows x64 (HIP)
openEuler:
- DISABLED
- openEuler x86 (310p)
- openEuler x86 (910b, ACL Graph)
- openEuler aarch64 (310p)
- openEuler aarch64 (910b, ACL Graph)
UI: