An error occurred when directly deploying the Qwen3-Next 80B-A3B-Thinking model saved by the save_pretrained of transformers using vllm. However, directly deploying the original model was normal

[zhanlun150729]

System Info

System Info

Who can help?

(zhihu_0210_bak_0403) root@nb-zhanlun-zl-0914-1-0:/home# python collect_env.py
Collecting environment information...

    System Info

==============================
OS : Ubuntu 22.04.4 LTS (x86_64)
GCC version : (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version : Could not collect
CMake version : version 3.28.3
Libc version : glibc-2.35

==============================
PyTorch Info

PyTorch version : 2.8.0+cu128
Is debug build : False
CUDA used to build PyTorch : 12.8
ROCM used to build PyTorch : N/A

==============================
Python Environment

Python version : 3.10.14 (main, May 6 2024, 19:42:50) [GCC 11.2.0] (64-bit runtime)
Python platform : Linux-5.4.0-42-generic-x86_64-with-glibc2.35

==============================
CUDA / GPU Info

Is CUDA available : True
CUDA runtime version : 12.4.99
CUDA_MODULE_LOADING set to : LAZY
GPU models and configuration :
GPU 0: NVIDIA A800-SXM4-80GB
GPU 1: NVIDIA A800-SXM4-80GB
GPU 2: NVIDIA A800-SXM4-80GB
GPU 3: NVIDIA A800-SXM4-80GB
GPU 4: NVIDIA A800-SXM4-80GB
GPU 5: NVIDIA A800-SXM4-80GB
GPU 6: NVIDIA A800-SXM4-80GB
GPU 7: NVIDIA A800-SXM4-80GB

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

An error occurred when directly deploying the Qwen3-Next 80B-A3B-Thinking model saved by the save_pretrained of transformers using vllm. However, directly deploying the original model was normal

Expected behavior

正常部署，实际报错

[hmellor]

Hi @zhanlun150729,

The error you're seeing happens during modle loading when vLLM has found weights in your checkpoint that it has no place for. However, looking at https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Thinking/blob/main/model.safetensors.index.json, it's clear that the weights in your error are present in the original checkpoint.

I need a bit more information to know what's going on:

• Which version of vLLM are you using?
• Can you share the whole log as text (not a screenshot) so I can see if anything looks unusual before the weight loading starts?

[pt-coder]

I also encountered the same problem. I found that the model after saving did not have the MTP layer. Is this the reason why vllm cannot be loaded?

VLLM version: v0.10.2
transformers version: transformers-4.57.0.dev0

[zhanlun150729]

Hi @zhanlun150729,

The error your seeing happens during modle loading when vLLM has found weights in your checkpoint that it has no place for. However, looking at https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Thinking/blob/main/model.safetensors.index.json, it's clear that the weights in your error are present in the original checkpoint.

I need a bit more information to know what's going on:

• Which version of vLLM are you using?
• Can you share the whole log as text (not a screenshot) so I can see if anything looks unusual before the weight loading starts?

完整日志如下：

logs

(Worker_TP0 pid=217223) 
(Worker_TP1 pid=217224) INFO 09-19 09:27:33 [default_loader.py:268] Loading weights took 1073.92 seconds
(Worker_TP1 pid=217224) INFO 09-19 09:27:33 [gpu_model_runner.py:2380] Loading drafter model...
(Worker_TP2 pid=217225) INFO 09-19 09:27:33 [default_loader.py:268] Loading weights took 1074.11 seconds
(Worker_TP3 pid=217226) INFO 09-19 09:27:33 [default_loader.py:268] Loading weights took 1074.11 seconds
(Worker_TP0 pid=217223) INFO 09-19 09:27:33 [default_loader.py:268] Loading weights took 1074.18 seconds
(Worker_TP2 pid=217225) INFO 09-19 09:27:33 [gpu_model_runner.py:2380] Loading drafter model...
(Worker_TP3 pid=217226) INFO 09-19 09:27:33 [gpu_model_runner.py:2380] Loading drafter model...
(Worker_TP0 pid=217223) INFO 09-19 09:27:33 [gpu_model_runner.py:2380] Loading drafter model...
Loading safetensors checkpoint shards:   0% Completed | 0/64 [00:00<?, ?it/s]
Loading safetensors checkpoint shards:   5% Completed | 3/64 [00:00<00:02, 21.31it/s]
Loading safetensors checkpoint shards:   9% Completed | 6/64 [00:00<00:02, 21.45it/s]
Loading safetensors checkpoint shards:  14% Completed | 9/64 [00:00<00:02, 21.39it/s]
Loading safetensors checkpoint shards:  19% Completed | 12/64 [00:00<00:02, 21.38it/s]
Loading safetensors checkpoint shards:  23% Completed | 15/64 [00:00<00:02, 21.34it/s]
Loading safetensors checkpoint shards:  28% Completed | 18/64 [00:02<00:08,  5.46it/s]
Loading safetensors checkpoint shards:  33% Completed | 21/64 [00:02<00:05,  7.23it/s]
Loading safetensors checkpoint shards:  38% Completed | 24/64 [00:02<00:04,  9.15it/s]
Loading safetensors checkpoint shards:  42% Completed | 27/64 [00:02<00:03, 11.11it/s]
Loading safetensors checkpoint shards:  47% Completed | 30/64 [00:02<00:02, 13.02it/s]
Loading safetensors checkpoint shards:  52% Completed | 33/64 [00:02<00:02, 14.77it/s]
Loading safetensors checkpoint shards:  56% Completed | 36/64 [00:02<00:01, 16.31it/s]
Loading safetensors checkpoint shards:  61% Completed | 39/64 [00:03<00:01, 17.59it/s]
Loading safetensors checkpoint shards:  66% Completed | 42/64 [00:03<00:01, 18.65it/s]
Loading safetensors checkpoint shards:  70% Completed | 45/64 [00:03<00:00, 19.38it/s]
Loading safetensors checkpoint shards:  75% Completed | 48/64 [00:03<00:00, 19.98it/s]
Loading safetensors checkpoint shards:  80% Completed | 51/64 [00:03<00:00, 20.42it/s]
Loading safetensors checkpoint shards:  84% Completed | 54/64 [00:03<00:00, 20.71it/s]
Loading safetensors checkpoint shards:  89% Completed | 57/64 [00:03<00:00, 20.91it/s]
(Worker_TP2 pid=217225) INFO 09-19 09:27:39 [default_loader.py:268] Loading weights took 3.97 seconds
Loading safetensors checkpoint shards:  94% Completed | 60/64 [00:04<00:00, 21.06it/s]
Loading safetensors checkpoint shards:  98% Completed | 63/64 [00:04<00:00, 21.18it/s]
(Worker_TP1 pid=217224) INFO 09-19 09:27:39 [default_loader.py:268] Loading weights took 4.20 seconds
Loading safetensors checkpoint shards: 100% Completed | 64/64 [00:04<00:00, 15.26it/s]
(Worker_TP0 pid=217223) 
(Worker_TP3 pid=217226) INFO 09-19 09:27:39 [default_loader.py:268] Loading weights took 4.21 seconds
(Worker_TP0 pid=217223) INFO 09-19 09:27:39 [default_loader.py:268] Loading weights took 4.22 seconds
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585] WorkerProc failed to start.
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585] Traceback (most recent call last):
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 559, in worker_main
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     worker = WorkerProc(*args, **kwargs)
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 427, in __init__
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.worker.load_model()
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/worker/gpu_worker.py", line 213, in load_model
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.model_runner.load_model(eep_scale_up=eep_scale_up)
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 2381, in load_model
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.drafter.load_model(self.model)
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/spec_decode/eagle.py", line 620, in load_model
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.model = get_model(vllm_config=self.vllm_config,
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/model_executor/model_loader/__init__.py", line 119, in get_model
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     return loader.load_model(vllm_config=vllm_config,
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/model_executor/model_loader/base_loader.py", line 50, in load_model
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.load_weights(model, model_config)
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/model_executor/model_loader/default_loader.py", line 277, in load_weights
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     raise ValueError("Following weights were not initialized from "
(Worker_TP2 pid=217225) ERROR 09-19 09:27:40 [multiproc_executor.py:585] ValueError: Following weights were not initialized from checkpoint: {'model.layers.0.mlp.gate.weight', 'model.layers.0.mlp.experts.w13_weight', 'model.layers.0.mlp.shared_expert_gate.weight', 'model.layers.0.post_attention_layernorm.weight', 'model.layers.0.mlp.shared_expert.down_proj.weight', 'model.fc.weight', 'model.layers.0.self_attn.qkv_proj.weight', 'model.norm.weight', 'model.layers.0.self_attn.k_norm.weight', 'model.pre_fc_norm_hidden.weight', 'model.layers.0.mlp.shared_expert.gate_up_proj.weight', 'model.layers.0.self_attn.q_norm.weight', 'model.pre_fc_norm_embedding.weight', 'model.layers.0.mlp.experts.w2_weight', 'model.layers.0.self_attn.o_proj.weight', 'model.layers.0.input_layernorm.weight'}
(Worker_TP2 pid=217225) INFO 09-19 09:27:40 [multiproc_executor.py:546] Parent process exited, terminating worker
(Worker_TP0 pid=217223) Exception ignored in: <function _xla_gc_callback at 0x7fd96523bd90>
(Worker_TP0 pid=217223) Traceback (most recent call last):
(Worker_TP0 pid=217223)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/jax/_src/lib/__init__.py", line 118, in _xla_gc_callback
(Worker_TP0 pid=217223)     def _xla_gc_callback(*args):
(Worker_TP0 pid=217223)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 526, in signal_handler
(Worker_TP0 pid=217223) INFO 09-19 09:27:40 [multiproc_executor.py:546] Parent process exited, terminating worker
(Worker_TP0 pid=217223)     raise SystemExit()
(Worker_TP0 pid=217223) SystemExit: 
(Worker_TP1 pid=217224) INFO 09-19 09:27:40 [multiproc_executor.py:546] Parent process exited, terminating worker
(Worker_TP1 pid=217224) Exception ignored in: <function _xla_gc_callback at 0x7f6a3af53d90>
(Worker_TP1 pid=217224) Traceback (most recent call last):
(Worker_TP1 pid=217224)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/jax/_src/lib/__init__.py", line 118, in _xla_gc_callback
(Worker_TP1 pid=217224)     def _xla_gc_callback(*args):
(Worker_TP1 pid=217224)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 526, in signal_handler
(Worker_TP1 pid=217224)     raise SystemExit()
(Worker_TP1 pid=217224) SystemExit: 
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585] WorkerProc failed to start.
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585] Traceback (most recent call last):
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 559, in worker_main
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     worker = WorkerProc(*args, **kwargs)
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 427, in __init__
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.worker.load_model()
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/worker/gpu_worker.py", line 213, in load_model
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.model_runner.load_model(eep_scale_up=eep_scale_up)
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 2381, in load_model
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.drafter.load_model(self.model)
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/spec_decode/eagle.py", line 620, in load_model
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.model = get_model(vllm_config=self.vllm_config,
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/model_executor/model_loader/__init__.py", line 119, in get_model
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     return loader.load_model(vllm_config=vllm_config,
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/model_executor/model_loader/base_loader.py", line 50, in load_model
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.load_weights(model, model_config)
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/model_executor/model_loader/default_loader.py", line 277, in load_weights
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     raise ValueError("Following weights were not initialized from "
(Worker_TP1 pid=217224) ERROR 09-19 09:27:40 [multiproc_executor.py:585] ValueError: Following weights were not initialized from checkpoint: {'model.pre_fc_norm_embedding.weight', 'model.layers.0.mlp.shared_expert.down_proj.weight', 'model.layers.0.mlp.experts.w13_weight', 'model.layers.0.self_attn.o_proj.weight', 'model.fc.weight', 'model.pre_fc_norm_hidden.weight', 'model.layers.0.input_layernorm.weight', 'model.layers.0.self_attn.qkv_proj.weight', 'model.layers.0.mlp.shared_expert.gate_up_proj.weight', 'model.layers.0.post_attention_layernorm.weight', 'model.layers.0.self_attn.q_norm.weight', 'model.norm.weight', 'model.layers.0.mlp.shared_expert_gate.weight', 'model.layers.0.self_attn.k_norm.weight', 'model.layers.0.mlp.gate.weight', 'model.layers.0.mlp.experts.w2_weight'}
(Worker_TP3 pid=217226) INFO 09-19 09:27:40 [multiproc_executor.py:546] Parent process exited, terminating worker
(Worker_TP3 pid=217226) Exception ignored in: <function _xla_gc_callback at 0x7f3110043d90>
(Worker_TP3 pid=217226) Traceback (most recent call last):
(Worker_TP3 pid=217226)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/jax/_src/lib/__init__.py", line 118, in _xla_gc_callback
(Worker_TP3 pid=217226)     def _xla_gc_callback(*args):
(Worker_TP3 pid=217226)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 526, in signal_handler
(Worker_TP3 pid=217226)     raise SystemExit()
(Worker_TP3 pid=217226) SystemExit: 
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585] WorkerProc failed to start.
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585] Traceback (most recent call last):
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 559, in worker_main
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     worker = WorkerProc(*args, **kwargs)
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 427, in __init__
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.worker.load_model()
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/worker/gpu_worker.py", line 213, in load_model
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.model_runner.load_model(eep_scale_up=eep_scale_up)
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 2381, in load_model
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.drafter.load_model(self.model)
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/spec_decode/eagle.py", line 620, in load_model
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.model = get_model(vllm_config=self.vllm_config,
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/model_executor/model_loader/__init__.py", line 119, in get_model
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     return loader.load_model(vllm_config=vllm_config,
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/model_executor/model_loader/base_loader.py", line 50, in load_model
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.load_weights(model, model_config)
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/model_executor/model_loader/default_loader.py", line 277, in load_weights
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     raise ValueError("Following weights were not initialized from "
(Worker_TP3 pid=217226) ERROR 09-19 09:27:40 [multiproc_executor.py:585] ValueError: Following weights were not initialized from checkpoint: {'model.layers.0.mlp.shared_expert.down_proj.weight', 'model.pre_fc_norm_embedding.weight', 'model.layers.0.self_attn.o_proj.weight', 'model.layers.0.self_attn.k_norm.weight', 'model.pre_fc_norm_hidden.weight', 'model.layers.0.mlp.experts.w13_weight', 'model.layers.0.mlp.gate.weight', 'model.layers.0.input_layernorm.weight', 'model.layers.0.self_attn.q_norm.weight', 'model.layers.0.mlp.shared_expert_gate.weight', 'model.fc.weight', 'model.layers.0.self_attn.qkv_proj.weight', 'model.norm.weight', 'model.layers.0.mlp.experts.w2_weight', 'model.layers.0.mlp.shared_expert.gate_up_proj.weight', 'model.layers.0.post_attention_layernorm.weight'}
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585] WorkerProc failed to start.
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585] Traceback (most recent call last):
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 559, in worker_main
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     worker = WorkerProc(*args, **kwargs)
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 427, in __init__
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.worker.load_model()
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/worker/gpu_worker.py", line 213, in load_model
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.model_runner.load_model(eep_scale_up=eep_scale_up)
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 2381, in load_model
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.drafter.load_model(self.model)
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/spec_decode/eagle.py", line 620, in load_model
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.model = get_model(vllm_config=self.vllm_config,
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/model_executor/model_loader/__init__.py", line 119, in get_model
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     return loader.load_model(vllm_config=vllm_config,
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/model_executor/model_loader/base_loader.py", line 50, in load_model
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     self.load_weights(model, model_config)
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/model_executor/model_loader/default_loader.py", line 277, in load_weights
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585]     raise ValueError("Following weights were not initialized from "
(Worker_TP0 pid=217223) ERROR 09-19 09:27:40 [multiproc_executor.py:585] ValueError: Following weights were not initialized from checkpoint: {'model.layers.0.self_attn.k_norm.weight', 'model.layers.0.mlp.gate.weight', 'model.layers.0.self_attn.q_norm.weight', 'model.layers.0.mlp.shared_expert.down_proj.weight', 'model.pre_fc_norm_hidden.weight', 'model.layers.0.mlp.experts.w2_weight', 'model.pre_fc_norm_embedding.weight', 'model.fc.weight', 'model.layers.0.self_attn.o_proj.weight', 'model.layers.0.mlp.experts.w13_weight', 'model.layers.0.self_attn.qkv_proj.weight', 'model.layers.0.mlp.shared_expert.gate_up_proj.weight', 'model.norm.weight', 'model.layers.0.input_layernorm.weight', 'model.layers.0.post_attention_layernorm.weight', 'model.layers.0.mlp.shared_expert_gate.weight'}
[rank0]:[W919 09:27:40.002776961 ProcessGroupNCCL.cpp:1538] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718] EngineCore failed to start.
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718] Traceback (most recent call last):
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 709, in run_engine_core
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718]     engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 505, in __init__
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718]     super().__init__(vllm_config, executor_class, log_stats,
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 82, in __init__
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718]     self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/executor/executor_base.py", line 54, in __init__
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718]     self._init_executor()
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 99, in _init_executor
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718]     self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718]   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 497, in wait_for_ready
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718]     raise e from None
(EngineCore_DP0 pid=217089) ERROR 09-19 09:27:42 [core.py:718] Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(EngineCore_DP0 pid=217089) Process EngineCore_DP0:
(EngineCore_DP0 pid=217089) Traceback (most recent call last):
(EngineCore_DP0 pid=217089)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore_DP0 pid=217089)     self.run()
(EngineCore_DP0 pid=217089)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/multiprocessing/process.py", line 108, in run
(EngineCore_DP0 pid=217089)     self._target(*self._args, **self._kwargs)
(EngineCore_DP0 pid=217089)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 722, in run_engine_core
(EngineCore_DP0 pid=217089)     raise e
(EngineCore_DP0 pid=217089)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 709, in run_engine_core
(EngineCore_DP0 pid=217089)     engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=217089)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 505, in __init__
(EngineCore_DP0 pid=217089)     super().__init__(vllm_config, executor_class, log_stats,
(EngineCore_DP0 pid=217089)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 82, in __init__
(EngineCore_DP0 pid=217089)     self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=217089)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/executor/executor_base.py", line 54, in __init__
(EngineCore_DP0 pid=217089)     self._init_executor()
(EngineCore_DP0 pid=217089)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 99, in _init_executor
(EngineCore_DP0 pid=217089)     self.workers = WorkerProc.wait_for_ready(unready_workers)
(EngineCore_DP0 pid=217089)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 497, in wait_for_ready
(EngineCore_DP0 pid=217089)     raise e from None
(EngineCore_DP0 pid=217089) Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
(APIServer pid=216690) Traceback (most recent call last):
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/runpy.py", line 196, in _run_module_as_main
(APIServer pid=216690)     return _run_code(code, main_globals, None,
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/runpy.py", line 86, in _run_code
(APIServer pid=216690)     exec(code, run_globals)
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 2011, in <module>
(APIServer pid=216690)     uvloop.run(run_server(args))
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/uvloop/__init__.py", line 82, in run
(APIServer pid=216690)     return loop.run_until_complete(wrapper())
(APIServer pid=216690)   File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/uvloop/__init__.py", line 61, in wrapper
(APIServer pid=216690)     return await main
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 1941, in run_server
(APIServer pid=216690)     await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 1961, in run_server_worker
(APIServer pid=216690)     async with build_async_engine_client(
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/contextlib.py", line 199, in __aenter__
(APIServer pid=216690)     return await anext(self.gen)
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 179, in build_async_engine_client
(APIServer pid=216690)     async with build_async_engine_client_from_engine_args(
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/contextlib.py", line 199, in __aenter__
(APIServer pid=216690)     return await anext(self.gen)
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 221, in build_async_engine_client_from_engine_args
(APIServer pid=216690)     async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/utils/__init__.py", line 1589, in inner
(APIServer pid=216690)     return fn(*args, **kwargs)
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/async_llm.py", line 212, in from_vllm_config
(APIServer pid=216690)     return cls(
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/async_llm.py", line 136, in __init__
(APIServer pid=216690)     self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/core_client.py", line 102, in make_async_mp_client
(APIServer pid=216690)     return AsyncMPClient(*client_args)
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/core_client.py", line 769, in __init__
(APIServer pid=216690)     super().__init__(
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/core_client.py", line 448, in __init__
(APIServer pid=216690)     with launch_core_engines(vllm_config, executor_class,
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/contextlib.py", line 142, in __exit__
(APIServer pid=216690)     next(self.gen)
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/utils.py", line 729, in launch_core_engines
(APIServer pid=216690)     wait_for_engine_startup(
(APIServer pid=216690)   File "/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/site-packages/vllm/v1/engine/utils.py", line 782, in wait_for_engine_startup
(APIServer pid=216690)     raise RuntimeError("Engine core initialization failed. "
(APIServer pid=216690) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
/home/jovyan/zhanlun-cipan/anaconda/miniconda/miniconda/envs/zhihu_0210_bak_0403/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There

[hmellor]

Good tip @pt-coder!

The weights that are in your checkpoint @zhanlun150729 that didn't get loaded are:

{
    'model.layers.0.mlp.gate.weight',
    'model.layers.0.mlp.experts.w13_weight',
    'model.layers.0.mlp.shared_expert_gate.weight',
    'model.layers.0.post_attention_layernorm.weight',
    'model.layers.0.mlp.shared_expert.down_proj.weight',
    'model.fc.weight',
    'model.layers.0.self_attn.qkv_proj.weight',
    'model.norm.weight',
    'model.layers.0.self_attn.k_norm.weight',
    'model.pre_fc_norm_hidden.weight',
    'model.layers.0.mlp.shared_expert.gate_up_proj.weight',
    'model.layers.0.self_attn.q_norm.weight',
    'model.pre_fc_norm_embedding.weight',
    'model.layers.0.mlp.experts.w2_weight',
    'model.layers.0.self_attn.o_proj.weight',
    'model.layers.0.input_layernorm.weight'
}

Some of these weights are only present in vLLM's Qwen3NextMTP/Qwen3NextMultiTokenPredictor class (pre_fc_norm_embedding for example).

This means that your model is being saved as Qwen3NextForCausalLM, but it contains some weights that are only used in Qwen3NextMTP in vLLM.

Which architecture do you see in the Resolved architecture: log?

[pt-coder]

Good tip @pt-coder!

The weights that are in your checkpoint @zhanlun150729 that didn't get loaded are:

{
'model.layers.0.mlp.gate.weight',
'model.layers.0.mlp.experts.w13_weight',
'model.layers.0.mlp.shared_expert_gate.weight',
'model.layers.0.post_attention_layernorm.weight',
'model.layers.0.mlp.shared_expert.down_proj.weight',
'model.fc.weight',
'model.layers.0.self_attn.qkv_proj.weight',
'model.norm.weight',
'model.layers.0.self_attn.k_norm.weight',
'model.pre_fc_norm_hidden.weight',
'model.layers.0.mlp.shared_expert.gate_up_proj.weight',
'model.layers.0.self_attn.q_norm.weight',
'model.pre_fc_norm_embedding.weight',
'model.layers.0.mlp.experts.w2_weight',
'model.layers.0.self_attn.o_proj.weight',
'model.layers.0.input_layernorm.weight'
}
Some of these weights are only present in vLLM's Qwen3NextMTP/Qwen3NextMultiTokenPredictor class (pre_fc_norm_embedding for example).

This means that your model is being saved as Qwen3NextForCausalLM, but it contains some weights that are only used in Qwen3NextMTP in vLLM.

Which architecture do you see in the Resolved architecture: log?

The resaved config.json file is as follows:
{
"architectures": [
"Qwen3NextForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 151643,
"decoder_sparse_step": 1,
"dtype": "bfloat16",
"eos_token_id": 151645,
"full_attention_interval": 4,
"head_dim": 256,
"hidden_act": "silu",
"hidden_size": 2048,
"initializer_range": 0.02,
"intermediate_size": 5120,
"layer_types": [
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention"
],
"linear_conv_kernel_dim": 4,
"linear_key_head_dim": 128,
"linear_num_key_heads": 16,
"linear_num_value_heads": 32,
"linear_value_head_dim": 128,
"max_position_embeddings": 262144,
"mlp_only_layers": [],
"model_type": "qwen3_next",
"moe_intermediate_size": 512,
"norm_topk_prob": true,
"num_attention_heads": 16,
"num_experts": 512,
"num_experts_per_tok": 10,
"num_hidden_layers": 48,
"num_key_value_heads": 2,
"output_router_logits": false,
"partial_rotary_factor": 0.25,
"rms_norm_eps": 1e-06,
"rope_scaling": null,
"rope_theta": 10000000,
"router_aux_loss_coef": 0.001,
"shared_expert_intermediate_size": 512,
"tie_word_embeddings": false,
"transformers_version": "4.57.0.dev0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 151936
}

[hmellor]

Ok, as suspected, the model contains mtp layers but is being saved as Qwen3NextForCausalLM.

I think you have 2 options:

• Save your model as Qwen3NextMultiTokenPredictor and make sure that the mtp gets saved with it
• Continue saving your model as Qwen3NextForCausalLM, but remove the layers which are only present in the mtp version of the model

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.