[Bug]: ValueError: There is no module or parameter named 'mlp_AR' in TransformersForCausalLM

[jiaohuix]

Your current environment

error when using PaddleOCR-VL

vllm serve PaddlePaddle/PaddleOCR-VL \
    --trust-remote-code \
    --max-num-batched-tokens 16384 \
    --no-enable-prefix-caching \
    --mm-processor-cache-gb 0

🐛 Describe the bug

(EngineCore_DP0 pid=560891) INFO 11-12 13:42:16 [transformers.py:442] Using Transformers backend.
(EngineCore_DP0 pid=560891) torch_dtype is deprecated! Use dtype instead!
(EngineCore_DP0 pid=560891) INFO 11-12 13:42:17 [cuda.py:366] Using Flash Attention backend on V1 engine.
Loading safetensors checkpoint shards: 0% Completed | 0/1 [00:00<?, ?it/s]
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] EngineCore failed to start.
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] Traceback (most recent call last):
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 699, in run_engine_core
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 498, in init
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] super().init(vllm_config, executor_class, log_stats,
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 83, in init
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/executor/executor_base.py", line 54, in init
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] self._init_executor()
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/executor/uniproc_executor.py", line 55, in _init_executor
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] self.collective_rpc("load_model")
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/executor/uniproc_executor.py", line 83, in collective_rpc
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] return [run_method(self.driver_worker, method, args, kwargs)]
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/utils/init.py", line 3122, in run_method
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] return func(*args, **kwargs)
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 213, in load_model
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] self.model_runner.load_model(eep_scale_up=eep_scale_up)
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_model_runner.py", line 2635, in load_model
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] self.model = model_loader.load_model(
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/model_executor/model_loader/base_loader.py", line 50, in load_model
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] self.load_weights(model, model_config)
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/model_executor/model_loader/default_loader.py", line 264, in load_weights
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] loaded_weights = model.load_weights(
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] ^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/model_executor/models/transformers.py", line 700, in load_weights
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] return loader.load_weights(weights, mapper=self.hf_to_vllm_mapper)
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 294, in load_weights
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] autoloaded_weights = set(self._load_module("", self.module, weights))
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 280, in _load_module
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] raise ValueError(msg)
(EngineCore_DP0 pid=560891) ERROR 11-12 13:42:18 [core.py:708] ValueError: There is no module or parameter named 'mlp_AR' in TransformersForCausalLM
(EngineCore_DP0 pid=560891) Process EngineCore_DP0:
(EngineCore_DP0 pid=560891) Traceback (most recent call last):
(EngineCore_DP0 pid=560891) File "/home/suser/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore_DP0 pid=560891) self.run()
(EngineCore_DP0 pid=560891) File "/home/suser/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/multiprocessing/process.py", line 108, in run
(EngineCore_DP0 pid=560891) self._target(*self._args, **self._kwargs)
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 712, in run_engine_core
(EngineCore_DP0 pid=560891) raise e
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 699, in run_engine_core
(EngineCore_DP0 pid=560891) engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=560891) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 498, in init
(EngineCore_DP0 pid=560891) super().init(vllm_config, executor_class, log_stats,
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 83, in init
(EngineCore_DP0 pid=560891) self.model_executor = executor_class(vllm_config)
(EngineCore_DP0 pid=560891) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/executor/executor_base.py", line 54, in init
(EngineCore_DP0 pid=560891) self._init_executor()
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/executor/uniproc_executor.py", line 55, in _init_executor
(EngineCore_DP0 pid=560891) self.collective_rpc("load_model")
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/executor/uniproc_executor.py", line 83, in collective_rpc
(EngineCore_DP0 pid=560891) return [run_method(self.driver_worker, method, args, kwargs)]
(EngineCore_DP0 pid=560891) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/utils/init.py", line 3122, in run_method
(EngineCore_DP0 pid=560891) return func(*args, **kwargs)
(EngineCore_DP0 pid=560891) ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 213, in load_model
(EngineCore_DP0 pid=560891) self.model_runner.load_model(eep_scale_up=eep_scale_up)
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_model_runner.py", line 2635, in load_model
(EngineCore_DP0 pid=560891) self.model = model_loader.load_model(
(EngineCore_DP0 pid=560891) ^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/model_executor/model_loader/base_loader.py", line 50, in load_model
(EngineCore_DP0 pid=560891) self.load_weights(model, model_config)
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/model_executor/model_loader/default_loader.py", line 264, in load_weights
(EngineCore_DP0 pid=560891) loaded_weights = model.load_weights(
(EngineCore_DP0 pid=560891) ^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/model_executor/models/transformers.py", line 700, in load_weights
(EngineCore_DP0 pid=560891) return loader.load_weights(weights, mapper=self.hf_to_vllm_mapper)
(EngineCore_DP0 pid=560891) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 294, in load_weights
(EngineCore_DP0 pid=560891) autoloaded_weights = set(self._load_module("", self.module, weights))
(EngineCore_DP0 pid=560891) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=560891) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 280, in _load_module
(EngineCore_DP0 pid=560891) raise ValueError(msg)
(EngineCore_DP0 pid=560891) ValueError: There is no module or parameter named 'mlp_AR' in TransformersForCausalLM
Loading safetensors checkpoint shards: 0% Completed | 0/1 [00:01<?, ?it/s]
(EngineCore_DP0 pid=560891)
[rank0]:[W1112 13:42:19.509718767 ProcessGroupNCCL.cpp:1538] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
(APIServer pid=560804) Traceback (most recent call last):
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/bin/vllm", line 10, in
(APIServer pid=560804) sys.exit(main())
(APIServer pid=560804) ^^^^^^
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/entrypoints/cli/main.py", line 54, in main
(APIServer pid=560804) args.dispatch_function(args)
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/entrypoints/cli/serve.py", line 57, in cmd
(APIServer pid=560804) uvloop.run(run_server(args))
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/uvloop/init.py", line 96, in run
(APIServer pid=560804) return __asyncio.run(
(APIServer pid=560804) ^^^^^^^^^^^^^^
(APIServer pid=560804) File "/home/suser/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/asyncio/runners.py", line 195, in run
(APIServer pid=560804) return runner.run(main)
(APIServer pid=560804) ^^^^^^^^^^^^^^^^
(APIServer pid=560804) File "/home/suser/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=560804) return self._loop.run_until_complete(task)
(APIServer pid=560804) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=560804) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/uvloop/init.py", line 48, in wrapper
(APIServer pid=560804) return await main
(APIServer pid=560804) ^^^^^^^^^^
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 1884, in run_server
(APIServer pid=560804) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 1902, in run_server_worker
(APIServer pid=560804) async with build_async_engine_client(
(APIServer pid=560804) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=560804) File "/home/suser/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 210, in aenter
(APIServer pid=560804) return await anext(self.gen)
(APIServer pid=560804) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 180, in build_async_engine_client
(APIServer pid=560804) async with build_async_engine_client_from_engine_args(
(APIServer pid=560804) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=560804) File "/home/suser/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 210, in aenter
(APIServer pid=560804) return await anext(self.gen)
(APIServer pid=560804) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 225, in build_async_engine_client_from_engine_args
(APIServer pid=560804) async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=560804) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/utils/init.py", line 1572, in inner
(APIServer pid=560804) return fn(*args, **kwargs)
(APIServer pid=560804) ^^^^^^^^^^^^^^^^^^^
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 207, in from_vllm_config
(APIServer pid=560804) return cls(
(APIServer pid=560804) ^^^^
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 134, in init
(APIServer pid=560804) self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=560804) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 102, in make_async_mp_client
(APIServer pid=560804) return AsyncMPClient(*client_args)
(APIServer pid=560804) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 769, in init
(APIServer pid=560804) super().init(
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 448, in init
(APIServer pid=560804) with launch_core_engines(vllm_config, executor_class,
(APIServer pid=560804) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=560804) File "/home/suser/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 144, in exit
(APIServer pid=560804) next(self.gen)
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/utils.py", line 732, in launch_core_engines
(APIServer pid=560804) wait_for_engine_startup(
(APIServer pid=560804) File "/mnt/sdb/zjh/ARPO/submit/ppocr/.venv/lib/python3.12/site-packages/vllm/v1/engine/utils.py", line 785, in wait_for_engine_startup
(APIServer pid=560804) raise RuntimeError("Engine core initialization failed. "
(APIServer pid=560804) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[Shi-jack]

the same question.

[hglee98]

The same question! 🙏

[zhanglichen666]

Me Too

[Isotr0py]

v0.11.1 which will include this model hasn't released yet. Please install the nightly wheel to try it 😅

uv pip install -U vllm \
    --torch-backend=auto \
    --extra-index-url https://wheels.vllm.ai/nightly

[hwang136]

I installed 0.11.1 but it did not solve the problem

[yujinzheng]

you can try to install vllm from source, I install it from source and solve this problem.
The version info is:
vllm 0.10.2+empty
vllm_ascend 0.10.2rc1
transformers 4.56.1

I modify 'vllm/model_executor/models/paddleocr_vl.py' and 'vllm/model_executor/models/registry.py' because vllm v0.10.2 not support paddleocr-vl

I think maybe you can try to install vllm v0.11.1 from source and do not need to modify 'paddleocr_vl.py'