PaddleOCR-VL inference error

[yzoaim]

🔎 Search before asking

• I have searched the PaddleOCR Docs and found no similar bug report.
• I have searched the PaddleOCR Issues and found no similar bug report.
• I have searched the PaddleOCR Discussions and found no similar bug report.

🐛 Bug (问题描述)

pipeline = PaddleOCRVL(
use_layout_detection=False,
vl_rec_model_dir="/home/user/.paddlex/official_models/PaddleOCR-VL"
)
output = pipeline.predict("/home/user/data/0651ab7e72135245af5332a665ca323.jpg")
print(output)
for res in output:
    res.print()
    res.save_to_json(save_path="output")
    res.save_to_markdown(save_path="output")

report:

raise RuntimeError(
RuntimeError: Exception from the 'vlm' worker: Image features and image tokens do not match: tokens: 0, features 1440

🏃‍♂️ Environment (运行环境)

accelerate==1.11.0
aiohappyeyeballs==2.6.1
aiohttp==3.13.2
aiosignal==1.4.0
aistudio-sdk==0.3.8
annotated-doc==0.0.3
annotated-types==0.7.0
anyio==4.11.0
astor==0.8.1
async-timeout==5.0.1
attrs==25.4.0
av==16.0.1
bce-python-sdk==0.9.46
blake3==1.0.8
boto3==1.40.41
botocore==1.40.41
cachetools==6.2.1
cbor2==5.7.1
certifi==2025.10.5
cffi==2.0.0
chardet==5.2.0
charset-normalizer==3.4.4
click==8.2.1
cloudpickle==3.1.1
colorlog==6.10.1
compressed-tensors==0.11.0
cssselect==1.3.0
cssutils==2.11.1
cupy-cuda12x==13.6.0
decorator==5.2.1
depyf==0.19.0
dill==0.4.0
diskcache==5.6.3
distro==1.9.0
dnspython==2.8.0
einops==0.8.1
email-validator==2.3.0
et_xmlfile==2.0.0
exceptiongroup==1.3.0
fastapi==0.120.4
fastapi-cli==0.0.14
fastapi-cloud-cli==0.3.1
fastrlock==0.8.3
filelock==3.20.0
flash_attn @ file:///home/user/flash_attn-2.8.3%2Bcu12torch2.8cxx11abiFALSE-cp310-cp310-linux_x86_64.whl#sha256=7aa420a0f6a7fb0a75d3d4f62f7e3a41b51dba7f1fee0defaed051c5465ccb32
frozendict==2.4.6
frozenlist==1.8.0
fsspec==2025.10.0
ftfy==6.3.1
future==1.0.0
gguf==0.17.1
h11==0.16.0
hf-xet==1.2.0
httpcore==1.0.9
httptools==0.7.1
httpx==0.28.1
huggingface-hub==0.36.0
idna==3.11
imagesize==1.4.1
interegular==0.3.3
Jinja2==3.1.6
jiter==0.11.1
jmespath==1.0.1
joblib==1.5.2
jsonschema==4.25.1
jsonschema-specifications==2025.9.1
lark==1.2.2
llguidance==0.7.30
llvmlite==0.44.0
lm-format-enforcer==0.11.3
loguru==0.7.3
lxml==6.0.2
markdown-it-py==4.0.0
MarkupSafe==3.0.3
mdurl==0.1.2
mistral_common==1.8.5
modelscope==1.31.0
more-itertools==10.8.0
mpmath==1.3.0
msgpack==1.1.2
msgspec==0.19.0
multidict==6.7.0
networkx==3.4.2
ninja==1.13.0
numba==0.61.2
numpy==2.2.6
nvidia-cublas-cu12==12.8.4.1
nvidia-cuda-cupti-cu12==12.8.90
nvidia-cuda-nvrtc-cu12==12.8.93
nvidia-cuda-runtime-cu12==12.8.90
nvidia-cudnn-cu12==9.10.2.21
nvidia-cufft-cu12==11.3.3.83
nvidia-cufile-cu12==1.13.1.3
nvidia-curand-cu12==10.3.9.90
nvidia-cusolver-cu12==11.7.3.90
nvidia-cusparse-cu12==12.5.8.93
nvidia-cusparselt-cu12==0.7.1
nvidia-nccl-cu12==2.27.3
nvidia-nvjitlink-cu12==12.8.93
nvidia-nvshmem-cu12==3.3.20
nvidia-nvtx-cu12==12.8.90
openai==2.6.1
openai-harmony==0.0.4
opencv-contrib-python==4.10.0.84
opencv-python-headless==4.12.0.88
openpyxl==3.1.5
opt-einsum==3.3.0
outlines_core==0.2.11
packaging==25.0
paddleocr==3.3.0
paddlepaddle==3.1.1
paddlex==3.3.5
pandas==2.3.3
partial-json-parser==0.2.1.1.post6
pillow==12.0.0
premailer==3.10.0
prettytable==3.16.0
prometheus-fastapi-instrumentator==7.1.0
prometheus_client==0.23.1
propcache==0.4.1
protobuf==6.33.0
psutil==7.1.2
py-cpuinfo==9.0.0
pybase64==1.4.2
pyclipper==1.3.0.post6
pycountry==24.6.1
pycparser==2.23
pycryptodome==3.23.0
pydantic==2.12.3
pydantic-extra-types==2.10.6
pydantic_core==2.41.4
Pygments==2.19.2
pypdfium2==5.0.0
python-bidi==0.6.7
python-dateutil==2.9.0.post0
python-dotenv==1.2.1
python-json-logger==4.0.0
python-multipart==0.0.20
pytz==2025.2
PyYAML==6.0.3
pyzmq==27.1.0
qwen-vl-utils==0.0.14
ray==2.51.1
referencing==0.37.0
regex==2025.10.23
requests==2.32.5
rich==14.2.0
rich-toolkit==0.15.1
rignore==0.7.3
rpds-py==0.28.0
ruamel.yaml==0.18.16
ruamel.yaml.clib==0.2.14
s3transfer==0.14.0
safetensors @ https://paddle-whl.bj.bcebos.com/nightly/cu126/safetensors/safetensors-0.6.2.dev0-cp38-abi3-linux_x86_64.whl#sha256=c38867a8551517dd1f0a8c9336c99906dcb5f6b62db4f9d90bb31cfc626505d9
scikit-learn==1.7.2
scipy==1.15.3
sentencepiece==0.2.1
sentry-sdk==2.43.0
setproctitle==1.3.7
shapely==2.1.2
shellingham==1.5.4
six==1.17.0
sniffio==1.3.1
soundfile==0.13.1
soxr==1.0.0
starlette==0.49.3
sympy==1.14.0
threadpoolctl==3.6.0
tiktoken==0.12.0
tokenizers==0.22.1
torch==2.8.0
torchaudio==2.8.0
torchvision==0.23.0
tqdm==4.67.1
transformers==4.57.1
triton==3.4.0
typer==0.20.0
typing-inspection==0.4.2
typing_extensions==4.15.0
tzdata==2025.2
ujson==5.11.0
urllib3==2.5.0
uvicorn==0.38.0
uvloop==0.22.1
vllm==0.11.0
watchfiles==1.1.1
wcwidth==0.2.14
websockets==15.0.1
xformers==0.0.32.post1
xgrammar==0.1.25
yarl==1.22.0

Distributor ID: Ubuntu
Description: Ubuntu 22.04 LTS
GPU:NVIDIA H200

🌰 Minimal Reproducible Example (最小可复现问题的Demo)

pipeline = PaddleOCRVL(
use_layout_detection=False,
vl_rec_model_dir="/home/user/.paddlex/official_models/PaddleOCR-VL"
)
output = pipeline.predict("/home/user/data/0651ab7e72135245af5332a665ca323.jpg")
print(output)
for res in output:
    res.print()
    res.save_to_json(save_path="output")
    res.save_to_markdown(save_path="output")

under reported environment

[wangjiezhe]

我之前遇到了同样的问题，升级到最新的 paddlex==3.3.6 就可以解决了。

[TingquanGao]

@yzoaim This error has been fixed. Please upgrade paddlex to version v3.3.6.
I am going to close this issue now. If you have any further questions or encounter any other problems, feel free to reopen this issue or create a new one.

Thank you for your feedback!

[kim0806]

你好，为什么我的版本是paddlex==3.3.6，且vllm已经成功部署还是会报上述错误，

config：

pipeline_name: PaddleOCR-VL

batch_size: 64

use_queues: True

use_doc_preprocessor: False
use_layout_detection: True
use_chart_recognition: False
format_block_content: False

SubModules:
LayoutDetection:
module_name: layout_detection
model_name: PP-DocLayoutV2
model_dir: /mnt/afs/yangziteng/.paddlex/official_models/PP-DocLayoutV2
batch_size: 8
threshold:
0: 0.5 # abstract
1: 0.5 # algorithm
2: 0.5 # aside_text
3: 0.5 # chart
4: 0.5 # content
5: 0.4 # formula
6: 0.4 # doc_title
7: 0.5 # figure_title
8: 0.5 # footer
9: 0.5 # footer
10: 0.5 # footnote
11: 0.5 # formula_number
12: 0.5 # header
13: 0.5 # header
14: 0.5 # image
15: 0.4 # formula
16: 0.5 # number
17: 0.4 # paragraph_title
18: 0.5 # reference
19: 0.5 # reference_content
20: 0.45 # seal
21: 0.5 # table
22: 0.4 # text
23: 0.4 # text
24: 0.5 # vision_footnote
layout_nms: True
layout_unclip_ratio: [1.0, 1.0]
layout_merge_bboxes_mode:
0: "union" # abstract
1: "union" # algorithm
2: "union" # aside_text
3: "large" # chart
4: "union" # content
5: "large" # display_formula
6: "large" # doc_title
7: "union" # figure_title
8: "union" # footer
9: "union" # footer
10: "union" # footnote
11: "union" # formula_number
12: "union" # header
13: "union" # header
14: "union" # image
15: "large" # inline_formula
16: "union" # number
17: "large" # paragraph_title
18: "union" # reference
19: "union" # reference_content
20: "union" # seal
21: "union" # table
22: "union" # text
23: "union" # text
24: "union" # vision_footnote
VLRecognition:
module_name: vl_recognition
model_name: PaddleOCR-VL-0.9B
model_dir: /mnt/afs/yangziteng/.paddlex/official_models/PaddleOCR-VL
batch_size: 2048
genai_config:
backend: vllm-server
server_url: http://localhost:8118/v1/
max_concurrency: 128

SubPipelines:
DocPreprocessor:
pipeline_name: doc_preprocessor
batch_size: 8
use_doc_orientation_classify: True
use_doc_unwarping: True
SubModules:
DocOrientationClassify:
module_name: doc_text_orientation
model_name: PP-LCNet_x1_0_doc_ori
model_dir: null
batch_size: 8
DocUnwarping:
module_name: image_unwarping
model_name: UVDoc
model_dir: null

运行指令：
from paddlex import create_pipeline
pipeline = create_pipeline("/mnt/afs/yangziteng/project/Tabletool/TableToolRL/config/PaddleOCR-VL.yaml")

for res in pipeline.predict("/mnt/afs/yangziteng/project/Tabletool/ToTTo_train_table_10017.jpg"):
res.print()

报错信息：
Creating model: ('PP-DocLayoutV2', '/mnt/afs/yangziteng/.paddlex/official_models/PP-DocLayoutV2')
/mnt/afs/yangziteng/.conda/envs/ppocr/lib/python3.11/site-packages/paddle/utils/cpp_extension/extension_utils.py:718: UserWarning: No ccache found. Please be aware that recompiling all source files may be required. You can download and install ccache from: https://github.com/ccache/ccache/blob/master/doc/INSTALL.md
warnings.warn(warning_message)
Creating model: ('PaddleOCR-VL-0.9B', '/mnt/afs/yangziteng/.paddlex/official_models/PaddleOCR-VL')
/mnt/afs/yangziteng/.conda/envs/ppocr/lib/python3.11/site-packages/paddlex/inference/models/base/predictor/base_predictor.py:131: UserWarning: model_dir will be ignored, as it is not needed.
warnings.warn("model_dir will be ignored, as it is not needed.")
Traceback (most recent call last):
File "/mnt/afs/yangziteng/project/Tabletool/process/anays/tool/ocr.py", line 20, in
for res in pipeline.predict("/mnt/afs/yangziteng/project/Tabletool/ToTTo_train_table_10017.jpg"):
File "/mnt/afs/yangziteng/.conda/envs/ppocr/lib/python3.11/site-packages/paddlex/inference/pipelines/_parallel.py", line 129, in predict
yield from self._pipeline.predict(
File "/mnt/afs/yangziteng/.conda/envs/ppocr/lib/python3.11/site-packages/paddlex/inference/pipelines/paddleocr_vl/pipeline.py", line 673, in predict
raise RuntimeError(
RuntimeError: Exception from the 'vlm' worker: Connection error.

希望能得到您的回复感谢