safetensors_rust.SafetensorError: framework paddle is invalid

[wencan]

🔎 Search before asking

• I have searched the PaddleOCR Docs and found no similar bug report.
• I have searched the PaddleOCR Issues and found no similar bug report.
• I have searched the PaddleOCR Discussions and found no similar bug report.

🐛 Bug (问题描述)

paddleocr doc_parser 报错

🏃‍♂️ Environment (运行环境)

Debian GNU/Linux 13 (trixie)
Python 3.13.5
GeForce GTX 1080 Ti
NVIDIA-SMI 550.163.01 Driver Version: 550.163.01 CUDA Version: 12.4
paddleocr 3.3.0
paddlepaddle-gpu 3.2.0
paddlex 3.3.3
paddlepaddle-gpu注册方法为： install paddlepaddle-gpu==3.2.0 -i https://www.paddlepaddle.org.cn/packages/stable/cu126/

🌰 Minimal Reproducible Example (最小可复现问题的Demo)

目标文件：2024ltr.pdf

执行: paddleocr doc_parser -i /home/wencan/Downloads/2024ltr.pdf
输出:

/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddle/utils/cpp_extension/extension_utils.py:718: UserWarning: No ccache found. Please be aware that recompiling all source files may be required. You can download and install ccache from: https://github.com/ccache/ccache/blob/master/doc/INSTALL.md
  warnings.warn(warning_message)
Creating model: ('PP-DocLayoutV2', None)
Using official model (PP-DocLayoutV2), the model files will be automatically downloaded and saved in `/home/wencan/.paddlex/official_models/PP-DocLayoutV2`.
<Response [404]>
Encounter exception when download model from aistudio: 
repo not found.
PaddleX would try to download from other model sources.
Using official model (PP-DocLayoutV2), the model files will be automatically downloaded and saved in `/home/wencan/.paddlex/official_models/PP-DocLayoutV2`.
2025-10-19 19:03:09,827 - modelscope - WARNING - Repo PaddlePaddle/PP-DocLayoutV2 not exists on https://www.modelscope.cn, will try on alternative endpoint https://www.modelscope.ai.
2025-10-19 19:03:10,491 - modelscope - ERROR - Repo PaddlePaddle/PP-DocLayoutV2 not exists on either https://www.modelscope.cn or https://www.modelscope.ai
Encounter exception when download model from modelscope: 
<Response [404]>.
PaddleX would try to download from other model sources.
Using official model (PP-DocLayoutV2), the model files will be automatically downloaded and saved in `/home/wencan/.paddlex/official_models/PP-DocLayoutV2`.
Connecting to https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayoutV2_infer.tar ...
Downloading PP-DocLayoutV2_infer.tar ...
[==================================================] 100.00%
Extracting PP-DocLayoutV2_infer.tar
[==================================================] 100.00%
Creating model: ('PaddleOCR-VL-0.9B', None)
Using official model (PaddleOCR-VL), the model files will be automatically downloaded and saved in `/home/wencan/.paddlex/official_models/PaddleOCR-VL`.
<Response [404]>
Encounter exception when download model from aistudio: 
repo not found.
PaddleX would try to download from other model sources.
Using official model (PaddleOCR-VL), the model files will be automatically downloaded and saved in `/home/wencan/.paddlex/official_models/PaddleOCR-VL`.
Downloading Model from https://www.modelscope.cn to directory: /tmp/tmp7jmpntkd/temp_dir
2025-10-19 19:03:53,887 - modelscope - INFO - Got 42 files, start to download ...
Downloading [PaddleOCR-VL-0.9B/.gitkeep]: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.00/1.00 [00:00<00:00, 550B/s]
Downloading [PaddleOCR-VL-0.9B/configuration_paddleocr_vl.py]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7.91k/7.91k [00:00<00:00, 14.0kB/s]
Downloading [PaddleOCR-VL-0.9B/config.json]: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.97k/1.97k [00:00<00:00, 3.26kB/s]
Downloading [PaddleOCR-VL-0.9B/chat_template.jinja]: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 799/799 [00:00<00:00, 1.23kB/s]
Downloading [PaddleOCR-VL-0.9B/added_tokens.json]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 24.8k/24.8k [00:00<00:00, 37.9kB/s]
Downloading [PP-DocLayoutV2/config.json]: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4.43k/4.43k [00:00<00:00, 6.59kB/s]
Downloading [imgs/allmetric.png]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 466k/466k [00:00<00:00, 571kB/s]
Downloading [imgs/chart_02.jpg]: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 727k/727k [00:00<00:00, 764kB/s]
Downloading [imgs/chart_01.jpg]: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 703k/703k [00:01<00:00, 701kB/s]
Downloading [PaddleOCR-VL-0.9B/image_processing.py]: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 24.4k/24.4k [00:00<00:00, 33.1kB/s]
Downloading [PaddleOCR-VL-0.9B/generation_config.json]: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 112/112 [00:00<00:00, 124B/s]
Downloading [PP-DocLayoutV2/inference.yml]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.45k/1.45k [00:00<00:00, 1.90kB/s]
Downloading [PaddleOCR-VL-0.9B/inference.yml]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 39.0/39.0 [00:00<00:00, 57.2B/s]
Downloading [imgs/formula_EN.jpg]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 561k/561k [00:01<00:00, 493kB/s]
Downloading [PP-DocLayoutV2/inference.pdmodel]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.44M/1.44M [00:01<00:00, 1.39MB/s]
Downloading [imgs/inhouse-formula.png]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 75.4k/75.4k [00:00<00:00, 93.8kB/s]
Downloading [imgs/inhousechart.png]: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 83.6k/83.6k [00:01<00:00, 82.7kB/s]
Downloading [imgs/inhousetable.png]: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 125k/125k [00:01<00:00, 113kB/s]
Downloading [imgs/inhouseocr.png]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 238k/238k [00:01<00:00, 181kB/s]
Downloading [PaddleOCR-VL-0.9B/modeling_paddleocr_vl.py]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 110k/110k [00:01<00:00, 97.4kB/s]
Downloading [imgs/omni10.png]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 595k/595k [00:01<00:00, 385kB/s]
Downloading [imgs/omni15.png]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 228k/228k [00:01<00:00, 152kB/s]
Downloading [imgs/omnibenchocr.png]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 101k/101k [00:01<00:00, 63.4kB/s]
Downloading [imgs/overview1.jpg]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 834k/834k [00:02<00:00, 398kB/s]
Downloading [imgs/overview2.jpg]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 620k/620k [00:02<00:00, 293kB/s]
Downloading [imgs/paddleocrvl.png]: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 284k/284k [00:02<00:00, 124kB/s]
Downloading [imgs/overview3.jpg]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 836k/836k [00:02<00:00, 296kB/s]
Downloading [PaddleOCR-VL-0.9B/preprocessor_config.json]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 687/687 [00:01<00:00, 391B/s]
Downloading [imgs/overview4.jpg]: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.27M/1.27M [00:02<00:00, 480kB/s]
Downloading [PaddleOCR-VL-0.9B/processing_paddleocr_vl.py]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12.0k/12.0k [00:01<00:00, 7.80kB/s]
Downloading [PaddleOCR-VL-0.9B/processor_config.json]: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 137/137 [00:01<00:00, 83.8B/s]
Downloading [README.md]: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 19.5k/19.5k [00:01<00:00, 15.9kB/s]
Downloading [PaddleOCR-VL-0.9B/special_tokens_map.json]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.12k/1.12k [00:01<00:00, 648B/s]
Downloading [imgs/table_02.jpg]: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 635k/635k [00:02<00:00, 253kB/s]
Downloading [imgs/table_01.jpg]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.37M/1.37M [00:03<00:00, 443kB/s]
Downloading [imgs/text_handwriting_02.jpg]: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 939k/939k [00:02<00:00, 331kB/s]
Downloading [imgs/text_english_arabic.jpg]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.01M/1.01M [00:03<00:00, 319kB/s]
Downloading [PaddleOCR-VL-0.9B/tokenizer.model]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.54M/1.54M [00:03<00:00, 467kB/s]
Downloading [PaddleOCR-VL-0.9B/tokenizer_config.json]: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 181k/181k [00:02<00:00, 74.4kB/s]
Downloading [PaddleOCR-VL-0.9B/tokenizer.json]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10.7M/10.7M [00:08<00:00, 1.37MB/s]
Downloading [PP-DocLayoutV2/inference.pdiparams]: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 202M/202M [01:08<00:00, 3.12MB/s]
Downloading [PaddleOCR-VL-0.9B/model.safetensors]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.79G/1.79G [05:16<00:00, 6.06MB/s]
Processing 42 items: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42.0/42.0 [05:17<00:00, 7.57s/it]
2025-10-19 19:09:11,809 - modelscope - INFO - Download model 'PaddlePaddle/PaddleOCR-VL' successfully.                                                                                            | 26.0M/1.79G [00:13<08:54, 3.53MB/s]
Loading configuration file /home/wencan/.paddlex/official_models/PaddleOCR-VL/PaddleOCR-VL-0.9B/config.json                                                                                        | 227M/1.79G [01:07<06:03, 4.62MB/s]
Loading weights file /home/wencan/.paddlex/official_models/PaddleOCR-VL/PaddleOCR-VL-0.9B/model.safetensors██████████████████████████████████████████████████████████████████████████████████████▉| 1.79G/1.79G [05:16<00:00, 5.76MB/s]
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddle/utils/decorator_utils.py:420: Warning: 
Non compatible API. Please refer to https://www.paddlepaddle.org.cn/documentation/docs/en/develop/guides/model_convert/convert_from_pytorch/api_difference/torch/torch.split.html first.
  warnings.warn(
Traceback (most recent call last):
  File "/home/wencan/Projects/venv/paddleocr/./bin/paddleocr", line 8, in <module>
    sys.exit(console_entry())
             ~~~~~~~~~~~~~^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/__main__.py", line 26, in console_entry
    main()
    ~~~~^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_cli.py", line 192, in main
    _execute(args)
    ~~~~~~~~^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_cli.py", line 181, in _execute
    args.executor(args)
    ~~~~~~~~~~~~~^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_pipelines/paddleocr_vl.py", line 363, in execute_with_args
    perform_simple_inference(
    ~~~~~~~~~~~~~~~~~~~~~~~~^
        PaddleOCRVL,
        ^^^^^^^^^^^^
    ...<9 lines>...
        },
        ^^
    )
    ^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_utils/cli.py", line 62, in perform_simple_inference
    wrapper = wrapper_cls(**init_params)
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_pipelines/paddleocr_vl.py", line 63, in __init__
    super().__init__(**kwargs)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_pipelines/base.py", line 67, in __init__
    self.paddlex_pipeline = self._create_paddlex_pipeline()
                            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_pipelines/base.py", line 105, in _create_paddlex_pipeline
    return create_pipeline(config=self._merged_paddlex_config, **kwargs)
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/pipelines/__init__.py", line 167, in create_pipeline
    pipeline = BasePipeline.get(pipeline_name)(
        config=config,
    ...<5 lines>...
        **kwargs,
    )
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/utils/deps.py", line 206, in _wrapper
    return old_init_func(self, *args, **kwargs)
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/pipelines/_parallel.py", line 103, in __init__
    self._pipeline = self._create_internal_pipeline(config, self.device)
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/pipelines/_parallel.py", line 158, in _create_internal_pipeline
    return self._pipeline_cls(
           ~~~~~~~~~~~~~~~~~~^
        config,
        ^^^^^^^
    ...<3 lines>...
        hpi_config=self.hpi_config,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/pipelines/paddleocr_vl/pipeline.py", line 127, in __init__
    self.vl_rec_model = self.create_model(vl_rec_config)
                        ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/pipelines/base.py", line 106, in create_model
    model = create_predictor(
        model_name=config["model_name"],
    ...<7 lines>...
        **kwargs,
    )
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/models/__init__.py", line 87, in create_predictor
    return BasePredictor.get(model_name)(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        model_dir=model_dir,
        ^^^^^^^^^^^^^^^^^^^^
    ...<8 lines>...
        **kwargs,
        ^^^^^^^^^
    )
    ^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/models/doc_vlm/predictor.py", line 66, in __init__
    self.infer, self.processor = self._build(**kwargs)
                                 ~~~~~~~~~~~^^^^^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/models/doc_vlm/predictor.py", line 152, in _build
    model = PaddleOCRVLForConditionalGeneration.from_pretrained(
        self.model_dir,
        dtype=self.dtype,
        convert_from_hf=True,
    )
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/models/common/vlm/transformers/model_utils.py", line 1937, in from_pretrained
    state_dict = load_state_dict(
        resolved_archive_file,
        convert_from_hf=convert_from_hf,
        transpose_weight_keys=transpose_weight_keys,
    )
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/models/common/vlm/transformers/model_utils.py", line 254, in load_state_dict
    with safe_open(checkpoint_file, framework="paddle") as f:
         ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
safetensors_rust.SafetensorError: framework paddle is invalid

[wencan]

@LiJianmin6706 活人？？如果是生成的，请加标注，减少不必要的污染

[Bobholamovic]

目前需要使用特殊版本的safetensors加载模型，请参考文档中的说明安装～

[muhammadyaseen]

目前需要使用特殊版本的safetensors加载模型，请参考文档中的说明安装～

To save others the translation, the above comment says:

Currently, you need to use a special version of safetensors to load the model. Please refer to the instructions in the document to install it.

Sine i'm using uv, I ran:

uv pip install https://paddle-whl.bj.bcebos.com/nightly/cu126/safetensors/safetensors-0.6.2.dev0-cp38-abi3-linux_x86_64.whl

but got the following error:

Resolved 1 package in 3.08s
error: Failed to determine installation plan
  Caused by: A URL dependency is incompatible with the current platform: https://paddle-whl.bj.bcebos.com/nightly/cu126/safetensors/safetensors-0.6.2.dev0-cp38-abi3-linux_x86_64.whl

I tried to look for MacOS version of the dependency, but so far no luck.

[Bobholamovic]

@muhammadyaseen MacOS is currently not supported. Please run the code on Linux with GPU support.

[SigureMo]

@muhammadyaseen You can try installing the latest safetensors directly from source — it should work as far as I can tell. Just make sure you have Rust installed on your machine so it can be built.

❯ uv pip install 'git+https://github.com/huggingface/safetensors.git#subdirectory=bindings/python' 
Resolved 1 package in 7.73s
      Built safetensors @ git+https://github.com/huggingface/safetensors.git@fa22263979d4964f8be4d34a4bfb0bd2bd04ddba#subdirectory=bindings/python
Prepared 1 package in 9.55s
Installed 1 package in 0.89ms
 + safetensors==0.6.2.dev0 (from git+https://github.com/huggingface/safetensors.git@fa22263979d4964f8be4d34a4bfb0bd2bd04ddba#subdirectory=bindings/python)

[wencan]

@SigureMo
install 'git+https://github.com/huggingface/safetensors.git#subdirectory=bindings/python'之后
报错是：

/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddle/utils/cpp_extension/extension_utils.py:718: UserWarning: No ccache found. Please be aware that recompiling all source files may be required. You can download and install ccache from: https://github.com/ccache/ccache/blob/master/doc/INSTALL.md
  warnings.warn(warning_message)
Creating model: ('PP-DocLayoutV2', None)
Model files already exist. Using cached files. To redownload, please delete the directory manually: `/home/wencan/.paddlex/official_models/PP-DocLayoutV2`.
Creating model: ('PaddleOCR-VL-0.9B', None)
Model files already exist. Using cached files. To redownload, please delete the directory manually: `/home/wencan/.paddlex/official_models/PaddleOCR-VL`.
Loading configuration file /home/wencan/.paddlex/official_models/PaddleOCR-VL/PaddleOCR-VL-0.9B/config.json
Loading weights file /home/wencan/.paddlex/official_models/PaddleOCR-VL/PaddleOCR-VL-0.9B/model.safetensors
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
use GQA - num_heads: 16- num_key_value_heads: 2
/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddle/utils/decorator_utils.py:420: Warning: 
Non compatible API. Please refer to https://www.paddlepaddle.org.cn/documentation/docs/en/develop/guides/model_convert/convert_from_pytorch/api_difference/torch/torch.split.html first.
  warnings.warn(
Loaded weights file from disk, setting weights to model.
Traceback (most recent call last):
  File "/home/wencan/Projects/venv/paddleocr/./bin/paddleocr", line 8, in <module>
    sys.exit(console_entry())
             ~~~~~~~~~~~~~^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/__main__.py", line 26, in console_entry
    main()
    ~~~~^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_cli.py", line 192, in main
    _execute(args)
    ~~~~~~~~^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_cli.py", line 181, in _execute
    args.executor(args)
    ~~~~~~~~~~~~~^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_pipelines/paddleocr_vl.py", line 363, in execute_with_args
    perform_simple_inference(
    ~~~~~~~~~~~~~~~~~~~~~~~~^
        PaddleOCRVL,
        ^^^^^^^^^^^^
    ...<9 lines>...
        },
        ^^
    )
    ^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_utils/cli.py", line 62, in perform_simple_inference
    wrapper = wrapper_cls(**init_params)
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_pipelines/paddleocr_vl.py", line 63, in __init__
    super().__init__(**kwargs)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_pipelines/base.py", line 67, in __init__
    self.paddlex_pipeline = self._create_paddlex_pipeline()
                            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddleocr/_pipelines/base.py", line 105, in _create_paddlex_pipeline
    return create_pipeline(config=self._merged_paddlex_config, **kwargs)
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/pipelines/__init__.py", line 167, in create_pipeline
    pipeline = BasePipeline.get(pipeline_name)(
        config=config,
    ...<5 lines>...
        **kwargs,
    )
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/utils/deps.py", line 206, in _wrapper
    return old_init_func(self, *args, **kwargs)
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/pipelines/_parallel.py", line 103, in __init__
    self._pipeline = self._create_internal_pipeline(config, self.device)
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/pipelines/_parallel.py", line 158, in _create_internal_pipeline
    return self._pipeline_cls(
           ~~~~~~~~~~~~~~~~~~^
        config,
        ^^^^^^^
    ...<3 lines>...
        hpi_config=self.hpi_config,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/pipelines/paddleocr_vl/pipeline.py", line 127, in __init__
    self.vl_rec_model = self.create_model(vl_rec_config)
                        ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/pipelines/base.py", line 106, in create_model
    model = create_predictor(
        model_name=config["model_name"],
    ...<7 lines>...
        **kwargs,
    )
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/models/__init__.py", line 87, in create_predictor
    return BasePredictor.get(model_name)(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        model_dir=model_dir,
        ^^^^^^^^^^^^^^^^^^^^
    ...<8 lines>...
        **kwargs,
        ^^^^^^^^^
    )
    ^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/models/doc_vlm/predictor.py", line 66, in __init__
    self.infer, self.processor = self._build(**kwargs)
                                 ~~~~~~~~~~~^^^^^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/models/doc_vlm/predictor.py", line 152, in _build
    model = PaddleOCRVLForConditionalGeneration.from_pretrained(
        self.model_dir,
        dtype=self.dtype,
        convert_from_hf=True,
    )
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/models/common/vlm/transformers/model_utils.py", line 1986, in from_pretrained
    cls._load_pretrained_model(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        model=model,
        ^^^^^^^^^^^^
    ...<13 lines>...
        convert_from_hf=convert_from_hf,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/models/common/vlm/transformers/model_utils.py", line 1543, in _load_pretrained_model
    error_msgs = _load_state_dict_into_model(
        model_to_load,
    ...<2 lines>...
        convert_from_hf=convert_from_hf,
    )
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/models/common/vlm/transformers/model_utils.py", line 299, in _load_state_dict_into_model
    model_to_load.set_hf_state_dict(state_dict)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddlex/inference/models/doc_vlm/modeling/paddleocr_vl/_paddleocr_vl.py", line 626, in set_hf_state_dict
    return self.set_state_dict(state_dict, *args, **kwargs)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddle/base/framework.py", line 794, in wrapper
    return func(*args, **kwargs)
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddle/nn/layer/layers.py", line 2280, in set_state_dict
    param.set_value(state)
    ~~~~~~~~~~~~~~~^^^^^^^
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddle/base/wrapped_decorator.py", line 50, in wrapper
    return decorated(*args, **kwargs)
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddle/base/framework.py", line 722, in __impl__
    return func(*args, **kwargs)
  File "/home/wencan/Projects/venv/paddleocr/lib/python3.13/site-packages/paddle/base/dygraph/tensor_patch_methods.py", line 237, in set_value
    assert self.dtype == dtype, (
           ^^^^^^^^^^^^^^^^^^^
AssertionError: Variable dtype not match, Variable [ linear_164.w_0 ] need tensor with dtype paddle.float32  but load tensor with dtype paddle.bfloat16

[Bobholamovic]

@wencan 这是因为当前设备不支持bf16，模型暂时不可用。我们目前正在适配中，预计本周内完成。

[hzy]

目前需要使用特殊版本的safetensors加载模型，请参考文档中的说明安装～

@Bobholamovic 你好，这个 https://paddle-whl.bj.bcebos.com/nightly/cu126/safetensors/safetensors-0.6.2.dev0-cp38-abi3-linux_x86_64.whl 可以提供一下windows平台（win_amd64）的whl吗？

[Bobholamovic]

目前需要使用特殊版本的safetensors加载模型，请参考文档中的说明安装～

@Bobholamovic 你好，这个 https://paddle-whl.bj.bcebos.com/nightly/cu126/safetensors/safetensors-0.6.2.dev0-cp38-abi3-linux_x86_64.whl 可以提供一下windows平台（win_amd64）的whl吗？

我们正在测试中，将在近期提供。

[BenjiKCF]

目前需要使用特殊版本的safetensors加载模型，请参考文档中的说明安装～

@Bobholamovic 你好，这个 https://paddle-whl.bj.bcebos.com/nightly/cu126/safetensors/safetensors-0.6.2.dev0-cp38-abi3-linux_x86_64.whl 可以提供一下windows平台（win_amd64）的whl吗？

我们正在测试中，将在近期提供。
請問將來cpu能做inference嗎? 謝謝

[Bobholamovic]

目前需要使用特殊版本的safetensors加载模型，请参考文档中的说明安装～

@Bobholamovic 你好，这个 https://paddle-whl.bj.bcebos.com/nightly/cu126/safetensors/safetensors-0.6.2.dev0-cp38-abi3-linux_x86_64.whl 可以提供一下windows平台（win_amd64）的whl吗？

我们正在测试中，将在近期提供。
請問將來cpu能做inference嗎? 謝謝

目前已经可以了，可以升级最新版本的PaddleX试试：pip install -U paddlex

[wencan]

目前需要使用特殊版本的safetensors加载模型，请参考文档中的说明安装～

@Bobholamovic 你好，这个 https://paddle-whl.bj.bcebos.com/nightly/cu126/safetensors/safetensors-0.6.2.dev0-cp38-abi3-linux_x86_64.whl 可以提供一下windows平台（win_amd64）的whl吗？

我们正在测试中，将在近期提供。
請問將來cpu能做inference嗎? 謝謝

目前已经可以了，可以升级最新版本的PaddleX试试：pip install -U paddlex

framework paddle is invalid 的问题解决了
但是RuntimeError了