当onnxruntime-gpu版本高于1.20时，无法使用cuda加速

[liaoyu-qing]

问题描述 / Problem Description

当onnxruntime-gpu版本高于1.20时，无法使用cuda加速

运行环境 / Runtime Environment

Ubuntu20.04 x86
Driver Version: 595.58.03
cuda: 13.2

日志：

(sjzserver) root@aiserver0:/home/Qing/sjzserver/models/ocr# pip install onnxruntime-gpu==1.19
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Collecting onnxruntime-gpu==1.19
  Downloading https://mirrors.ustc.edu.cn/pypi/packages/9d/6b/eef5816e1f51c75bd33fa875f9b97c831b5b0bd0fc1b3b2697d5a509e837/onnxruntime_gpu-1.19.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (223.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 223.1/223.1 MB 87.4 MB/s  0:00:02
Requirement already satisfied: coloredlogs in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.19) (15.0.1)
Requirement already satisfied: flatbuffers in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.19) (25.12.19)
Requirement already satisfied: numpy>=1.21.6 in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.19) (1.26.4)
Requirement already satisfied: packaging in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.19) (26.0)
Requirement already satisfied: protobuf in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.19) (7.35.0)
Requirement already satisfied: sympy in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.19) (1.14.0)
Requirement already satisfied: humanfriendly>=9.1 in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from coloredlogs->onnxruntime-gpu==1.19) (10.0)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from sympy->onnxruntime-gpu==1.19) (1.3.0)
Installing collected packages: onnxruntime-gpu
  Attempting uninstall: onnxruntime-gpu
    Found existing installation: onnxruntime-gpu 1.20.0
    Uninstalling onnxruntime-gpu-1.20.0:
      Successfully uninstalled onnxruntime-gpu-1.20.0
Successfully installed onnxruntime-gpu-1.19.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
(sjzserver) root@aiserver0:/home/Qing/sjzserver/models/ocr# 
(sjzserver) root@aiserver0:/home/Qing/sjzserver/models/ocr# 
(sjzserver) root@aiserver0:/home/Qing/sjzserver/models/ocr# python ocr_onnx.py 
[INFO] 2026-05-20 17:48:03,171 [RapidOCR] base.py:22: Using engine_name: onnxruntime
[INFO] 2026-05-20 17:48:03,207 [RapidOCR] download_file.py:60: File exists and is valid: /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_PP-OCRv4_det_mobile.onnx
[INFO] 2026-05-20 17:48:03,207 [RapidOCR] main.py:57: Using /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_PP-OCRv4_det_mobile.onnx
[INFO] 2026-05-20 17:48:03,387 [RapidOCR] base.py:22: Using engine_name: onnxruntime
[INFO] 2026-05-20 17:48:03,388 [RapidOCR] download_file.py:60: File exists and is valid: /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_ppocr_mobile_v2.0_cls_mobile.onnx
[INFO] 2026-05-20 17:48:03,388 [RapidOCR] main.py:57: Using /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_ppocr_mobile_v2.0_cls_mobile.onnx
[INFO] 2026-05-20 17:48:03,414 [RapidOCR] base.py:22: Using engine_name: onnxruntime
[INFO] 2026-05-20 17:48:03,419 [RapidOCR] download_file.py:60: File exists and is valid: /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_PP-OCRv4_rec_mobile.onnx
[INFO] 2026-05-20 17:48:03,420 [RapidOCR] main.py:57: Using /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_PP-OCRv4_rec_mobile.onnx

=== 开始OCR性能测试 ===
预热次数: 5, 测试次数: 20
正在预热...
开始正式测试...
第  10 轮耗时: 30.82 ms
第  20 轮耗时: 31.86 ms

=== 性能测试结果 ===
平均耗时: 31.11 ms
最小耗时: 30.10 ms
最大耗时: 32.12 ms
耗时标准差: 0.56 ms (越小越稳定)
理论QPS (每秒处理图片数): 32.14

=== 功能正确性验证 ===
识别文本数量: 0

✅ OCR性能测试完成！
(sjzserver) root@aiserver0:/home/Qing/sjzserver/models/ocr# 
(sjzserver) root@aiserver0:/home/Qing/sjzserver/models/ocr# 
(sjzserver) root@aiserver0:/home/Qing/sjzserver/models/ocr# 
(sjzserver) root@aiserver0:/home/Qing/sjzserver/models/ocr# 
(sjzserver) root@aiserver0:/home/Qing/sjzserver/models/ocr# pip install onnxruntime-gpu==1.19
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Requirement already satisfied: onnxruntime-gpu==1.19 in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (1.19.0)
Requirement already satisfied: coloredlogs in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.19) (15.0.1)
Requirement already satisfied: flatbuffers in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.19) (25.12.19)
Requirement already satisfied: numpy>=1.21.6 in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.19) (1.26.4)
Requirement already satisfied: packaging in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.19) (26.0)
Requirement already satisfied: protobuf in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.19) (7.35.0)
Requirement already satisfied: sympy in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.19) (1.14.0)
Requirement already satisfied: humanfriendly>=9.1 in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from coloredlogs->onnxruntime-gpu==1.19) (10.0)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from sympy->onnxruntime-gpu==1.19) (1.3.0)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
(sjzserver) root@aiserver0:/home/Qing/sjzserver/models/ocr# python ocr_onnx.py 
[INFO] 2026-05-20 17:51:25,864 [RapidOCR] base.py:22: Using engine_name: onnxruntime
[INFO] 2026-05-20 17:51:25,898 [RapidOCR] download_file.py:60: File exists and is valid: /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_PP-OCRv4_det_mobile.onnx
[INFO] 2026-05-20 17:51:25,898 [RapidOCR] main.py:57: Using /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_PP-OCRv4_det_mobile.onnx
[INFO] 2026-05-20 17:51:26,083 [RapidOCR] base.py:22: Using engine_name: onnxruntime
[INFO] 2026-05-20 17:51:26,084 [RapidOCR] download_file.py:60: File exists and is valid: /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_ppocr_mobile_v2.0_cls_mobile.onnx
[INFO] 2026-05-20 17:51:26,084 [RapidOCR] main.py:57: Using /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_ppocr_mobile_v2.0_cls_mobile.onnx
[INFO] 2026-05-20 17:51:26,105 [RapidOCR] base.py:22: Using engine_name: onnxruntime
[INFO] 2026-05-20 17:51:26,111 [RapidOCR] download_file.py:60: File exists and is valid: /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_PP-OCRv4_rec_mobile.onnx
[INFO] 2026-05-20 17:51:26,111 [RapidOCR] main.py:57: Using /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_PP-OCRv4_rec_mobile.onnx

=== 开始OCR性能测试 ===
预热次数: 5, 测试次数: 20
正在预热...
开始正式测试...
第  10 轮耗时: 30.56 ms
第  20 轮耗时: 31.13 ms

=== 性能测试结果 ===
平均耗时: 30.55 ms
最小耗时: 29.82 ms
最大耗时: 31.57 ms
耗时标准差: 0.44 ms (越小越稳定)
理论QPS (每秒处理图片数): 32.74

=== 功能正确性验证 ===
识别文本数量: 0

✅ OCR性能测试完成！
(sjzserver) root@aiserver0:/home/Qing/sjzserver/models/ocr# pip install onnxruntime-gpu==1.20
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Collecting onnxruntime-gpu==1.20
  Using cached https://mirrors.ustc.edu.cn/pypi/packages/06/24/ba71597b90802853d768e5598739a06a96600658df679ee1a7aa2b1ba335/onnxruntime_gpu-1.20.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (291.5 MB)
Requirement already satisfied: coloredlogs in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.20) (15.0.1)
Requirement already satisfied: flatbuffers in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.20) (25.12.19)
Requirement already satisfied: numpy>=1.21.6 in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.20) (1.26.4)
Requirement already satisfied: packaging in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.20) (26.0)
Requirement already satisfied: protobuf in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.20) (7.35.0)
Requirement already satisfied: sympy in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from onnxruntime-gpu==1.20) (1.14.0)
Requirement already satisfied: humanfriendly>=9.1 in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from coloredlogs->onnxruntime-gpu==1.20) (10.0)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages (from sympy->onnxruntime-gpu==1.20) (1.3.0)
Installing collected packages: onnxruntime-gpu
  Attempting uninstall: onnxruntime-gpu
    Found existing installation: onnxruntime-gpu 1.19.0
    Uninstalling onnxruntime-gpu-1.19.0:
      Successfully uninstalled onnxruntime-gpu-1.19.0
Successfully installed onnxruntime-gpu-1.20.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
(sjzserver) root@aiserver0:/home/Qing/sjzserver/models/ocr# python ocr_onnx.py 
[INFO] 2026-05-20 17:51:41,287 [RapidOCR] base.py:22: Using engine_name: onnxruntime
[INFO] 2026-05-20 17:51:41,322 [RapidOCR] download_file.py:60: File exists and is valid: /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_PP-OCRv4_det_mobile.onnx
[INFO] 2026-05-20 17:51:41,322 [RapidOCR] main.py:57: Using /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_PP-OCRv4_det_mobile.onnx
[INFO] 2026-05-20 17:51:41,515 [RapidOCR] base.py:22: Using engine_name: onnxruntime
[INFO] 2026-05-20 17:51:41,515 [RapidOCR] download_file.py:60: File exists and is valid: /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_ppocr_mobile_v2.0_cls_mobile.onnx
[INFO] 2026-05-20 17:51:41,515 [RapidOCR] main.py:57: Using /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_ppocr_mobile_v2.0_cls_mobile.onnx
[INFO] 2026-05-20 17:51:41,540 [RapidOCR] base.py:22: Using engine_name: onnxruntime
[INFO] 2026-05-20 17:51:41,546 [RapidOCR] download_file.py:60: File exists and is valid: /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_PP-OCRv4_rec_mobile.onnx
[INFO] 2026-05-20 17:51:41,546 [RapidOCR] main.py:57: Using /root/anaconda3/envs/sjzserver/lib/python3.10/site-packages/rapidocr/models/ch_PP-OCRv4_rec_mobile.onnx

=== 开始OCR性能测试 ===
预热次数: 5, 测试次数: 20
正在预热...
开始正式测试...
第  10 轮耗时: 152.64 ms
第  20 轮耗时: 156.91 ms

=== 性能测试结果 ===
平均耗时: 165.74 ms
最小耗时: 152.64 ms
最大耗时: 197.12 ms
耗时标准差: 14.53 ms (越小越稳定)
理论QPS (每秒处理图片数): 6.03

=== 功能正确性验证 ===
识别文本数量: 0

✅ OCR性能测试完成！

[SWHL]

ort gpu 版加速效果本来就十分有限哈。GPU加速不建议有这个。建议用PyTorch或者Paddle。

参见文档：ONNX Runtime GPU 推理