NotImplementedError in oneDNN PIR conversion breaks CPU inference in PaddlePaddle 3.3.0

[glenn-jocher]

bug描述 Describe the Bug

Environment

• PaddlePaddle version: 3.3.0 (just released on PyPI)
• Python version: 3.12
• OS: Ubuntu Linux (x86_64)
• Hardware: CPU with oneDNN (default configuration)

Issue Description

CPU inference using paddle.inference fails with NotImplementedError in PaddlePaddle 3.3.0. This regression breaks inference for models exported from frameworks like Ultralytics YOLO. The issue occurs in the PIR (Paddle Intermediate Representation) to oneDNN conversion layer.

This worked correctly in PaddlePaddle 3.2.x and earlier.

Steps to Reproduce

import paddle.inference as pdi
import numpy as np

# Load exported PaddlePaddle model
model_dir = "yolo11n_paddle_model"
config = pdi.Config(f"{model_dir}/model.pdmodel", f"{model_dir}/model.pdiparams")
predictor = pdi.create_predictor(config)

# Setup input
input_handle = predictor.get_input_handle(predictor.get_input_names()[0])
input_handle.copy_from_cpu(np.random.randn(1, 3, 640, 640).astype(np.float32))

# Run inference - FAILS in 3.3.0
predictor.run()

Error Message

NotImplementedError: (Unimplemented) ConvertPirAttribute2RuntimeAttribute not support [pir::ArrayAttribute<pir::DoubleAttribute>].
  [Hint: Expected false, but it was true.] (at /paddle/paddle/fluid/pir/transforms/onednn/onednn_instruction.cc:116)

Expected vs Actual Behavior

- Expected: Inference completes successfully (as in 3.2.x)
- Actual: NotImplementedError raised during predictor.run()

Workaround

Downgrade to PaddlePaddle 3.2.2:
pip install paddlepaddle==3.2.2

Impact

This is a breaking change affecting:
- All CPU inference using oneDNN (default on Linux)
- Ultralytics YOLO PaddlePaddle export users
- Any application using paddle.inference on CPU

### 其他补充信息 Additional Supplementary Information

- Ultralytics CI failure: https://github.com/ultralytics/ultralytics/actions
- Affected code path: onednn_instruction.cc:116

[davidkhala]

I see the same issue when I am getting started on PaddleOCR.

[alexaryn]

When can a fix be released? I see there is a PR that needs some more work. This bug is preventing upgrades to 3.3.

[mpilioun]

I am using version 3.4.0 and I am having the same issue. Which version is stable so that try to use PaddleOCR?

[sawoz]

Having the same issue, is there any fix possibly on the way? Thanks!

[GarlicToothpaste]

Also have the same problem also.

[makdinoven]

PaddleOCR 3.4.0 + PaddlePaddle 3.3.0: MKLDNN Backend Crash

Symptoms

When running PaddleOCR inference on CPU, the process crashes with:

NotImplementedError: (Unimplemented) ConvertPirAttribute2RuntimeAttribute not support
[pir::ArrayAttribute<pir::DoubleAttribute>]
(at /paddle/paddle/fluid/framework/new_executor/instruction/onednn/onednn_instruction.cc:116)

Other variations of the same root cause:

• ResourceExhaustedError: Fail to alloc memory of 24546115584 size
• PaddleOCR.predict() got an unexpected keyword argument 'cls'

Root Cause

PaddleOCR 3.4.0 uses PaddleX as its inference backend. For CPU inference, PaddleX defaults to run_mode="mkldnn" (Intel oneDNN). You can verify this yourself:

from paddlex.inference.models.common.static_infer import get_default_run_mode

print(get_default_run_mode("PP-OCRv4_mobile_det", "cpu"))  # "mkldnn"
print(get_default_run_mode("en_PP-OCRv4_mobile_rec", "cpu"))  # "mkldnn"

PaddlePaddle 3.3.0 has a bug where its new PIR (Paddle Intermediate Representation) is incompatible with the MKLDNN instruction converter. The crash happens deep inside the static inference engine at predictor.run().

Important: Setting FLAGS_use_mkldnn=0 as an environment variable does not help. PaddleX controls the inference backend through its own PaddlePredictorOption.run_mode and ignores the Paddle-level flag.

Fix

Pass enable_mkldnn=False to the PaddleOCR() constructor:

from paddleocr import PaddleOCR

ocr = PaddleOCR(
    lang="en",
    ocr_version="PP-OCRv4",
    enable_mkldnn=False,  # prevents MKLDNN/PIR crash
)

results = list(ocr.predict(image_array))

How it works

The enable_mkldnn kwarg flows through the following chain:

1. PaddleOCR.__init__(**kwargs) passes unknown kwargs to super().__init__()
2. PaddleXPipelineWrapper.__init__() calls parse_common_args(kwargs)
3. parse_common_args() recognizes enable_mkldnn and stores it in common config
4. PaddleX configures the inference predictor with run_mode="paddle" instead of run_mode="mkldnn"

This bypasses the broken MKLDNN/PIR code path entirely.

Docker Compose

As additional insurance, set the environment variable in your docker-compose.yml:

services:
  worker:
    environment:
      - FLAGS_use_mkldnn=0
      - PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK=True

The second variable skips a slow connectivity check to HuggingFace on startup.

Affected Versions

• PaddleOCR 3.4.0
• PaddlePaddle 3.3.0
• CPU-only inference on x86_64 Linux

GPU inference is not affected since it uses CUDA, not MKLDNN.

[krudo-taco]

This error occurred because I forgot to activate the correct conda environment containing the proper PaddlePaddle installation. After activating the correct environment, CPU inference worked normally. This was an environment issue rather than a framework bug.