[OCR flask deployment] When using flask to deploy the OCR model with enable mkldnn, an error occurred in the second prediction. #33831

LDOUBLEV · 2021-06-29T07:14:39Z

Env:

linux + python3.7
paddlepaddle 2.1.1 pip3 install paddlepaddle-gpu==2.1.1
paddleocr 2.0.6 pip3 install paddleocr==2.0.6

I expect to deploy PaddleOCR on my machine using Flask. And I want to speed up the prediction using Mkldnn. However, an error occurred in the second prediction When using flask to deploy the OCR model with enable mkldnn.

However, the third prediction will not be wrong, and the fourth prediction will be wrong again. Moreover, there is no error occurred when I set threaded to False.

The error message:

The code to reproduce the problem is as follows.

from flask import Flask
from paddleocr import PaddleOCR, draw_ocr

app = Flask(__name__)

ocr_japan = PaddleOCR(use_angle_cls=True, use_gpu=False, lang="japan")
ocr_en = PaddleOCR(use_angle_cls=True, use_gpu=False, lang="en")
ocr_korean = PaddleOCR(use_angle_cls=True, use_gpu=False, lang="korean")
ocr_chinese = PaddleOCR(use_angle_cls=False, use_gpu=False, lang="ch", enable_mkldnn=True)

@app.route('/')
def hello_world():
    result = ocr_chinese.ocr("./image.jpg", cls=False)
    print(result)
    return "1"

if __name__ == '__main__':
    app.run(host="0.0.0.0", port=8088, threaded=True)  # failed
    #app.run(host="0.0.0.0", port=8088, threaded=False)  # success

When enable is set to False, the correct result can be obtained and no error is reported, but the prediction speed is too slow.

I found that there is a conv2d OP dnnl warning before every error.

Paddle/paddle/fluid/framework/operator.cc

Line 242 in 89fdd6c

LOG(WARNING) << Type() << " raises an exception "

The tested image.jpg
https://raw.githubusercontent.com/PaddlePaddle/PaddleOCR/dygraph/doc/imgs/11.jpg

The text was updated successfully, but these errors were encountered:

paddle-bot-old · 2021-06-29T07:14:42Z

您好，我们已经收到了您的问题，会安排技术人员尽快解答您的问题，请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时，您也可以通过查看官网API文档、常见问题、历史Issue、AI社区来寻求解答。祝您生活愉快～

Hi! We've received your issue and please be patient to get responded. We will arrange technicians to answer your questions as soon as possible. Please make sure that you have posted enough message to demo your request. You may also check out the API，FAQ，Github Issue and AI community to get the answer.Have a nice day!

lidanqing-intel · 2022-05-19T15:58:01Z

Reproduced with

paddlepaddle==2.3.0rc
paddleocr==2.5.0.3
To run it successfully, need to manually vim /home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/paddleocr/paddleocr.py, as solution to this issue No module named 'tools.infer' PaddleOCR#1024
Open 0.0.0.0:8088 on browser, each fresh will execute once.
The error is: In very second step, paddle with mkldnn true failed could not create primitive

   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on all addresses.
   WARNING: This is a development server. Do not use it in a production deployment.
 * Running on http://172.22.230.23:8088/ (Press CTRL+C to quit)
[2022/05/19 09:08:36] ppocr DEBUG: dt_boxes num : 16, elapse : 0.15598368644714355
[2022/05/19 09:08:36] ppocr DEBUG: rec_res num  : 16, elapse : 0.28359246253967285
[[[[28.0, 37.0], [302.0, 39.0], [302.0, 72.0], [27.0, 70.0]], ('纯臻营养护发素', 0.9658746719360352)], [[[26.0, 81.0], [172.0, 83.0], [172.0, 104.0], [25.0, 101.0]], ('产品信息/参数', 0.9113160371780396)], [[[28.0, 115.0], [330.0, 115.0], [330.0, 132.0], [28.0, 132.0]], ('（45元/每公斤，100公斤起订）', 0.8843305706977844)], [[[27.0, 145.0], [282.0, 145.0], [282.0, 164.0], [27.0, 164.0]], ('每瓶22元，1000 瓶起订）', 0.9211586117744446)], [[[26.0, 179.0], [299.0, 179.0], [299.0, 195.0], [26.0, 195.0]], ('【品牌】：代加工方式/OEMODM', 0.9661454558372498)], [[[26.0, 210.0], [233.0, 210.0], [233.0, 227.0], [26.0, 227.0]], ('【品名】：纯臻营养护发素', 0.8831908106803894)], [[[26.0, 241.0], [241.0, 241.0], [241.0, 258.0], [26.0, 258.0]], ('【产品编号】：YM-X-3011', 0.8718008995056152)], [[[413.0, 236.0], [430.0, 236.0], [430.0, 303.0], [413.0, 303.0]], ('ODMOEM', 0.9539503455162048)], [[[23.0, 271.0], [180.0, 269.0], [180.0, 289.0], [24.0, 290.0]], ('【净含量】：220ml', 0.9348611235618591)], [[[26.0, 304.0], [252.0, 304.0], [252.0, 320.0], [26.0, 320.0]], ('适用人群）：适合所有 肤质', 0.8866131901741028)], [[[26.0, 335.0], [343.0, 335.0], [343.0, 352.0], [26.0, 352.0]], ('【主要成分】：鲸蜡硬脂醇、燕麦β-葡聚', 0.9245771169662476)], [[[27.0, 366.0], [281.0, 366.0], [281.0, 383.0], [27.0, 383.0]], ('糖、椰油酰胺丙基甜菜碱、泛醒', 0.9368147850036621)], [[[369.0, 370.0], [477.0, 370.0], [477.0, 387.0], [369.0, 387.0]], ('（成品包材）', 0.8927483558654785)], [[[26.0, 397.0], [361.0, 397.0], [361.0, 414.0], [26.0, 414.0]], ('【主要功能】：可紧致头发磷层，从而达到', 0.867779552936554)], [[[28.0, 429.0], [372.0, 429.0], [372.0, 445.0], [28.0, 445.0]], ('即时持久改善头发光泽的效果，给于燥的头', 0.8803292512893677)], [[[27.0, 459.0], [136.0, 459.0], [136.0, 479.0], [27.0, 479.0]], ('发足够的滋养', 0.9091534614562988)]]
163.33.54.55 - - [19/May/2022 09:08:36] "GET / HTTP/1.1" 200 -
[2022-05-19 09:09:06,120] ERROR in app: Exception on / [GET]
Traceback (most recent call last):
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/flask/app.py", line 2070, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/flask/app.py", line 1515, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/flask/app.py", line 1513, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/flask/app.py", line 1499, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "flask_test.py", line 13, in hello_world
    result = ocr_chinese.ocr("./image.jpg", cls=False)
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/paddleocr/paddleocr.py", line 474, in ocr
    dt_boxes, rec_res = self.__call__(img, cls)
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/paddleocr/tools/infer/predict_system.py", line 69, in __call__
    dt_boxes, elapse = self.text_detector(img)
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/paddleocr/tools/infer/predict_det.py", line 216, in __call__
    self.predictor.run()
RuntimeError: could not execute a primitive
163.33.54.55 - - [19/May/2022 09:09:06] "GET / HTTP/1.1" 500 -
[2022/05/19 09:09:31] ppocr DEBUG: dt_boxes num : 16, elapse : 0.05440521240234375
[2022/05/19 09:09:32] ppocr DEBUG: rec_res num  : 16, elapse : 0.13421416282653809
[[[[28.0, 37.0], [302.0, 39.0], [302.0, 72.0], [27.0, 70.0]], ('纯臻营养护发素', 0.9658746719360352)], [[[26.0, 81.0], [172.0, 83.0], [172.0, 104.0], [25.0, 101.0]], ('产品信息/参数', 0.9113160371780396)], [[[28.0, 115.0], [330.0, 115.0], [330.0, 132.0], [28.0, 132.0]], ('（45元/每公斤，100公斤起订）', 0.8843305706977844)], [[[27.0, 145.0], [282.0, 145.0], [282.0, 164.0], [27.0, 164.0]], ('每瓶22元，1000 瓶起订）', 0.9211586117744446)], [[[26.0, 179.0], [299.0, 179.0], [299.0, 195.0], [26.0, 195.0]], ('【品牌】：代加工方式/OEMODM', 0.9661454558372498)], [[[26.0, 210.0], [233.0, 210.0], [233.0, 227.0], [26.0, 227.0]], ('【品名】：纯臻营养护发素', 0.8831908106803894)], [[[26.0, 241.0], [241.0, 241.0], [241.0, 258.0], [26.0, 258.0]], ('【产品编号】：YM-X-3011', 0.8718008995056152)], [[[413.0, 236.0], [430.0, 236.0], [430.0, 303.0], [413.0, 303.0]], ('ODMOEM', 0.9539503455162048)], [[[23.0, 271.0], [180.0, 269.0], [180.0, 289.0], [24.0, 290.0]], ('【净含量】：220ml', 0.9348611235618591)], [[[26.0, 304.0], [252.0, 304.0], [252.0, 320.0], [26.0, 320.0]], ('适用人群）：适合所有 肤质', 0.8866131901741028)], [[[26.0, 335.0], [343.0, 335.0], [343.0, 352.0], [26.0, 352.0]], ('【主要成分】：鲸蜡硬脂醇、燕麦β-葡聚', 0.9245771169662476)], [[[27.0, 366.0], [281.0, 366.0], [281.0, 383.0], [27.0, 383.0]], ('糖、椰油酰胺丙基甜菜碱、泛醒', 0.9368147850036621)], [[[369.0, 370.0], [477.0, 370.0], [477.0, 387.0], [369.0, 387.0]], ('（成品包材）', 0.8927483558654785)], [[[26.0, 397.0], [361.0, 397.0], [361.0, 414.0], [26.0, 414.0]], ('【主要功能】：可紧致头发磷层，从而达到', 0.867779552936554)], [[[28.0, 429.0], [372.0, 429.0], [372.0, 445.0], [28.0, 445.0]], ('即时持久改善头发光泽的效果，给于燥的头', 0.8803292512893677)], [[[27.0, 459.0], [136.0, 459.0], [136.0, 479.0], [27.0, 479.0]], ('发足够的滋养', 0.9091534614562988)]]
163.33.54.55 - - [19/May/2022 09:09:32] "GET / HTTP/1.1" 200 -

Zhongyi-89 · 2022-12-22T22:51:52Z

Reproduced with

paddlepaddle==2.3.0rc
paddleocr==2.5.0.3
To run it successfully, need to manually vim /home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/paddleocr/paddleocr.py, as solution to this issue No module named 'tools.infer' PaddleOCR#1024
Open 0.0.0.0:8088 on browser, each fresh will execute once.
The error is: In very second step, paddle with mkldnn true failed could not create primitive

   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on all addresses.
   WARNING: This is a development server. Do not use it in a production deployment.
 * Running on http://172.22.230.23:8088/ (Press CTRL+C to quit)
[2022/05/19 09:08:36] ppocr DEBUG: dt_boxes num : 16, elapse : 0.15598368644714355
[2022/05/19 09:08:36] ppocr DEBUG: rec_res num  : 16, elapse : 0.28359246253967285
[[[[28.0, 37.0], [302.0, 39.0], [302.0, 72.0], [27.0, 70.0]], ('纯臻营养护发素', 0.9658746719360352)], [[[26.0, 81.0], [172.0, 83.0], [172.0, 104.0], [25.0, 101.0]], ('产品信息/参数', 0.9113160371780396)], [[[28.0, 115.0], [330.0, 115.0], [330.0, 132.0], [28.0, 132.0]], ('（45元/每公斤，100公斤起订）', 0.8843305706977844)], [[[27.0, 145.0], [282.0, 145.0], [282.0, 164.0], [27.0, 164.0]], ('每瓶22元，1000 瓶起订）', 0.9211586117744446)], [[[26.0, 179.0], [299.0, 179.0], [299.0, 195.0], [26.0, 195.0]], ('【品牌】：代加工方式/OEMODM', 0.9661454558372498)], [[[26.0, 210.0], [233.0, 210.0], [233.0, 227.0], [26.0, 227.0]], ('【品名】：纯臻营养护发素', 0.8831908106803894)], [[[26.0, 241.0], [241.0, 241.0], [241.0, 258.0], [26.0, 258.0]], ('【产品编号】：YM-X-3011', 0.8718008995056152)], [[[413.0, 236.0], [430.0, 236.0], [430.0, 303.0], [413.0, 303.0]], ('ODMOEM', 0.9539503455162048)], [[[23.0, 271.0], [180.0, 269.0], [180.0, 289.0], [24.0, 290.0]], ('【净含量】：220ml', 0.9348611235618591)], [[[26.0, 304.0], [252.0, 304.0], [252.0, 320.0], [26.0, 320.0]], ('适用人群）：适合所有 肤质', 0.8866131901741028)], [[[26.0, 335.0], [343.0, 335.0], [343.0, 352.0], [26.0, 352.0]], ('【主要成分】：鲸蜡硬脂醇、燕麦β-葡聚', 0.9245771169662476)], [[[27.0, 366.0], [281.0, 366.0], [281.0, 383.0], [27.0, 383.0]], ('糖、椰油酰胺丙基甜菜碱、泛醒', 0.9368147850036621)], [[[369.0, 370.0], [477.0, 370.0], [477.0, 387.0], [369.0, 387.0]], ('（成品包材）', 0.8927483558654785)], [[[26.0, 397.0], [361.0, 397.0], [361.0, 414.0], [26.0, 414.0]], ('【主要功能】：可紧致头发磷层，从而达到', 0.867779552936554)], [[[28.0, 429.0], [372.0, 429.0], [372.0, 445.0], [28.0, 445.0]], ('即时持久改善头发光泽的效果，给于燥的头', 0.8803292512893677)], [[[27.0, 459.0], [136.0, 459.0], [136.0, 479.0], [27.0, 479.0]], ('发足够的滋养', 0.9091534614562988)]]
163.33.54.55 - - [19/May/2022 09:08:36] "GET / HTTP/1.1" 200 -
[2022-05-19 09:09:06,120] ERROR in app: Exception on / [GET]
Traceback (most recent call last):
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/flask/app.py", line 2070, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/flask/app.py", line 1515, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/flask/app.py", line 1513, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/flask/app.py", line 1499, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "flask_test.py", line 13, in hello_world
    result = ocr_chinese.ocr("./image.jpg", cls=False)
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/paddleocr/paddleocr.py", line 474, in ocr
    dt_boxes, rec_res = self.__call__(img, cls)
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/paddleocr/tools/infer/predict_system.py", line 69, in __call__
    dt_boxes, elapse = self.text_detector(img)
  File "/home/li/miniconda3/envs/py3.7/lib/python3.7/site-packages/paddleocr/tools/infer/predict_det.py", line 216, in __call__
    self.predictor.run()
RuntimeError: could not execute a primitive
163.33.54.55 - - [19/May/2022 09:09:06] "GET / HTTP/1.1" 500 -
[2022/05/19 09:09:31] ppocr DEBUG: dt_boxes num : 16, elapse : 0.05440521240234375
[2022/05/19 09:09:32] ppocr DEBUG: rec_res num  : 16, elapse : 0.13421416282653809
[[[[28.0, 37.0], [302.0, 39.0], [302.0, 72.0], [27.0, 70.0]], ('纯臻营养护发素', 0.9658746719360352)], [[[26.0, 81.0], [172.0, 83.0], [172.0, 104.0], [25.0, 101.0]], ('产品信息/参数', 0.9113160371780396)], [[[28.0, 115.0], [330.0, 115.0], [330.0, 132.0], [28.0, 132.0]], ('（45元/每公斤，100公斤起订）', 0.8843305706977844)], [[[27.0, 145.0], [282.0, 145.0], [282.0, 164.0], [27.0, 164.0]], ('每瓶22元，1000 瓶起订）', 0.9211586117744446)], [[[26.0, 179.0], [299.0, 179.0], [299.0, 195.0], [26.0, 195.0]], ('【品牌】：代加工方式/OEMODM', 0.9661454558372498)], [[[26.0, 210.0], [233.0, 210.0], [233.0, 227.0], [26.0, 227.0]], ('【品名】：纯臻营养护发素', 0.8831908106803894)], [[[26.0, 241.0], [241.0, 241.0], [241.0, 258.0], [26.0, 258.0]], ('【产品编号】：YM-X-3011', 0.8718008995056152)], [[[413.0, 236.0], [430.0, 236.0], [430.0, 303.0], [413.0, 303.0]], ('ODMOEM', 0.9539503455162048)], [[[23.0, 271.0], [180.0, 269.0], [180.0, 289.0], [24.0, 290.0]], ('【净含量】：220ml', 0.9348611235618591)], [[[26.0, 304.0], [252.0, 304.0], [252.0, 320.0], [26.0, 320.0]], ('适用人群）：适合所有 肤质', 0.8866131901741028)], [[[26.0, 335.0], [343.0, 335.0], [343.0, 352.0], [26.0, 352.0]], ('【主要成分】：鲸蜡硬脂醇、燕麦β-葡聚', 0.9245771169662476)], [[[27.0, 366.0], [281.0, 366.0], [281.0, 383.0], [27.0, 383.0]], ('糖、椰油酰胺丙基甜菜碱、泛醒', 0.9368147850036621)], [[[369.0, 370.0], [477.0, 370.0], [477.0, 387.0], [369.0, 387.0]], ('（成品包材）', 0.8927483558654785)], [[[26.0, 397.0], [361.0, 397.0], [361.0, 414.0], [26.0, 414.0]], ('【主要功能】：可紧致头发磷层，从而达到', 0.867779552936554)], [[[28.0, 429.0], [372.0, 429.0], [372.0, 445.0], [28.0, 445.0]], ('即时持久改善头发光泽的效果，给于燥的头', 0.8803292512893677)], [[[27.0, 459.0], [136.0, 459.0], [136.0, 479.0], [27.0, 479.0]], ('发足够的滋养', 0.9091534614562988)]]
163.33.54.55 - - [19/May/2022 09:09:32] "GET / HTTP/1.1" 200 -

@lidanqing-intel @LDOUBLEV is there any follow up? Does this problem solved ? I face the same issue by calling predict_det.py.

with mkldnn set to "True"

Once set mkldnn to "False", this problem gone, but processing time increase.

tcmtang · 2023-09-25T03:38:40Z

应该与flask进程冲突有关，这个Issue有提到：https://github.com/PaddlePaddle/PaddleOCR/issues/3753

paddle-bot-old bot assigned huangjun12 Jun 29, 2021

OliverLPH assigned lidanqing-intel and unassigned huangjun12 Jun 29, 2021

OliverLPH added the Intel label Jun 29, 2021

lidanqing-intel assigned Aganlengzi and unassigned Aganlengzi Nov 29, 2021

lidanqing-intel assigned jczaja and unassigned lidanqing-intel Aug 1, 2022

paddle-bot-old bot closed this as completed Jan 11, 2023

paddle-bot bot added the status/close 已关闭 label Jan 11, 2023

n0099 mentioned this issue Apr 22, 2023

反复针对相同/不同的图片持续执行PaddleOCRAll.Run()10分钟后stdout显示'dnnl:error' could not execute a primitive并退出进程 sdcb/PaddleSharp#44

Open

w5688414 mentioned this issue Mar 29, 2024

[Question]: FAQ模型部署成功以后，经过异常捕获处理后，在多轮次问答过程中，出现一次成功一次失败的现象。 PaddlePaddle/PaddleNLP#8175

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[OCR flask deployment] When using flask to deploy the OCR model with enable mkldnn, an error occurred in the second prediction. #33831

[OCR flask deployment] When using flask to deploy the OCR model with enable mkldnn, an error occurred in the second prediction. #33831

LDOUBLEV commented Jun 29, 2021 •

edited

Loading

paddle-bot-old bot commented Jun 29, 2021

lidanqing-intel commented May 19, 2022

Zhongyi-89 commented Dec 22, 2022

tcmtang commented Sep 25, 2023

[OCR flask deployment] When using flask to deploy the OCR model with enable mkldnn, an error occurred in the second prediction. #33831

[OCR flask deployment] When using flask to deploy the OCR model with enable mkldnn, an error occurred in the second prediction. #33831

Comments

LDOUBLEV commented Jun 29, 2021 • edited Loading

paddle-bot-old bot commented Jun 29, 2021

lidanqing-intel commented May 19, 2022

Zhongyi-89 commented Dec 22, 2022

tcmtang commented Sep 25, 2023

LDOUBLEV commented Jun 29, 2021 •

edited

Loading