Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

多线程调用C++推理库进行OCR推理时出现报错 #6514

Closed
qiuming-93 opened this issue Jun 8, 2022 · 11 comments
Closed

多线程调用C++推理库进行OCR推理时出现报错 #6514

qiuming-93 opened this issue Jun 8, 2022 · 11 comments
Assignees

Comments

@qiuming-93
Copy link

qiuming-93 commented Jun 8, 2022

请提供下述完整信息以便快速定位问题/Please provide the following information to quickly locate the problem

系统环境/System Environment:Centos
版本号/Version:Paddle_inference:2.1 PaddleOCR: 2.4 cuda:10.1
运行指令/Command Code:
完整报错/Complete Error Message:
多线程调用C++推理库进行OCR推理时出现报错,使用的是官网下载的识别模型ch_ppocr_server_v2.0_rec_infer,
非必现,线程数越多,越容易出现。不报错时可以正常分析
报错信息如下

terminate called after throwing an instance of 'paddle::platform::EnforceNotMet'
  what():  

  Compile Traceback (most recent call last):
    File "tools/export_model.py", line 75, in <module>
      main()
    File "tools/export_model.py", line 70, in main
      paddle.jit.save(model, save_path)
    File "<decorator-gen-60>", line 2, in save
      
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
      return wrapped_func(*args, **kwargs)
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/base.py", line 38, in __impl__
      return func(*args, **kwargs)
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/jit.py", line 680, in save
      concrete_program = static_func.concrete_program
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 461, in concrete_program
      concrete_program, _ = self.get_concrete_program(*input_spec)
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 401, in get_concrete_program
      concrete_program, partial_program_layer = self._program_cache[cache_key]
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 684, in __getitem__
      self._caches[item] = self._build_once(item)
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 675, in _build_once
      class_instance=cache_key.class_instance)
    File "<decorator-gen-58>", line 2, in from_func_spec
      
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
      return wrapped_func(*args, **kwargs)
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/base.py", line 38, in __impl__
      return func(*args, **kwargs)
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 625, in from_func_spec
      outputs = static_func(*inputs)
    File "/tmp/tmpqk3xmm2v.py", line 22, in forward
      false_fn_1, (x,), (x,), (x,))
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 210, in convert_ifelse
      return _run_py_ifelse(pred, true_fn, false_fn, true_args, false_args)
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 235, in _run_py_ifelse
      return true_fn(*true_args) if pred else false_fn(*false_args)
    File "/paddle/code/gry/PaddleOCR_pub/ppocr/modeling/architectures/base_model.py", line 76, in forward
      x = self.neck(x)
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 884, in __call__
      outputs = self.forward(*inputs, **kwargs)
    File "/tmp/tmpmf6_4ac7.py", line 14, in forward
      (x,), (x,))
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 210, in convert_ifelse
      return _run_py_ifelse(pred, true_fn, false_fn, true_args, false_args)
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 235, in _run_py_ifelse
      return true_fn(*true_args) if pred else false_fn(*false_args)
    File "/paddle/code/gry/PaddleOCR_pub/ppocr/modeling/necks/rnn.py", line 91, in forward
      x = self.encoder(x)
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 884, in __call__
      outputs = self.forward(*inputs, **kwargs)
    File "/paddle/code/gry/PaddleOCR_pub/ppocr/modeling/necks/rnn.py", line 45, in forward
      x, _ = self.lstm(x)
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 884, in __call__
      outputs = self.forward(*inputs, **kwargs)
    File "/usr/local/lib/python3.7/site-packages/paddle/nn/layer/rnn.py", line 1035, in forward
      return self._cudnn_impl(inputs, initial_states, sequence_length)
    File "/usr/local/lib/python3.7/site-packages/paddle/nn/layer/rnn.py", line 1012, in _cudnn_impl
      type="rnn", inputs=inputs, outputs=outputs, attrs=attrs)
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/dygraph/layer_object_helper.py", line 52, in append_op
      stop_gradient=stop_gradient)
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/framework.py", line 2849, in append_op
      attrs=kwargs.get("attrs", None))
    File "/usr/local/lib/python3.7/site-packages/paddle/fluid/framework.py", line 1934, in __init__
      for frame in traceback.extract_stack():

--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
3   PaddleOCR::OcrModel::run(cv::Mat&)
4   PaddleOCR::CRNNRecognizer::Run(std::vector<std::vector<std::vector<int, std::allocator<int> >, std::allocator<std::vector<int, std::allocator<int> > > >, std::allocator<std::vector<std::vector<int, std::allocator<int> >, std::allocator<std::vector<int, std::allocator<int> > > > > >, cv::Mat&, PaddleOCR::Classifier*)
5   paddle::AnalysisPredictor::ZeroCopyRun()
6   paddle::framework::NaiveExecutor::Run()
7   paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, paddle::platform::Place const&)
8   paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&) const
9   paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&, paddle::framework::RuntimeContext*) const
10  std::_Function_handler<void (paddle::framework::ExecutionContext const&), paddle::framework::OpKernelRegistrarFunctor<paddle::platform::CUDAPlace, false, 0ul, paddle::operators::RNNCudnnKernel<float>, paddle::operators::RNNCudnnKernel<double> >::operator()(char const*, char const*, int) const::{lambda(paddle::framework::ExecutionContext const&)#1}>::_M_invoke(std::_Any_data const&, paddle::framework::ExecutionContext const&)
11  paddle::operators::RNNCudnnKernel<float>::Compute(paddle::framework::ExecutionContext const&) const
12  paddle::platform::EnforceNotMet::EnforceNotMet(paddle::platform::ErrorSummary const&, char const*, int)
13  paddle::platform::GetCurrentTraceBackString[abi:cxx11]()

----------------------
Error Message Summary:
----------------------
ExternalError:  Cudnn error, CUDNN_STATUS_EXECUTION_FAILED  (at /paddle/paddle/fluid/operators/rnn_op.cu.cc:515)
  [operator < rnn > error]
@littletomatodonkey
Copy link
Collaborator

你好,GPU模式下不建议使用多线程预测,可以尝试开多个应用程序去使用ocr系统

@shangzhizhou
Copy link
Member

您好,predictor不是线程安全的。可以使用多线程,只要创建多个predictor,保证每个predictor同一时刻,只在一个线程下处理一个请求即可。

你好,GPU模式下不建议使用多线程预测,可以尝试开多个应用程序去使用ocr系统

@qiuming-93
Copy link
Author

您好,predictor不是线程安全的。可以使用多线程,只要创建多个predictor,保证每个predictor同一时刻,只在一个线程下处理一个请求即可。

你好,GPU模式下不建议使用多线程预测,可以尝试开多个应用程序去使用ocr系统

每个线程是单独创建的predictor

@yeliang2258
Copy link

你好,麻烦提供一下复现的脚本和模型吧

@qiuming-93
Copy link
Author

qiuming-93 commented Jun 22, 2022

基于PaddleOCR2.4.release的cpp_infer的demo进行复现的,把main.cpp代码进行修改,如下所示:另外建议把ocr_rec.cpp中的106-109行的打印结果部分注释掉,提高多个线程同时调用predictor推理的概率,模型直接使用下载的中英文通用PP-OCR server模型即可,执行命令为:

./build/ppocr system --char_list_file=./ppocr_keys_v1.txt  --det_model_dir=./model/ch_ppocr_server_v2.0_det_infer --use_angle_cls=false --rec_model_dir=./model/ch_ppocr_server_v2.0_rec_infer --image_dir=./imgs --use_gpu=true --cpu_threads=2
// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#include "glog/logging.h"
#include "omp.h"
#include "opencv2/core.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/imgproc.hpp"
#include <chrono>
#include <iomanip>
#include <iostream>
#include <ostream>
#include <vector>

#include <cstring>
#include <fstream>
#include <numeric>

#include <glog/logging.h>
#include <include/ocr_det.h>
#include <include/ocr_cls.h>
#include <include/ocr_rec.h>
#include <include/utility.h>
#include <sys/stat.h>
#include <thread>

#include <gflags/gflags.h>
#include "auto_log/autolog.h"

DEFINE_bool(use_gpu, false, "Infering with GPU or CPU.");
DEFINE_int32(gpu_id, 0, "Device id of GPU to execute.");
DEFINE_int32(gpu_mem, 4000, "GPU id when infering with GPU.");
DEFINE_int32(cpu_threads, 10, "Num of threads with CPU.");
DEFINE_bool(enable_mkldnn, false, "Whether use mkldnn with CPU.");
DEFINE_bool(use_tensorrt, false, "Whether use tensorrt.");
DEFINE_string(precision, "fp32", "Precision be one of fp32/fp16/int8");
DEFINE_bool(benchmark, false, "Whether use benchmark.");
DEFINE_string(save_log_path, "./log_output/", "Save benchmark log path.");
// detection related
DEFINE_string(image_dir, "", "Dir of input image.");
DEFINE_string(det_model_dir, "", "Path of det inference model.");
DEFINE_int32(max_side_len, 960, "max_side_len of input image.");
DEFINE_double(det_db_thresh, 0.3, "Threshold of det_db_thresh.");
DEFINE_double(det_db_box_thresh, 0.5, "Threshold of det_db_box_thresh.");
DEFINE_double(det_db_unclip_ratio, 1.6, "Threshold of det_db_unclip_ratio.");
DEFINE_bool(use_polygon_score, false, "Whether use polygon score.");
DEFINE_bool(visualize, true, "Whether show the detection results.");
// classification related
DEFINE_bool(use_angle_cls, false, "Whether use use_angle_cls.");
DEFINE_string(cls_model_dir, "", "Path of cls inference model.");
DEFINE_double(cls_thresh, 0.9, "Threshold of cls_thresh.");
// recognition related
DEFINE_string(rec_model_dir, "", "Path of rec inference model.");
DEFINE_int32(rec_batch_num, 6, "rec_batch_num.");
DEFINE_string(char_list_file, "../../ppocr/utils/ppocr_keys_v1.txt", "Path of dictionary.");


using namespace std;
using namespace cv;
using namespace PaddleOCR;


static bool PathExists(const std::string& path){
#ifdef _WIN32
  struct _stat buffer;
  return (_stat(path.c_str(), &buffer) == 0);
#else
  struct stat buffer;
  return (stat(path.c_str(), &buffer) == 0);
#endif  // !_WIN32
}

void predictor_thread(std::vector<cv::String> cv_all_img_names, int thread_id) {
	std::vector<double> time_info_det = {0, 0, 0};
    std::vector<double> time_info_rec = {0, 0, 0};
    DBDetector det(FLAGS_det_model_dir, FLAGS_use_gpu, FLAGS_gpu_id,
                   FLAGS_gpu_mem, FLAGS_cpu_threads, 
                   FLAGS_enable_mkldnn, FLAGS_max_side_len, FLAGS_det_db_thresh,
                   FLAGS_det_db_box_thresh, FLAGS_det_db_unclip_ratio,
                   FLAGS_use_polygon_score, FLAGS_visualize,
                   FLAGS_use_tensorrt, FLAGS_precision);

    Classifier *cls = nullptr;
    if (FLAGS_use_angle_cls) {
      cls = new Classifier(FLAGS_cls_model_dir, FLAGS_use_gpu, FLAGS_gpu_id,
                           FLAGS_gpu_mem, FLAGS_cpu_threads,
                           FLAGS_enable_mkldnn, FLAGS_cls_thresh,
                           FLAGS_use_tensorrt, FLAGS_precision);
    }

    std::string char_list_file = FLAGS_char_list_file;
    if (FLAGS_benchmark) 
        char_list_file = FLAGS_char_list_file.substr(6);
    cout << "label file: " << char_list_file << endl;
        
    CRNNRecognizer rec(FLAGS_rec_model_dir, FLAGS_use_gpu, FLAGS_gpu_id,
                       FLAGS_gpu_mem, FLAGS_cpu_threads,
                       FLAGS_enable_mkldnn, char_list_file,
                       FLAGS_use_tensorrt, FLAGS_precision, FLAGS_rec_batch_num);
	while(1) {
		for (int i = 0; i < cv_all_img_names.size(); ++i) {
		  LOG(INFO) << "The predict id: " << thread_id << " img: " << cv_all_img_names[i];

		  cv::Mat srcimg = cv::imread(cv_all_img_names[i], cv::IMREAD_COLOR);
		  if (!srcimg.data) {
			std::cerr << "[ERROR] image read failed! image path: " << cv_all_img_names[i] << endl;
			exit(1);
		  }
		  std::vector<std::vector<std::vector<int>>> boxes;
		  std::vector<double> det_times;
		  std::vector<double> rec_times;
			
		  det.Run(srcimg, boxes, &det_times);
		  time_info_det[0] += det_times[0];
		  time_info_det[1] += det_times[1];
		  time_info_det[2] += det_times[2];
			
		  std::vector<cv::Mat> img_list;
		  for (int j = 0; j < boxes.size(); j++) {
			  cv::Mat crop_img;
			  crop_img = Utility::GetRotateCropImage(srcimg, boxes[j]);
			  if (cls != nullptr) {
				  crop_img = cls->Run(crop_img);
			  }
			  img_list.push_back(crop_img);
		  }

		  rec.Run(img_list, &rec_times);
		  time_info_rec[0] += rec_times[0];
		  time_info_rec[1] += rec_times[1];
		  time_info_rec[2] += rec_times[2];
		}
	}
    if (FLAGS_benchmark) {
        AutoLogger autolog_det("ocr_det", 
                            FLAGS_use_gpu,
                            FLAGS_use_tensorrt,
                            FLAGS_enable_mkldnn,
                            FLAGS_cpu_threads,
                            1, 
                            "dynamic", 
                            FLAGS_precision, 
                            time_info_det, 
                            cv_all_img_names.size());
        AutoLogger autolog_rec("ocr_rec", 
                            FLAGS_use_gpu,
                            FLAGS_use_tensorrt,
                            FLAGS_enable_mkldnn,
                            FLAGS_cpu_threads,
                            FLAGS_rec_batch_num, 
                            "dynamic", 
                            FLAGS_precision, 
                            time_info_rec, 
                            cv_all_img_names.size());
        autolog_det.report();
        std::cout << endl;
        autolog_rec.report();
    } 
	std::cout<<"end"<<std::endl;	
}

int main_system(std::vector<cv::String> cv_all_img_names) {
    for(int i = 0; i < FLAGS_cpu_threads; i++) {
		thread t(predictor_thread, cv_all_img_names, i);
		t.detach();
		sleep(2);
	}
	while(1){
		sleep(100000);
	}
    return 0;
}


void check_params(char* mode) {
    if (strcmp(mode, "det")==0) {
        if (FLAGS_det_model_dir.empty() || FLAGS_image_dir.empty()) {
            std::cout << "Usage[det]: ./ppocr --det_model_dir=/PATH/TO/DET_INFERENCE_MODEL/ "
                      << "--image_dir=/PATH/TO/INPUT/IMAGE/" << std::endl;      
            exit(1);      
        }
    }
    if (strcmp(mode, "rec")==0) {
        if (FLAGS_rec_model_dir.empty() || FLAGS_image_dir.empty()) {
            std::cout << "Usage[rec]: ./ppocr --rec_model_dir=/PATH/TO/REC_INFERENCE_MODEL/ "
                      << "--image_dir=/PATH/TO/INPUT/IMAGE/" << std::endl;      
            exit(1);
        }
    }
    if (strcmp(mode, "system")==0) {
        if ((FLAGS_det_model_dir.empty() || FLAGS_rec_model_dir.empty() || FLAGS_image_dir.empty()) ||
           (FLAGS_use_angle_cls && FLAGS_cls_model_dir.empty())) {
            std::cout << "Usage[system without angle cls]: ./ppocr --det_model_dir=/PATH/TO/DET_INFERENCE_MODEL/ "
                        << "--rec_model_dir=/PATH/TO/REC_INFERENCE_MODEL/ "
                        << "--image_dir=/PATH/TO/INPUT/IMAGE/" << std::endl;
            std::cout << "Usage[system with angle cls]: ./ppocr --det_model_dir=/PATH/TO/DET_INFERENCE_MODEL/ "
                        << "--use_angle_cls=true "
                        << "--cls_model_dir=/PATH/TO/CLS_INFERENCE_MODEL/ "
                        << "--rec_model_dir=/PATH/TO/REC_INFERENCE_MODEL/ "
                        << "--image_dir=/PATH/TO/INPUT/IMAGE/" << std::endl;
            exit(1);      
        }
    }
    if (FLAGS_precision != "fp32" && FLAGS_precision != "fp16" && FLAGS_precision != "int8") {
        cout << "precison should be 'fp32'(default), 'fp16' or 'int8'. " << endl;
        exit(1);
    }
}


int main(int argc, char **argv) {
    if (argc<=1 || (strcmp(argv[1], "det")!=0 && strcmp(argv[1], "rec")!=0 && strcmp(argv[1], "system")!=0)) {
        std::cout << "Please choose one mode of [det, rec, system] !" << std::endl;
        return -1;
    }
    std::cout << "mode: " << argv[1] << endl;

    // Parsing command-line
    google::ParseCommandLineFlags(&argc, &argv, true);
    check_params(argv[1]);
        
    if (!PathExists(FLAGS_image_dir)) {
        std::cerr << "[ERROR] image path not exist! image_dir: " << FLAGS_image_dir << endl;
        exit(1);      
    }
    
    std::vector<cv::String> cv_all_img_names;
    cv::glob(FLAGS_image_dir, cv_all_img_names);
    std::cout << "total images num: " << cv_all_img_names.size() << endl;
      
    if (strcmp(argv[1], "system")==0) {
        return main_system(cv_all_img_names);
    } 

}

@WenmuZhou
Copy link
Collaborator

你好,我这边测试paddle 2.3+ paddleocr2.5没有问题,辛苦升级之后测试一下

@qiuming-93
Copy link
Author

你好,我这边测试paddle 2.3+ paddleocr2.5没有问题,辛苦升级之后测试一下

你好,我这边测试paddle 2.3+ paddleocr2.5没有问题,辛苦升级之后测试一下

OK,我们这边测试下

@qiuming-93
Copy link
Author

你好,我这边测试paddle 2.3+ paddleocr2.5没有问题,辛苦升级之后测试一下

还有个问题,如果使用paddle inference2.3的话,之前训练的OCR模型需要转换吗?还是直接可以使用

@WenmuZhou
Copy link
Collaborator

直接就可使用

@qiuming-93
Copy link
Author

直接就可使用

你好,目前基于之前paddleocr2.4的cpp_infer代码改写了支持ppinference2.3进行推理,经过测试发现目前该报错已经不再出现了,但是另一个问题还是会出现,#6512

@github-actions
Copy link
Contributor

github-actions bot commented Jul 1, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants