How to use the onnx (converted from .pt) in opencv C++ #5378

NacerFaraj · 2021-10-28T18:21:53Z

I am using OpenCV 4.5.4, which supports the use of converted yolov5 from pythorch (*.pt) to onnx.

I got a pre-trained yolov5 model. After converting it to onnx, I can load the model successfully using: cv::dnn::readNetFromONNX("best.onnx"). BUT, I do not know how to use it to find the bounding boxes around the objects given a test image!

There are plenty of samples showing how to do so with the darknet. But, I couldn't find any sample for using yolov5.onnx in opencv & C++.

any help?

The text was updated successfully, but these errors were encountered:

github-actions · 2021-10-28T18:22:29Z

👋 Hello @NacerFaraj, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python>=3.6.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

$ git clone https://github.com/ultralytics/yolov5
$ cd yolov5
$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

Google Colab and Kaggle notebooks with free GPU:
Google Cloud Deep Learning VM. See GCP Quickstart Guide
Amazon Deep Learning AMI. See AWS Quickstart Guide
Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

glenn-jocher · 2021-10-29T09:57:43Z

@NacerFaraj we don't have C/C++ examples of cv2 DNN ONNX inference, but you can use detect.py as a reference example of python cv2 DNN ONNX inference:

python export.py --weights yolov5s.pt --include onnx

python detect.py --weights yolov5s.onnx --dnn  # DNN inference
python detect.py --weights yolov5s.onnx  # ONNX Runtime inference

NacerFaraj · 2021-10-29T16:16:28Z

@NacerFaraj we don't have C/C++ examples of cv2 DNN ONNX inference, but you can use detect.py as a reference example of python cv2 DNN ONNX inference:
python export.py --weights yolov5s.pt --include onnx

python detect.py --weights yolov5s.onnx --dnn  # DNN inference
python detect.py --weights yolov5s.onnx  # ONNX Runtime inference

Then, would you plz explain the the structure of the output? And possibly, how to extract the confidences and boxes from it.

net = cv::dnn::readNetFromONNX("yolov5.onnx");
cv::Mat blob;
cv::dnn::blobFromImage(img, blob, 1.0/255, cv::Size(blobSize, blobSize), cv::Scalar(0, 0, 0), true, false);
net.setInput(blob);
cv::Mat output = net.forward();

glenn-jocher · 2021-10-29T17:18:58Z

@NacerFaraj for COCO with 80 classes outputs will be shape(n,85) with 85 dimension = (x,y,w,h,object_conf, class0_conf, class1_conf, ...)

futureflsl · 2021-11-01T08:08:21Z

have you solved your problem?I just stuck in this problem like you.can you share the code here or for me?

NacerFaraj · 2021-11-01T08:49:30Z

have you solved your problem?I just stuck in this problem like you.can you share the code here or for me?

not yet :-(

glenn-jocher · 2021-11-01T09:32:45Z

@NacerFaraj we don't provide assistance debugging custom code due to limited resources, however for custom workflows be aware that YOLOv5 models expect inputs in RGB order.

hilevelhdd · 2021-11-02T17:55:26Z

I don't think yolov5 can be used in this opencv4.5.4.
The model is loaded but not detected

PauloMendes33 · 2021-12-02T17:22:13Z

You can use Yolo the next way. In C++

auto net = cv::dnn::readNetFromONNX("yolov5.onnx");
// using GPU for image processing
//net.setPreferableBackend(cv::dnn::DNN_BACKEND_CUDA);
//net.setPreferableTarget(cv::dnn::DNN_TARGET_CUDA);
// using CPU for image processing
net.setPreferableBackend(cv::dnn::DNN_BACKEND_OPENCV);
net.setPreferableTarget(cv::dnn::DNN_TARGET_CPU);
auto output_names = net.getUnconnectedOutLayersNames();
cv::Mat myROI_redefined_YOLO, blob;
std::vectorcv::Mat detections;
std::vector indices[NUM_CLASSES];
std::vectorcv::Rect boxes[NUM_CLASSES];
std::vector scores[NUM_CLASSES];

// Creates 4-dimensional blob from image.
cv::dnn::blobFromImage(myROI_redefined_YOLO, blob, 0.00392, cv::Size(myROI_redefined_YOLO.rows, myROI_redefined_YOLO.cols), cv::Scalar(), true, false, CV_32F);
net.setInput(blob);
net.forward(detections, output_names);
// object detection using YOLOV5
for (auto& output : detections){
const auto num_boxes = output.rows;
for (int i = 0; i < num_boxes; i++){
//calculo das 5 predições para cada bounding box: x, y, w, h , confiança
auto x = output.at(i, 0) * myROI_redefined_YOLO.cols;
auto y = output.at(i, 1) * myROI_redefined_YOLO.rows;
auto width = output.at(i, 2) * myROI_redefined_YOLO.cols;
auto height = output.at(i, 3) * myROI_redefined_YOLO.rows;
cv::Rect rect(x - width/2, y - height/2, width, height);

            for (int c = 0; c < NUM_CLASSES; c++){
                auto confidence = *output.ptr<float>(i, 5 + c);
                if (confidence >= CONFIDENCE_THRESHOLD){
                    boxes[c].push_back(rect);
                    scores[c].push_back(confidence);
                }
            }
        }
}
// Realiza a supressão não máxima das bounding boxes e das pontuações  de confiança correspondentes.
// eliminação de bounding boxes repetidas que identificam o mesmo objecto.
for (int c = 0; c < NUM_CLASSES; c++)
        cv::dnn::NMSBoxes(boxes[c], scores[c], 0.0, NMS_THRESHOLD, indices[c]);
    
// identificação dos objectos e correspondentes pontuações de confiança através de bounding boxes. 
for (int c= 0; c < NUM_CLASSES; c++){
        for (size_t i = 0; i < indices[c].size(); ++i){
            const auto color = colors[c % NUM_COLORS];

            auto idx = indices[c][i];
            const auto& rect = boxes[c][idx];
            cv::rectangle(myROI_redefined_YOLO, cv::Point(rect.x, rect.y), cv::Point(rect.x + rect.width, rect.y + rect.height), color, 3);
            
            // coloco a identificação da classe do objeto contido na bounding box - pedestre ou garrafa por ex.
            std::ostringstream label_ss;
            label_ss << class_names[c] << ": " << std::fixed << std::setprecision(2) << scores[c][idx];
            auto label = label_ss.str();
            
            int baseline;
            auto label_bg_sz = cv::getTextSize(label.c_str(), cv::FONT_HERSHEY_COMPLEX_SMALL, 1, 1, &baseline);
            // defino o rectangulo que define o objeto detectado
            cv::rectangle(myROI_redefined_YOLO, cv::Point(rect.x, rect.y - label_bg_sz.height - baseline - 10), cv::Point(rect.x + label_bg_sz.width, rect.y), color, cv::FILLED);
            // coloco a identificação da classe do objecto detectado.
            cv::putText(myROI_redefined_YOLO, label.c_str(), cv::Point(rect.x, rect.y - baseline - 5), cv::FONT_HERSHEY_COMPLEX_SMALL, 1, cv::Scalar(0, 0, 0));
        }
}

There must be changed one or other things, but I think it will work.
If you are able to detect something please answer this comment.

PauloMendes33 · 2021-12-02T17:23:35Z

@hilevelhdd @futureflsl @NacerFaraj please see the last comment and tell us your result.

Good luck

PauloMendes33 · 2021-12-03T15:49:02Z

I have installed OpenCV 4.5.4 and correctly loaded the model with cv::dnn::readNetFromONNX("yolov5.onnx") after python3 export.py --weights best.pt --include onnx --simplify.

However I was unable to make detections. The output is a downscaled image without any prediction.

glenn-jocher · 2021-12-03T16:30:10Z

@PauloMendes33 DNN inference is simple:

python export.py --weights yolov5s.pt --include onnx
python detect.py --weights yolov5s.onnx --dnn

rtrahms · 2021-12-28T06:50:49Z

I have the same issue...

python detect.py using the exported onnx weights files and a sample image works.
OpenCV 4.5.4 can load the same onnx file in a simple CPP app.
No detections found from the same sample image in that CPP app.

rchaud99 · 2022-01-17T01:38:48Z

I have the same issue...

python detect.py using the exported onnx weights files and a sample image works.

OpenCV 4.5.4 can load the same onnx file in a simple CPP app.

No detections found from the same sample image in that CPP app.

Were you able to figure this out @rtrahms ?

Hexmagic · 2022-01-17T02:16:47Z

I have the same issue...

python detect.py using the exported onnx weights files and a sample image works.

OpenCV 4.5.4 can load the same onnx file in a simple CPP app.

No detections found from the same sample image in that CPP app.

have a look at my repo ONNX-yolov5

glenn-jocher · 2022-01-17T02:20:49Z

@Hexmagic awesome, nice work! We don't have any good C++ inference example ourselves unfortunately.

rchaud99 · 2022-01-17T02:30:08Z

Thanks @Hexmagic! Are you getting the same results for predictions as you are using the detect.py that @glenn-jocher recommends? I'm getting wildly different results on OpenCV (even on python) than using detect.py, with the same model and inputs.

rchaud99 · 2022-01-17T05:41:55Z

Thanks @Hexmagic! Are you getting the same results for predictions as you are using the detect.py that @glenn-jocher recommends? I'm getting wildly different results on OpenCV (even on python) than using detect.py, with the same model and inputs.

detect.py use img original shape，not 640x640

Where in detect.py is this? I see that the default --imgsz parameter is 640x640 on line 54

Hexmagic · 2022-01-17T06:01:07Z

Thanks @Hexmagic! Are you getting the same results for predictions as you are using the detect.py that @glenn-jocher recommends? I'm getting wildly different results on OpenCV (even on python) than using detect.py, with the same model and inputs.

detect.py use img original shape，not 640x640

Where in detect.py is this? I see that the default --imgsz parameter is 640x640 on line 54

I was wrong,detect.py not use original shape or 640x640,108 line pt = True and img_size = 640 and stride = 32

dataset = LoadImages(source, img_size=imgsz, stride=stride, auto=pt)

augmentation.py line 105, auto=True will make minimum rectangle,

dw, dh = new_shape[1] - new_unpad[0], new_shape[0] - new_unpad[1]  # wh padding  (640-480)=160, (640-640)=0
if auto:  # minimum rectangle
    dw, dh = np.mod(dw, stride), np.mod(dh, stride)  # wh padding  160%320=0, 0%32=0

rchaud99 · 2022-01-17T06:11:42Z

I was wrong,detect.py not use original shape or 640x640,108 line pt = True and img_size = 640 and stride = 32
dataset = LoadImages(source, img_size=imgsz, stride=stride, auto=pt)
augmentation.py line 105, auto=True will make minimum rectangle,
dw, dh = new_shape[1] - new_unpad[0], new_shape[0] - new_unpad[1]  # wh padding  (640-480)=160, (640-640)=0
if auto:  # minimum rectangle
    dw, dh = np.mod(dw, stride), np.mod(dh, stride)  # wh padding  160%320=0, 0%32=0

Thanks - any idea why OpenCV net is much lower confidence than detect.py even when same resolution? Or is it the same for you?

Hexmagic · 2022-01-17T09:55:05Z

I was wrong,detect.py not use original shape or 640x640,108 line pt = True and img_size = 640 and stride = 32
dataset = LoadImages(source, img_size=imgsz, stride=stride, auto=pt)
augmentation.py line 105, auto=True will make minimum rectangle,
dw, dh = new_shape[1] - new_unpad[0], new_shape[0] - new_unpad[1]  # wh padding  (640-480)=160, (640-640)=0
if auto:  # minimum rectangle
    dw, dh = np.mod(dw, stride), np.mod(dh, stride)  # wh padding  160%320=0, 0%32=0
Thanks - any idea why OpenCV net is much lower confidence than detect.py even when same resolution? Or is it the same for you?

general.py line 701, function non_max_suppression

# Compute conf
x[:, 5:] *= x[:, 4:5]  # conf = obj_conf * cls_conf

and this is my cpp file,whice get same result compare to detect.py

rchaud99 · 2022-01-17T19:11:31Z

Thank you very much @Hexmagic 😄

github-actions · 2022-02-17T00:13:09Z

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

Wiki – https://github.com/ultralytics/yolov5/wiki
Tutorials – https://docs.ultralytics.com/yolov5
Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

Ultralytics HUB – https://ultralytics.com/hub
Vision API – https://ultralytics.com/yolov5
About Us – https://ultralytics.com/about
Join Our Team – https://ultralytics.com/work
Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

godlovehill · 2022-04-14T07:16:05Z

@Hexmagic great job! I noticed some blogs but all of them accuracy goes down compared official version, your works really help me a lot! ^_^

fisakhan · 2022-09-06T09:22:04Z

I encounter the following error using net = cv::dnn::readNetFromONNX("yolov5.onnx"); What am I missing or doing wrong?

[ERROR:0@0.003] global /home/opencv_build/opencv/modules/dnn/src/onnx/onnx_importer.cpp (2564) parseShape DNN/ONNX(Shape): dynamic 'zero' shapes are not supported, input 243 [ 0 0 0 51 ]
[ERROR:0@0.003] global /home/opencv_build/opencv/modules/dnn/src/onnx/onnx_importer.cpp (1042) handleNode DNN/ONNX: ERROR during processing node with 1 inputs and 1 outputs: [Shape]:(onnx_node!Shape_70) from domain='ai.onnx'
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.6.0-dev) /home/opencv_build/opencv/modules/dnn/src/onnx/onnx_importer.cpp:1064: error: (-2:Unspecified error) in function 'handleNode'

Node [Shape@ai.onnx]:(onnx_node!Shape_70) parse error: OpenCV(4.6.0-dev) /home/opencv_build/opencv/modules/dnn/src/onnx/onnx_importer.cpp:2565: error: (-215:Assertion failed) !isDynamicShape in function 'parseShape'

Aborted (core dumped)

glenn-jocher · 2022-09-06T11:09:50Z

@fisakhan since this error was generated by cv2 on a working ONNX model you should raise this directly with OpenCV.

mowais2170 · 2022-11-01T09:24:53Z

@fisakhan
export your model using this then try again it will work

python export.py --weights yolov5s.pt --include onnx --simplify

NacerFaraj added the question Further information is requested label Oct 28, 2021

futureflsl mentioned this issue Nov 1, 2021

how to run onnx model in opencv with C++ Code #5425

Closed

1 task

github-actions bot added the Stale label Feb 17, 2022

github-actions bot closed this as completed Feb 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to use the onnx (converted from .pt) in opencv C++ #5378

How to use the onnx (converted from .pt) in opencv C++ #5378

NacerFaraj commented Oct 28, 2021 •

edited

Loading

github-actions bot commented Oct 28, 2021 •

edited by glenn-jocher

Loading

glenn-jocher commented Oct 29, 2021

NacerFaraj commented Oct 29, 2021

glenn-jocher commented Oct 29, 2021

futureflsl commented Nov 1, 2021 •

edited

Loading

NacerFaraj commented Nov 1, 2021

glenn-jocher commented Nov 1, 2021

hilevelhdd commented Nov 2, 2021

PauloMendes33 commented Dec 2, 2021

PauloMendes33 commented Dec 2, 2021

PauloMendes33 commented Dec 3, 2021

glenn-jocher commented Dec 3, 2021 •

edited

Loading

rtrahms commented Dec 28, 2021

rchaud99 commented Jan 17, 2022

Hexmagic commented Jan 17, 2022

glenn-jocher commented Jan 17, 2022

rchaud99 commented Jan 17, 2022

rchaud99 commented Jan 17, 2022

Hexmagic commented Jan 17, 2022

rchaud99 commented Jan 17, 2022 •

edited

Loading

Hexmagic commented Jan 17, 2022 •

edited

Loading

rchaud99 commented Jan 17, 2022

github-actions bot commented Feb 17, 2022 •

edited by glenn-jocher

Loading

godlovehill commented Apr 14, 2022

fisakhan commented Sep 6, 2022

glenn-jocher commented Sep 6, 2022

mowais2170 commented Nov 1, 2022

How to use the onnx (converted from .pt) in opencv C++ #5378

How to use the onnx (converted from .pt) in opencv C++ #5378

Comments

NacerFaraj commented Oct 28, 2021 • edited Loading

github-actions bot commented Oct 28, 2021 • edited by glenn-jocher Loading

Requirements

Environments

Status

glenn-jocher commented Oct 29, 2021

NacerFaraj commented Oct 29, 2021

glenn-jocher commented Oct 29, 2021

futureflsl commented Nov 1, 2021 • edited Loading

NacerFaraj commented Nov 1, 2021

glenn-jocher commented Nov 1, 2021

hilevelhdd commented Nov 2, 2021

PauloMendes33 commented Dec 2, 2021

PauloMendes33 commented Dec 2, 2021

PauloMendes33 commented Dec 3, 2021

glenn-jocher commented Dec 3, 2021 • edited Loading

rtrahms commented Dec 28, 2021

rchaud99 commented Jan 17, 2022

Hexmagic commented Jan 17, 2022

glenn-jocher commented Jan 17, 2022

rchaud99 commented Jan 17, 2022

rchaud99 commented Jan 17, 2022

Hexmagic commented Jan 17, 2022

rchaud99 commented Jan 17, 2022 • edited Loading

Hexmagic commented Jan 17, 2022 • edited Loading

rchaud99 commented Jan 17, 2022

github-actions bot commented Feb 17, 2022 • edited by glenn-jocher Loading

godlovehill commented Apr 14, 2022

fisakhan commented Sep 6, 2022

glenn-jocher commented Sep 6, 2022

mowais2170 commented Nov 1, 2022

NacerFaraj commented Oct 28, 2021 •

edited

Loading

github-actions bot commented Oct 28, 2021 •

edited by glenn-jocher

Loading

futureflsl commented Nov 1, 2021 •

edited

Loading

glenn-jocher commented Dec 3, 2021 •

edited

Loading

rchaud99 commented Jan 17, 2022 •

edited

Loading

Hexmagic commented Jan 17, 2022 •

edited

Loading

github-actions bot commented Feb 17, 2022 •

edited by glenn-jocher

Loading