Support Yolov4

[l-bat]

Merge with extra: opencv/opencv_extra#753
resolves: #17148

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

• I agree to contribute to the project under OpenCV (BSD) License.
• To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
• The PR is proposed to proper branch
• There is reference to original bug report and related work
• There is accuracy test, performance test and test data in opencv_extra repository, if applicable
  Patch to opencv_extra has the same branch name.
• The feature is well documented and sample code can be built with the project CMake

force_builders=Custom,Custom Win,Custom Mac
build_image:Custom=ubuntu-openvino-2020.1.0:16.04
build_image:Custom Win=openvino-2020.2.0
build_image:Custom Mac=openvino-2020.2.0

test_modules:Custom=dnn,python2,python3,java
test_modules:Custom Win=dnn,python2,python3,java
test_modules:Custom Mac=dnn,python2,python3,java

buildworker:Custom=linux-1
# disabled due high memory usage: test_opencl:Custom=ON
test_opencl:Custom=OFF
test_bigdata:Custom=1
test_filter:Custom=*

Performance tests
CPU: Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz

Net	Backend	branch 3.4	branch yolov4
TinyYoloVOC	DNN_BACKEND_INFERENCE_ENGINE	23.29	23.15
TinyYoloVOC	DNN_BACKEND_OPENCV	35.24	35.29
YoloVOC @ 416x416	DNN_BACKEND_INFERENCE_ENGINE	88.42	88.54
YoloVOC @ 416x416	DNN_BACKEND_OPENCV	129.50	130.03
YOLOv3 @ 416x416	DNN_BACKEND_INFERENCE_ENGINE	200.76	200.93
YOLOv3 @ 416x416	DNN_BACKEND_OPENCV	273.93	274.48
YOLOv4 @ 416x416	DNN_BACKEND_INFERENCE_ENGINE	-	537.48
YOLOv4 @ 416x416	DNN_BACKEND_OPENCV	-	712.38

CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz x8

	DNN_BACKEND_OPENCV	DNN_BACKEND_INFERENCE_ENGINE
TinyYoloVOC	24.34ms	17.68ms
YoloVOC @ 416x416	92.22ms	68.46ms
YOLOv3 @ 416x416	208.74ms	155.62ms
YOLOv4 @ 416x416	471.89ms	369.82ms

import cv2 as cv

net = cv.dnn_DetectionModel(yolov4.cfg', 'yolov4.weights)
net.setInputSize(608, 608)
net.setInputScale(1.0 / 255)
net.setInputSwapRB(True)

frame = cv.imread('example.jpg')

with open('coco.names', 'rt') as f:
    names = f.read().rstrip('\n').split('\n')

classes, confidences, boxes = net.detect(frame, confThreshold=0.1, nmsThreshold=0.4)
for classId, confidence, box in zip(classes.flatten(), confidences.flatten(), boxes):
    label = '%.2f' % confidence
    label = '%s: %s' % (names[classId], label)
    labelSize, baseLine = cv.getTextSize(label, cv.FONT_HERSHEY_SIMPLEX, 0.5, 1)
    left, top, width, height = box
    top = max(top, labelSize[1])
    cv.rectangle(frame, box, color=(0, 255, 0), thickness=3)
    cv.rectangle(frame, (left, top - labelSize[1]), (left + labelSize[0], top + baseLine), (255, 255, 255), cv.FILLED)
    cv.putText(frame, label, (left, top), cv.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 0))

cv.imshow('out', frame)

Mish activation function
Mish activation function in origin paper
𝑓(𝑥) = 𝑥 * 𝑡𝑎𝑛ℎ(softplus(𝑥)) where softplus (𝑥) = ln(1 + 𝑒xp(𝑥))

Darknet implementation softplus

float softplus(float x, float threshold = 20) {
    if (x > threshold) return x;              // too large
    else if (x < -threshold) return expf(x);  // too small
    return logf(expf(x) + 1);
}

We can use the original softplus function for darknet.
Proof:

1. x < -20: let's compare ln (1 + exp( 𝑥 )) with exp(x). exp (x) - > 0 and ln(1 + exp(x)) - > 0 when x - > -infinity. The maximum difference between ln (1 + exp ( 𝑥 )) and x at the maximum value of x (x=-20). exp (-20)= 2.061153622438558e-09 vs ln(1 + exp (-20)) = 2.0611536900435727e-09. both values are very close.

2. x > 20: let's compare ln(1 + 𝑒xp(𝑥)) with x. ln(1 + 𝑒xp(𝑥)) -> ln(𝑒xp(𝑥)) = x at X->infinity
   The maximum difference between ln (1 + exp( 𝑥 )) and x with a minimum of x (x=20). ln(1 + exp(20)) = 20.000000002061153. So the difference is lower than 10e-8.

[dkurt]

Can you please describe what "before" and "after" means?

[dkurt]

Yolov3 | OpenCV | 17210.52 | 14720.49

Have you used perf_dnn for bahcnmarking? Seems incorrect for me

[l-bat]

Can you please describe what "before" and "after" means?

“Before” means the results of running performance tests before this PR changes (without Yolov4 support)

[l-bat]

Yolov3 | OpenCV | 17210.52 | 14720.49

Have you used perf_dnn for bahcnmarking? Seems incorrect for me

Yes, I used perf_dnn

[dkurt]

👍 Well done!

[rgov]

Great to see that this is in the 3.4.11 milestone. Will this reach the 4.x branch as well?

[l-bat]

Thank you for your interest @rgov, @wooseokyourself! These PR changes will be merged into master branch next week.

[dkurt]

@sqiprasanna, please read all the messages carefully - you're trying to use OpenCV 4.2.0. We just merged the changes yesterday.

[AlexeyAB]

@l-bat @dkurt
Is Yolov4 more than 2 times slower than Yolov3 due to MISH-activation?

[sqiprasanna]

@l-bat Thanks for the reply. I just used this yolov4 for training on my custom dataset. It really worked well on detecting images. I'm more excited to test this model on real-time detection using OpenCV. As I see here you're trying to include yolov4 in OpenCV dnn. I just want to really appreciate the work you've done and would like to see adding yolov4 into OpenCV.

Thank you again for your hard work. Have a nice day.

[dkurt]

@AlexeyAB, it might be, yes. I'd like to recommend to open an issue for YOLOv4 optimization (some research with per-layer performance will be very helpful). Thanks!

[marcusbrito]

Hello,
I had understood that yolov4 was already working on version 3.4.10, but I tried and got the error:

cv2.error: OpenCV(3.4.10) /opencv-3.4.10/modules/dnn/src/darknet/darknet_io.cpp:821: error: (-212:Parsing error) Unsupported activation: mish in function 'ReadDarknetFromCfgStream'

I'm buildin OpenCV 3.4.10 from source on a docker container. I'm cloning this repository:
https://github.com/Itseez/opencv/archive/3.4.10.zip

I'm not very familiar with this, so can someone please give me a simple answer like "It's not working yet" or "It's already working, you can git clone this link", instead of referencing another one confused thread.

[mshabunin]

It's not working yet, wait for the next release or get latest master or 3.4 branch version and build it manually.

[KyloEntro]

Hi,
Any date for the next release ?

Thanks

[akhilgkrishnan]

good

[dkurt]

@l-bat @dkurt
Is Yolov4 more than 2 times slower than Yolov3 due to MISH-activation?

@AlexeyAB, please take a look at Mish optimization for CPU: #17624