DNN: optimize the speed of general Depth-wise

[zihaomu]

Try to solve the issue: #23941

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

• I agree to contribute to the project under Apache 2 License.
• To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
• The PR is proposed to the proper branch
• There is a reference to the original bug report and related work
• There is accuracy test, performance test and test data in opencv_extra repository, if applicable
  Patch to opencv_extra has the same branch name.
• The feature is well documented and sample code can be built with the project CMake

[zihaomu]

@WanliZhong, Can you check if this patch fixes your issue?

[WanliZhong]

I have tested this patch on arm and x86 (use the min value). Palm detection model has many 5x5 depth wise layers. Handpose and person detection models have several 5x5 depth wise layers. So the effect is evident on palm detection model.

Intel chip:

Model	4.7	4.8	this patch
palm detection	5.91ms	12.1 ms	6.41ms
handpose estimation	4.36ms	3.65ms	3.21ms
person detection	12.4ms	11.0ms	9.07ms

M2 chip:

Model	4.7	4.8	this patch
palm detection	8.35ms	18.76 ms	9.44ms
handpose estimation	4.6ms	7.04ms	4.91ms
person detection	10.83ms	14.06ms	10.74ms

[vpisarev]

@asmorkalov, I believe, 4.8.1 should be released as soon as possible with this and a few other fixes that you mentioned (Python-related)

[opencv-alalek]

Related performance tests should be updated to avoid similar incidents in the future.

[zihaomu]

@wanli will be assigned to add such a performance test.

[WanliZhong]

Perf test (use min value) result on MacBook Air M2:

Name of Test	480-1th	481-1th	481-1th vs 480-1th (x-factor)
conv::Conv::(GFLOPS=0.472, K=[5 x 5], IN={1, 32, 96, 96}, OCN=32, G=32, P=[2 x 2], BIAS, OCV/CPU)	3.721	1.921	1.94
conv::Conv::(GFLOPS=0.472, K=[5 x 5], IN={1, 32, 96, 96}, OCN=32, G=32, P=[2 x 2], BIAS, OCV/CPU_FP16)	2.358	1.869	1.26

[zihaomu]

Looks like the default CI has a failure irrelated to this PR.

[asmorkalov]

Perf results for i5-2500K CPU @ 3.30GHz (No AVX2)

                                                                 Name of Test                                                                     4.x       fix       fix
                                                                                                                                               depthwise depthwise depthwise.
                                                                                                                                                  5x5       5x5       5x5
                                                                                                                                                   1         1         1
                                                                                                                                                                       vs
                                                                                                                                                                      4.x
                                                                                                                                                                   depthwise.
                                                                                                                                                                      5x5
                                                                                                                                                                       1
conv::Conv::(GFLOPS=0.472, K=[5 x 5], IN={1, 32, 96, 96}, OCN=32, G=32, P=[2 x 2], BIAS, OCV/CPU)                                               5.111     1.068      4.79

[asmorkalov]

Perf results for ARM v7 (Jetson tk1):

                                                                 Name of Test                                                                     4.x       fix       fix
                                                                                                                                               depthwise depthwise depthwise.
                                                                                                                                                  5x5       5x5       5x5
                                                                                                                                                   1         1         1
                                                                                                                                                                       vs
                                                                                                                                                                      4.x
                                                                                                                                                                   depthwise.
                                                                                                                                                                      5x5
                                                                                                                                                                       1
conv::Conv::(GFLOPS=0.472, K=[5 x 5], IN={1, 32, 96, 96}, OCN=32, G=32, P=[2 x 2], BIAS, OCV/CPU)                                               64.849    18.178      3.57  

[asmorkalov]

👍