Skip to content

Commit

Permalink
Merge pull request #23528 from WanliZhong:issue23278
Browse files Browse the repository at this point in the history 
DNN/CUDA: make 'abcd op 1b11' broadcast eltwise operator support cuda
  • Loading branch information
@asmorkalov
asmorkalov committed Apr 24, 2023
2 parents aa57833 + e436029 commit e3e1f70
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 5 deletions.
14 changes: 12 additions & 2 deletions modules/dnn/perf/perf_layer.cpp
View file
Expand Up @@ -66,8 +66,13 @@ struct Layer_NaryEltwise : public TestBaseWithParam<tuple<Backend, Target> >

if (!isRef && backendId == DNN_BACKEND_CUDA)
{
if (a_shape != b_shape)
throw SkipTestException("The test is skipped because inputs with different shapes are not supported.");
if (a_shape.size() != b_shape.size())
throw SkipTestException("The test is skipped because inputs with different shape size are not supported.");

for(int i = 0; i < a_shape.size(); i++)
if (a_shape[i] != b_shape[i] && a_shape[i] != 1 && b_shape[i] != 1)
throw SkipTestException("The test is skipped because inputs are not supported.");

if (nary_eltwise_cuda_deny_ops.find(op) != nary_eltwise_cuda_deny_ops.end())
throw SkipTestException("The operator '" + op + "' is skipped because is not support with cuda currently.");
}
Expand Down Expand Up @@ -215,6 +220,11 @@ PERF_TEST_P_(Layer_NaryEltwise, NHWC_C)
test_layer({N, H, W, C}, {1, C}, "sum");
}

PERF_TEST_P_(Layer_NaryEltwise, NHWC_H)
{
test_layer({N, H, W, C}, {1, H, 1, 1}, "sum");
}

PERF_TEST_P_(Layer_Slice, YOLOv4_tiny_1)
{
const int inputShape[4] = {1, 64, 104, 104};
Expand Down
11 changes: 8 additions & 3 deletions modules/dnn/src/layers/nary_eltwise_layers.cpp
View file
Expand Up @@ -673,12 +673,17 @@ class NaryEltwiseLayerImpl CV_FINAL : public NaryEltwiseLayer
{
auto context = reinterpret_cast<csl::CSLContext*>(context_);

auto input_wrapper = inputs[0].dynamicCast<CUDABackendWrapper>();
auto input_0_shape = inputs[0].dynamicCast<CUDABackendWrapper>()->getShape();
for (int i = 1; i < inputs.size(); i++)
{
auto from_wrapper = inputs[i].dynamicCast<CUDABackendWrapper>();
if (input_wrapper->getShape() != from_wrapper->getShape())
auto input_i_shape = inputs[i].dynamicCast<CUDABackendWrapper>()->getShape();
if (input_0_shape.size() != input_i_shape.size())
return Ptr<BackendNode>();
// check if the shape can be supported by `eltwise_ops.cu`, or return the default BackendNode
for (int j = 0; j < input_0_shape.size(); j++)
if (input_0_shape[j] != input_i_shape[j] &&
input_0_shape[j] != 1 && input_i_shape[j] != 1)
return Ptr<BackendNode>();
}

cuda4dnn::EltwiseOpType op_ = cuda4dnn::EltwiseOpType::SUM;
Expand Down

6 comments on commit e3e1f70

@peters
Copy link
Contributor

@peters peters commented on e3e1f70 Apr 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asmorkalov This commit breaks yolov7-tiny inference on CUDA (12.1.1) and CUDNN (8.9.0)

[ INFO:1@1.617] global op_cuda.cpp:80 initCUDABackend CUDA backend will fallback to the CPU implementation for the layer "onnx_node!Pow_144" of type NaryEltwise

[ INFO:1@1.618] global op_cuda.cpp:80 initCUDABackend CUDA backend will fallback to the CPU implementation for the layer "onnx_node!Pow_159" of type NaryEltwise

[ INFO:1@1.618] global op_cuda.cpp:80 initCUDABackend CUDA backend will fallback to the CPU implementation for the layer "onnx_node!Pow_174" of type NaryEltwise

[ INFO:0@1.625] global op_cuda.cpp:80 initCUDABackend CUDA backend will fallback to the CPU implementation for the layer "onnx_node!Pow_144" of type NaryEltwise

[ INFO:0@1.625] global op_cuda.cpp:80 initCUDABackend CUDA backend will fallback to the CPU implementation for the layer "onnx_node!Pow_159" of type NaryEltwise

[ INFO:0@1.626] global op_cuda.cpp:80 initCUDABackend CUDA backend will fallback to the CPU implementation for the layer "onnx_node!Pow_174" of type NaryEltwise

OpenCV(4.7.0-dev) Error: Assertion failed (inShape1[i] == outShape[i]) in eltwise_op, file /build/build_cuda/3p/opencv/linux-x64/ubuntu22.04/Debug/modules/dnn/src/cuda/eltwise_ops.cu, line 195
OpenCV(4.7.0-dev) Error: Assertion failed (inShape1[i] == outShape[i]) in eltwise_op, file /build/build_cuda/3p/opencv/linux-x64/ubuntu22.04/Debug/modules/dnn/src/cuda/eltwise_ops.cu, line 195
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.7.0-dev) /build/build_cuda/3p/opencv/linux-x64/ubuntu22.04/Debug/modules/dnn/src/cuda/eltwise_ops.cu:195: error: (-215:Assertion failed) inShape1[i] == outShape[i] in function 'eltwise_op'

@asmorkalov
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WanliZhong could you take a look?

@WanliZhong
Copy link
Member

@WanliZhong WanliZhong commented on e3e1f70 Apr 25, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I will check it tomorrow. Seems like current cuda op can't support some cases. I need to modify the .cu file and make it support more broadcast cases

@WanliZhong
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@peters @asmorkalov I make a test to check which cases we can't support currently. More details in #23556. I will try to fix runtime error first.

@WanliZhong
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it may fix this problem in with PR #23560

@peters
Copy link
Contributor

@peters peters commented on e3e1f70 Apr 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WanliZhong The problem with inference has now been fixed in PR #23560. Thank you! :)

Please sign in to comment.