DNN graph fusion with shared nodes

[dkurt]

Pull Request Readiness Checklist

For now, nodes from matched pattern are removed during the matching process so if nodes are used in similar subgraph, they cannot be found.

required for #24397

Merge with extra: opencv/opencv_extra#1115

A part from model_name with two Resize subgraphs with shared nodes:

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

• I agree to contribute to the project under Apache 2 License.
• To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
• The PR is proposed to the proper branch
• There is a reference to the original bug report and related work
• There is accuracy test, performance test and test data in opencv_extra repository, if applicable
  Patch to opencv_extra has the same branch name.
• The feature is well documented and sample code can be built with the project CMake

[fengyuentau]

Two questions come to my mind:

1. We do not have test cases for layer simplification (or fusion) yet. How do we know the changes in modules/dnn/src/graph_simplifier.cpp will not downgrade current performance, except thinking real hard in our brain?
2. Since this shared constant node issue is brought up, what about the case where a single Const node is used in several nodes, some of which can be fused (thus this single Const node is removed), while the others of which still need this Const node for input?

[dkurt]

@fengyuentau, let me try to answer your questions:

1. We do not have test cases for layer simplification (or fusion) yet. How do we know the changes in modules/dnn/src/graph_simplifier.cpp will not downgrade current performance, except thinking real hard in our brain?

For now all the tests relies that fusion is enabled and we do not check what will happen if fusion disabled. Probably, some of the tests will still work thanks to improvements in ONNX. And there would be a performance questions, that's right.

2. Since this shared constant node issue is brought up, what about the case where a single Const node is used in several nodes, some of which can be fused (thus this single Const node is removed), while the others of which still need this Const node for input?

Good point. Moreover, it's not only about Const but any type of nodes. For example, Shape which can be obtained once but reused in different subgraphs. Let's switch this PR to draft so I can try alternative solutions with consumers counting.

[fengyuentau]

Also worth mentioning is that some operators, such as Add and Mul, conform to commutative law (swapping input A and B does not yield different result). Sometimes the Const input can be input A, and the other times input B. Current graph simplifier cannot deal with this case unless we write down all the possible patterns. If there are N Add or Mul operators in a subgraph and they have one of the input to be constant, the combination would be 2^N if we want to match all possible subgraphs.

I have studied your code in graph_simplifier.cpp, especially match, but did not come up with a good solution without making a lot changes.

[fengyuentau]

For now all the tests relies that fusion is enabled and we do not check what will happen if fusion disabled. Probably, some of the tests will still work thanks to improvements in ONNX. And there would be a performance questions, that's right.

The performance part can be tricky, but at least we can add some fusion tests checking whether the net is simplified to the target form? For example, in most of the cases, if we provide the exact subgraph to be fused, the simplified net should have only one node with matching node type.

[dkurt]

@fengyuentau, I will try to solve Add/Mul commutative nodes matching in a separate PR.

[fengyuentau]

👍