Update DNN Optical Flow sample with RAFT model

[dkurt]

Describe the feature and motivation

Originally, sample https://github.com/opencv/opencv/blob/4.x/samples/dnn/optical_flow.py created for FlowNet v2. Recently, @Aser-Abdelfatah and @fengyuentau enabled support of RAFT model. Sample should work with both models.

Additional context

No response

[parthasdey2304]

would you please assign this to me

[fengyuentau]

@parthasdey2304 Feel free to submit pull reuqests.

[kayo09]

This is the summary to the issue:
https://arxiv.org/pdf/2003.12039.pdf

per pixel motion-->optical flow
Tradeoff: data and regularization
https://github.com/princeton-vl/RAFT

What is RAFT:
Recurrent All-Pairs Field Transforms: deep network architecture.

1. A feature encoder that extracts per-pixel features from both input images, along with a context encoder that extracts features from Only I1
2. A correlation layer which constructs a 4D WxHxWxH correction volume by taking the inner product of all pairs of feature vectors. The last 2-D of the 4D volume are pooled at multiple scales to construct a set of multi-scale volumes
3. An update operator which recurrently updates optical flow by using the current estimate to look up values from the set of correlation volumes

I don't quite understand what exactly needs to be updated. Mind my stupidness.

[kannishk]

This is not resolved yet, right?

[vagish-yadav]

is this issue still open to be resolved ??

[usyntest]

I am looking into this issue and have some questions regarding this, if any of the questions sound stupid tell me to do more research.

@dkurt

1. The RAFT model is implemented in PyTorch, can I use it in running this example?
2. Should the example code have a choice in using FlowNet v2 or RAFT, use RAFT, or use both of them for estimating the optical flow?

[dkurt]

@usyntest, see https://github.com/opencv/opencv/pull/24092/files#diff-94002668859ca5cfccd65220cb1812650fa91d5275b3ba995655f0e792d35b68 there is a complete test which runs RAFT already. We also expect that sample can handle run the model end-to-end.

[usyntest]

@dkurt I think this code that I added can run the RAFT model end-to-end and I didn't remove the FlowNet v2, this sample can run both of them all you have to do is pass a flag to choose which one to run. Please suggest if anything else is to be done. If not I think we can close this issue.