Add RIC method for sparse match interpolation #2367

tsenst · 2019-11-29T16:58:08Z

commit implementation of RIC interpolation method (from "Robust Interpolation of Correspondences for Large Displacement Optical Flow" Yinlin et. al 2017)
integrate RIC interpolation to DenseRLOFOpticalFlow class and functions
add RLOF_EPIC and RLOF_RIC to (sample)optical_flow_estimation project
implement interface to allow appling better edge estimators (e.g. SED) for edge-awar cost map computation that is used in the EdgeAwareInterpolator and RICInterpolator

Update master branch

…n of the maximal number of matches to be interpolated from SHORT_MAX to INT_MAX by substituting short by int types

…_EdgeAwareInterpolator

sparse_match_interpolators EdgeAwareInterpolation - enhance limitatio…

…github.com/tsenst/opencv_contrib into add_robust_interpolation_of_correspondence

…ssfull.

…t_interpolation_of_correspondence

* add test for RIC * add ASSERTION for curr image

* fix warnings

…github.com/tsenst/opencv_contrib into add_robust_interpolation_of_correspondence

alalek

Thank you for contribution!

alalek · 2019-12-01T21:17:16Z

modules/ximgproc/CMakeLists.txt

@@ -1,2 +1,2 @@
 set(the_description "Extended image processing module. It includes edge-aware filters and etc.")
-ocv_define_module(ximgproc opencv_core opencv_imgproc opencv_calib3d opencv_imgcodecs WRAP python java)
+ocv_define_module(ximgproc opencv_core opencv_imgproc opencv_calib3d opencv_imgcodecs opencv_tracking  WRAP python java)


We prefer to keep dependency list minimal - adding tracking here, this would block adding ximgproc as dependency into tracing module.

Do we really need this dependency?

Thank you for that comment. In fact tracking was not needed anymore but the video module which has been changed

modules/ximgproc/src/sparse_match_interpolators.cpp

alalek · 2019-12-01T21:20:43Z

modules/ximgproc/include/opencv2/ximgproc/sparse_match_interpolator.hpp

+    /** @copybrief setK
+     *    @see setK
+     */
+    CV_WRAP virtual int  getK() const = 0;


please remove extra double spaces here and below.

Double spaces have been removed

modules/ximgproc/include/opencv2/ximgproc/sparse_match_interpolator.hpp

alalek · 2019-12-01T21:24:21Z

modules/ximgproc/src/sparse_match_interpolators.cpp

@@ -201,13 +207,13 @@ void EdgeAwareInterpolatorImpl::interpolate(InputArray from_image, InputArray fr
    if(use_post_proc)
        fastGlobalSmootherFilter(src,dst,dst,fgs_lambda,fgs_sigma);

+    costMap = Mat();


costMap.release(); should do the same.

Oh sorry. Got the wrong line of code in line 1306 was the same issue

There are two extra cases:
= Mat(); => .release();

modules/ximgproc/src/sparse_match_interpolators.cpp

alalek · 2019-12-01T21:30:42Z

modules/ximgproc/src/sparse_match_interpolators.cpp

+    CV_Assert(!from_points.empty() && from_points.isVector() &&
+        !to_points.empty() && to_points.isVector() &&
+        from_points.sameSize(to_points));


Avoid using of complex assertion statements - they emits over-complicated unclear messages to Users.

CV_Assert(!from_points.empty() && from_points.isVector()); CV_Assert(!to_points.empty() && to_points.isVector()); CV_Assert(from_points.sameSize(to_points));

Their messages helping to localize parameters mistakes much better.

alalek · 2019-12-01T21:31:18Z

modules/ximgproc/src/sparse_match_interpolators.cpp

+    }
+
+private:
+    inline void swap(float & a, float & b)


Why is not std::swap()?

swap function has been removed

modules/ximgproc/src/sparse_match_interpolators.cpp

* underscore from parameters * substitute parallel_for_() classes with lambda functions * remove complex assertion statements * remove dead code * substitute swap function with std::swap() * ocv_define_module now contains video instead of tracking module

alalek

Thank you for update!

alalek · 2019-12-04T16:32:09Z

modules/ximgproc/src/sparse_match_interpolators.cpp

@@ -201,13 +207,13 @@ void EdgeAwareInterpolatorImpl::interpolate(InputArray from_image, InputArray fr
    if(use_post_proc)
        fastGlobalSmootherFilter(src,dst,dst,fgs_lambda,fgs_sigma);

+    costMap = Mat();


alalek · 2019-12-04T16:34:24Z

modules/ximgproc/src/sparse_match_interpolators.cpp

+        int end = std::min(range.end   * stripe_sz, match_num);
+        nodeHeap q((int)match_num);
+        int num_expanded_vertices;
+        unsigned char* expanded_flag = new unsigned char[match_num];


Please avoid manual memory management. Use cv::AutoBuffer<unsigned char> or std::vector instead.

alalek · 2019-12-04T16:34:55Z

modules/ximgproc/src/sparse_match_interpolators.cpp

+
+    geodesicDistanceTransform(matDistanceMap, costMap);
+
+    g = new vector<node>[match_num];


Please avoid manual memory management.

alalek · 2019-12-04T16:36:34Z

modules/ximgproc/src/sparse_match_interpolators.cpp

+        }
+    });
+
+    //parallel_for_(Range(0, iterCnt), GenerateHypothesis_ParBody(*this, &bestCost[0], outModels, supportCnt, spCnt, spNN, &supportMatchIds[0], &supportMatchDis[0], inputMatches, inLierFlag));


alalek · 2019-12-04T16:37:29Z

modules/tracking/samples/tracker.py

-    exit()
+#if len(sys.argv) != 2:
+#    print('Input video name is missing')
+#    exit()


please rollback unrelated changes here and below

alalek · 2019-12-04T16:38:22Z

modules/ximgproc/include/opencv2/ximgproc/sparse_match_interpolator.hpp

+    /** @copybrief setSuperpixelRuler
+     *    @see setSuperpixelRuler
+     */
+    CV_WRAP virtual float  getSuperpixelRuler() const = 0;


please remove double space

modules/ximgproc/src/sparse_match_interpolators.cpp

alalek · 2019-12-04T16:41:36Z

modules/ximgproc/src/sparse_match_interpolators.cpp

+{
+    RICInterpolatorImpl *eai = new RICInterpolatorImpl();
+    eai->init();
+    return Ptr<RICInterpolatorImpl>(eai);


Memory leak is possible in case of exception in "init".
Don't use raw pointers:

auto eai = makePtr<RICInterpolatorImpl>(); eai->init() return eai;

Thank you for that hint!

* remove double space * use static for inner functions * update create function * modify mem init. to avoid manual memory management

* uncomment paralellization

…github.com/tsenst/opencv_contrib into add_robust_interpolation_of_correspondence

alalek · 2019-12-05T11:53:43Z

modules/ximgproc/src/sparse_match_interpolators.cpp

+        if (m_validSize >= m_size)
+        {
+            CV_Error(CV_StsOutOfRange, " m_validSize >= m_size this problem can be resolved my decreasig k parameter");
+        };


; is not necessary

Has been removed

alalek · 2019-12-05T11:57:56Z

modules/ximgproc/src/sparse_match_interpolators.cpp

+    int *label_row;
+    int *label_row_prev;
+
+#define CHECK(cur_dist,cur_label,cur_cost,prev_dist,prev_label,prev_cost,coef)\


CHECK

Please avoid using common names for macros (see this issue about lowercase check).
Consider adding some prefix/suffix.

change CHECK to CHK_GD

alalek · 2019-12-05T11:59:00Z

modules/ximgproc/src/sparse_match_interpolators.cpp

+    //
+    vector<int> diffPairs(labels.cols*_labels.rows * 4, 0);


Please remove empty comment

alalek · 2019-12-05T12:08:27Z

modules/ximgproc/src/sparse_match_interpolators.cpp

@@ -201,13 +207,13 @@ void EdgeAwareInterpolatorImpl::interpolate(InputArray from_image, InputArray fr
    if(use_post_proc)
        fastGlobalSmootherFilter(src,dst,dst,fgs_lambda,fgs_sigma);

+    costMap = Mat();


There are two extra cases:
= Mat(); => .release();

alalek

Well done 👍

* change CHECK to CHK_GD * remove not necessary ;

tsenst and others added 22 commits July 9, 2019 09:46

Merge pull request #2 from opencv/master

9ca3bf2

Update master branch

Merge pull request #3 from opencv/master

a1f1f3b

Update master branch

* add RIC

d916f10

sparse_match_interpolators EdgeAwareInterpolation - enhance limitatio…

14ff3f4

…n of the maximal number of matches to be interpolated from SHORT_MAX to INT_MAX by substituting short by int types

Merge branch 'add_robust_interpolation_of_correspondence' into update…

c6c1fc7

…_EdgeAwareInterpolator

Merge pull request #4 from tsenst/update_EdgeAwareInterpolator

4c64c17

sparse_match_interpolators EdgeAwareInterpolation - enhance limitatio…

* add RIC citation

5ae94f2

Merge branch 'add_robust_interpolation_of_correspondence' of https://…

e40f58a

…github.com/tsenst/opencv_contrib into add_robust_interpolation_of_correspondence

Merge remote-tracking branch 'upstream/master'

1576a4c

Merge branch 'master' into add_robust_interpolation_of_correspondence

5a69151

* update EdgeAwareInterpolatorImpl

6a735a7

* add intermediate RIC implementation

90c3bfb

* implementation of new paralelization classes. First test where suce…

fde566b

…ssfull.

* update documentation RICInterpolatorImpl and RLOF

3b2f624

Merge remote-tracking branch 'remotes/upstream/master' into add_robus…

589f7b2

…t_interpolation_of_correspondence

* update CMakeLists.txt remove highgui

affa8d2

* add test for RIC * add ASSERTION for curr image

* add cost map interface

1f7c182

* fix internal cost map allocation bug

2364f49

* remove white spaces

63a462f

* fix warnings

* remove white spaces

1efbf7a

* fix warnings

Merge branch 'add_robust_interpolation_of_correspondence' of https://…

834408d

…github.com/tsenst/opencv_contrib into add_robust_interpolation_of_correspondence

*fix compiler warnings

1254687

alalek reviewed Dec 1, 2019

View reviewed changes

tsenst added 4 commits December 3, 2019 14:22

* remove double whitespaces

33bc431

* underscore from parameters * substitute parallel_for_() classes with lambda functions * remove complex assertion statements * remove dead code * substitute swap function with std::swap() * ocv_define_module now contains video instead of tracking module

* remove whitespace endings

f7abbd8

* documentation update

49251ae

* remove double declarations that lead to warnings

3bdce87

alalek reviewed Dec 4, 2019

View reviewed changes

tsenst added 2 commits December 4, 2019 22:59

* unrole tracker.py

c6d2317

* remove double space * use static for inner functions * update create function * modify mem init. to avoid manual memory management

* uncomment parallel_for_ for parallelization

f864dea

tsenst added 2 commits December 5, 2019 10:39

* unrole unwanted changes

e2318e1

* uncomment paralellization

Merge branch 'add_robust_interpolation_of_correspondence' of https://…

a212e2b

…github.com/tsenst/opencv_contrib into add_robust_interpolation_of_correspondence

alalek reviewed Dec 5, 2019

View reviewed changes

alalek approved these changes Dec 5, 2019

View reviewed changes

tsenst added 2 commits December 8, 2019 13:43

* remove empty comment

c9d01ac

* change CHECK to CHK_GD * remove not necessary ;

*documentation remove double space

aef6020

alalek merged commit 40b238b into opencv:master Dec 9, 2019

alalek mentioned this pull request Feb 23, 2023

RICInterpolator error: error appeares when using the gray image as input #3441

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add RIC method for sparse match interpolation #2367

Add RIC method for sparse match interpolation #2367

tsenst commented Nov 29, 2019

alalek left a comment

alalek Dec 1, 2019

tsenst Dec 2, 2019 •

edited

Loading

alalek Dec 1, 2019

tsenst Dec 3, 2019

alalek Dec 1, 2019

tsenst Dec 3, 2019

alalek Dec 4, 2019

tsenst Dec 4, 2019

alalek Dec 5, 2019

alalek Dec 1, 2019

tsenst Dec 3, 2019

alalek Dec 1, 2019

tsenst Dec 3, 2019

alalek left a comment

alalek Dec 4, 2019

alalek Dec 4, 2019

alalek Dec 4, 2019

alalek Dec 4, 2019

alalek Dec 4, 2019

alalek Dec 4, 2019

alalek Dec 4, 2019

tsenst Dec 4, 2019

alalek Dec 5, 2019

tsenst Dec 8, 2019

alalek Dec 5, 2019

tsenst Dec 8, 2019

alalek Dec 5, 2019

tsenst Dec 8, 2019

alalek Dec 5, 2019

alalek left a comment


		geodesicDistanceTransform(matDistanceMap, costMap);

		g = new vector<node>[match_num];

Add RIC method for sparse match interpolation #2367

Add RIC method for sparse match interpolation #2367

Conversation

tsenst commented Nov 29, 2019

alalek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tsenst Dec 2, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alalek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alalek left a comment

Choose a reason for hiding this comment

tsenst Dec 2, 2019 •

edited

Loading