Feature request within USAC framework: allow setting PolishingMethod to NonePolisher

[javierttgg]

Within USAC framework, a final refinement of the best estimated model is always done using all the inliers found. However, in the case that only the inliers need to be known (or when a non-refined model suffices), this last refinement is not needed.

The motivation behind this resides in infering the previous quantities (inliers and/or non-refined model) faster. The increase in execution speed can be significant. For instance, when solving the Perspective-n-Point problem, USAC uses DLS (J. Hesch, 2011) as the non-minimal solver which has significant execution time w.r.t. the number of points (e.g. see Fig. 2 of (S. Urban, 2016)). In this specific case, this also would allow to consider different PnP methods for the last refinement.

Within my little knowledge of USAC's implementation, I think that this could be done by allowing to set PoilishingMethod to NonePolisher (defined here), since this is checked when deciding to polish the model:

opencv/modules/calib3d/src/usac/ransac_solvers.cpp

Line 354 in 456137f

if (params->getFinalPolisher() != PolishingMethod::NonePolisher) {

Which I believe the previous condition will always hold, since the Polishing method is hard coded here:

opencv/modules/calib3d/src/usac/ransac_solvers.cpp

Lines 606 to 607 in 456137f

	// apply polisher for final RANSAC model
	PolishingMethod polisher = PolishingMethod ::LSQPolisher;

and seems to not be reassigned after.

Thanks in advance,

[asmorkalov]

/cc @ivashmak

[ivashmak]

@javierttgg your concern regarding PnP DLS optimization is totally understandable, it is indeed very expensive. Moreover, the final LSQ (least squares) on all-inliers is done multiple times (3x for PnP) if last LSQ optimization was succesfull. So it's true that in simple cases final optimization could be even more costly than whole RANSAC. Nevertheless, I think accuracy should be prioritized, and for homography and epipolar geometry the final LSQ is quite fast and does not require changes.

However, I disagree with you regarding that last refinement is not needed if only inliers are required. When the final optimization is done, it often happens that some points become outliers, and since the model is more accurate you can have new inliers as well.

I would suggest, that we can put PolishingMethod feature and number of final least squares iterations in UsacParams structure defined here: https://github.com/opencv/opencv/blob/4.x/modules/calib3d/include/opencv2/calib3d.hpp#L558
So, users can set extended parameters there.

I am working now on USAC update in the pull request: #23078
I can make these avaliable there. And you can also write your other suggestions.

[javierttgg]

@ivashmak Thanks a lot for your response. The USAC's implementation is being really useful in my application, thank you :)

I see your point regarding accuracy. Indeed, I hadn't notice that USAC refines the outliers after the non-minimal estimation. However, I feel that in such cases (when there are still outliers to be pruned) refining with all the so-far inliers may deteriorate the so-far best model, since non-minimal solvers like DLS would minimize the squared error of those outliers. This of course won't usually happen since clear outliers will be pruned in advance.

I think your suggestion is great since it would increase the flexibility from the user's pov. Particularly, if the suggestion about allowing NonePolisher is also deemed, then this would allow the user to define its own final refinement and/or leverage rough (and fast) model estimates (used e.g. in the literature for uncertainty propagation [Vakhitov, 2021]).

[ivashmak]

@javierttgg glad to hear this!

The RANSAC model found from a minimal number of points could be a noisy, therefore, it does not really contain outliers but rather noisy inliers. Consequently, polishing with all inliers, in most of cases finds more inliers and prunes noisy inliers that don’t fit to model anymore.

Alright, then it is settled. I will extend ‘UsacParams’ by adding final polisher option together with number of final polisher iterations.

[javierttgg]

@ivashmak indeed I wasn't right, thank you for the correction.

Great about exposing these options in UsacParams!

and thank you for your time :)