Bundle Adjuster in the Stitcher module is not converging with a high number of images

[opencv-pushbot]

Transferred from http://code.opencv.org/issues/3657

|| Iacopo Masi on 2014-04-19 14:49
|| Priority: High
|| Affected: 2.4.8 (latest release)
|| Category: stitching
|| Tracker: Bug
|| Difficulty: Medium
|| PR: 
|| Platform: x64 / Linux

Bundle Adjuster in the Stitcher module is not converging with a high number of images

I found out that the *Bundle Adjuster module* in the recent *Stitcher module* is not converging if the software is provided with more than 20/30 images. The same bug is also reported from other users here [1]. The stitcher works for 5 to 10 images but not with a higher number. I found out that the problem is in the camera parameters refinement process (Bundle Adjuster). In particular I guess that the unbounded for loop [3] is never ending since the break sentence in this case is never reached. This only affect the camera parameters refinement module of the stitcher, the matcher and compositor work fine.

I guess that this break [2] in these cases where you have a high number of images >= 20 is _NEVER_ reached and if the module is forced to exit it reports nan in rotation matrices.
Please see an example attached to this ticket issue and a README that point on the dataset that has been used to fix the bug.


[1] http://answers.opencv.org/question/12713/nans-in-stitching_detailed-example-due-to-camera/
[2] https://github.com/Itseez/opencv/blob/master/modules/stitching/src/motion_estimators.cpp#L229
[3] https://github.com/Itseez/opencv/blob/master/modules/stitching/src/motion_estimators.cpp#L218

History

Iacopo Masi on 2014-04-22 14:40

In particular today I discovered that the _err mat present in [4] after 3 or 4 iterations converges to a matrix full of nans (Not A Number) and this causes the infinite loop.

Iacopo Masi wrote:
> I found out that the *Bundle Adjuster module* in the recent *Stitcher module* is not converging if the software is provided with more than 20/30 images. The same bug is also reported from other users here [1]. The stitcher works for 5 to 10 images but not with a higher number. I found out that the problem is in the camera parameters refinement process (Bundle Adjuster). In particular I guess that the unbounded for loop [3] is never ending since the break sentence in this case is never reached. This only affect the camera parameters refinement module of the stitcher, the matcher and compositor work fine.
> 
> I guess that this break [2] in these cases where you have a high number of images >= 20 is _NEVER_ reached and if the module is forced to exit it reports nan in rotation matrices.
> Please see an example attached to this ticket issue and a README that point on the dataset that has been used to fix the bug.
> 
> 
> [1] http://answers.opencv.org/question/12713/nans-in-stitching_detailed-example-due-to-camera/
> [2] https://github.com/Itseez/opencv/blob/master/modules/stitching/src/motion_estimators.cpp#L229
> [3] https://github.com/Itseez/opencv/blob/master/modules/stitching/src/motion_estimators.cpp#L218

[4] https://github.com/Itseez/opencv/blob/master/modules/stitching/src/motion_estimators.cpp#L222

Alexander Smorkalov on 2014-04-30 19:06

-   Target version changed from 2.4.9 to 2.4.10

Rui Marques on 2014-06-11 11:51

Hello Iacopo Masi,

I am a bit confused by your last statement, you say "has been used to fix the bug.". Do you mean "to find the bug"? 
I looked briefly at your code and seems like a test, not a fix, am I missing something?