Ill-defined distortion model found by calibrateCamera

[lpea]

Hi everyone,

The calibrateCamera() function can return distortion coefficients k1, k2... k6 that lead to a distortion model that is ill-defined. The denominator of the radial distortion part (1 + k4 r^2 + k5 r^4 + k6 r^6) can have roots r_i such that r_i^2 = ((u-cx)/fx)^2 + ((v-cy)/fy)^2 for some real-valued image points coordinates (u, v) within the image boundaries.

As a consequence, all image points located in the vicinity of this ellipsis may be attributed an incorrect value by the remap function, as the rectification maps will contain invalid coordinates.

This was observed after calibrating with a set of images in which the chessboard was placed mostly near the image center, so my guess is that the lack of constraints further from the center could explain why the optimization could converge towards such a solution. A work around is to check at the end of the calibration process that the radial distortion model is well-defined for every image pixel coordinates − i.e. compute the roots of the polynomial at the denominator.

My question is: when the rational distortion model is used, shouldn't the optimization be constrained in a way that prevents convergence towards such ill-defined sets of coefficients?

Here are the parameters found:

	fx:530.416 fy:530.429 cx:511.626 cy:383.541
	k1:22.4427 k2:-18.5172 p1:0.000102929 p2:0.000468845 k3:-4.44312 k4:22.8537 k5:-10.515 k6:-12.7093
	width:1024 height:768

Steps to reproduce

I can provide calibration images upon request.

System information (version)

• OpenCV => 4.1.0
• Operating System / Platform => Linux 64 Bit
• Compiler => GCC 8.3.1

[alalek]

set of images in which the chessboard was placed mostly near the image center

This is incomplete input for calibrateCamera(). For best results you should cover all parts of camera frame including sides and corners.

Check reprojection error. If it is minimal then algo works "correctly" with provided constraints(input).
Detection of lack of constraints is currently out of scope.

[lpea]

Yes, I understand that with the input data set the calibration cannot be expected to be accurate over the whole field of view, only on the sub-part covered by the calibration images. Still, I think there is a difference between having a model which only approximates the actual distortion over a part of the field of view and a model which exhibits singularities (division by zero).

Also, except for very low distortion lenses, it is sometimes difficult if not impossible to place a chessboard corner near the image corners, which means that there is always a part of the field of view left unexplored.

Then I agree that with a good acquisition procedure (i.e. enough constraints) this should not happen.

Detection of lack of constraints is currently out of scope.

Got it, we can live with it and implement our own checks.