Calibrate camera without moving external parameters

[thewoz]

It is then possible to add a flag to do not move the external parameter during the optimisation in the calibrate camera function?

thanks

[alalek]

Did you try these flags?

Please provide more information about use case.

[thewoz]

Yes, I have tried those flags.
There is no flag to fix the external parameters.

I try to explain me better.

In the calibrate camera function, the algorithm tries to find the best internal (K) and external parameters (R|T) to minimize the reprojection error between the 2D and 3D points.

In my case, I know precisely the external parameters, but I do not know the internal one.
So when I call that function the solver, since I can not fix the external parameters, will also move the external one.
In this way the internal parameters found will be suitable for the external parameters that the algorithm found but not for the right external parameters that I know that are good for sure.

so it will be helpful to add two flags: one to fix the translation and one to fix the rotation.

[thezane]

I'm interested in implementing the two flags in this request but would like some guidance.

After checking out the helper function cvCalibrateCamera2Internal(), I think a different calibration approach than the one available in OpenCV is needed. Currently, OpenCV computes intrinsic parameters first and uses them to compute extrinsic parameters (i.e. Zhang's method).

However, this feature needs intrinsic parameters to be computed from user-supplied extrinsic parameters. Perhaps Tsai's method can be used as it computes extrinsic parameters before intrinsic parameters?

Is this intuition reasonable?

[thewoz]

Hi thezane,
here if you see there is the "initialize extrinsic parameters" (point 2) before the "run the optimization" step (point 3).
What I proposed was the possibility in case of skipping the point 2.

Indeed the calibration algorithm is based on the idea that by exploiting the correspondences between the 3D and 2D positions it is possible to find the internal parameters of the system. To reach this goal the algorithm needs also to find the external parameters (i.e. the RT matrix).

My point is, for example, that moving the camera's Z value has the same effect as changing the omega (focal length) of the camera. For this reason, if one knows exactly the real position of the camera (the RT matrix) it would be useful to be able to calibrate the camera iby fixing the external parameters.

Maybe adding some flags here. One to fix the translation and one to fix the rotation of the camera.

[thezane]

Thanks thewoz.

After stepping through the helper function cvCalibrateCamera2Internal() with the debugger, I think your proposal will work. What I have to do is:
(1) Initialize the solver params to user-provided extrinsic parameters.
(2) Skip point 2.
(3) Set the solver mask at the extrinsic parameter indices to 0.

The solver should then optimize the params without changing the initial extrinsic parameters. This is how user-provided focal lengths are fixed during calibration process via flags (CALIB_FIX_FOCAL_LENGTH | CALIB_USE_INTRINSIC_GUESS).

I'll try and open a PR with unit tests soon.

[thezane]

@thewoz Just an update. I implemented the ability to fix the user-provided rotation and/or translation. After some testing, I noticed fixing just the rotation or just the translation frequently leads to a high projection error. Thus, I'm going to recommend we only allow fixing the extrinsic parameters as a whole. Would you be ok with this?

If you're curious, the reason why fixing just the rotation leads to a high projection error is the camera matrix is estimated first without using the user provided rotation:

opencv/modules/calib3d/src/calibration.cpp

Line 1589 in b67c0e5

cvInitIntrinsicParams2D( &_matM, &m, npoints, imageSize, &matA, aspectRatio );

Then translation is computed using the estimated camera matrix:

opencv/modules/calib3d/src/calibration.cpp

Line 1045 in b67c0e5

CV_IMPL void cvFindExtrinsicCameraParams2( const CvMat* objectPoints,

This leads to a high initial projection error as the initial guess for camera and translation parameters are incompatible with the user provided rotation matrix. Then the LM method gets stuck in a local minimum with a high projection error. Similarly when just fixing the translation parameters.

However, fixing the extrinsic parameters as a whole leads to a good estimate for the camera matrix as only the initial guess for the camera matrix is off. In fact, I found passing and fixing a good 3x3 rotation and 3x1 translation leads to a lower projection error than if you didn't pass them in at all.

If you think fixing just the rotation or just the translation is crucial, then invasive changes are needed to the initial camera matrix computation to ensure it's compatible with the provided rotation or translation:

opencv/modules/calib3d/src/calibration.cpp

Line 1263 in b67c0e5

CV_IMPL void cvInitIntrinsicParams2D( const CvMat* objectPoints,

[thewoz]

@thezane
Hi zane sorry for the late replay!
And great job.. thanks!
I agree with you that the best is to allow only to fixing the extrinsic parameters as a whole.