Undistorting the image

[anushl9o5]

Hey, I followed what you mentioned in #22 .

• First I unproject with pinhole parameters.
• Then I project with the calibrated model.

Here are my results, it looks like it gets even more distorted.

Distorted

Undistorted

Pinhole unprojection code

bool Unproject_pinhole(Eigen::Vector2d pix, Eigen::Vector3d* direction, Eigen::Matrix3d K_inv)
{
    Eigen::Vector3d pix_hmg;
    pix_hmg[0] = pix[0];
    pix_hmg[1] = pix[1];
    pix_hmg[2] = 1.0;
    
    *direction = K_inv*pix_hmg;

    return true;
}

[puzzlepaint]

The result indeed clearly looks wrong. The lines which are straight in reality should become straight in the undistorted image.

If the coefficients of K_inv are set correctly for your unprojection function, then I think that it should work. The mistake might be somewhere else. (Side note: I think that it might be helpful to pass objects such as Eigen::Matrix3d by reference rather than by value to ensure that they are not copied.)

So I can only guess. One possible mistake which can result in increasing the distortion rather than removing it would be to swap the roles of the coordinates. The correct algorithm is to use the pixel coordinate that is computed by unprojection and projection to look up the pixel color in the distorted image, and then write this color into the undistorted image at the original pixel coordinate (that was used for unprojection). If instead the original pixel coordinate is used to look up the color in the distorted image, and this color is written into the result image at the distorted coordinate, then the distortion may be increased.

[anushl9o5]

So I can only guess. One possible mistake which can result in increasing the distortion rather than removing it would be to swap the roles of the coordinates. The correct algorithm is to use the pixel coordinate that is computed by unprojection and projection to look up the pixel color in the distorted image, and then write this color into the undistorted image at the original pixel coordinate (that was used for unprojection). If instead the original pixel coordinate is used to look up the color in the distorted image, and this color is written into the result image at the distorted coordinate, then the distortion may be increased.

Thanks a lot, for some reason I was doing the opposite.