Problem using initUndistortRectifyMap and remap

[matthewc84]

I am trying to undistort an image that was captured using a fisheye lens on an ESP-32 CAM. I am using a tutorial here. Even though I am following the steps, my images appear like the incorrect ones in the example (copied/pasted below). The tempByteArray is a JPG image I am receiving as a raw byte[] stream. The camMatrix and distCoeff are gathered from calibrating the camera with the checkerboard pattern, like in the example. I have also used this same camera and the python code shown in the example, and it works perfectly. The issue is somewhere in the OpenCVForunity code I am using. Any ideas?

`
camMatrix = new Mat(3, 3, CvType.CV_64FC1);
camMatrix.put(0, 0, 1225);
camMatrix.put(0, 1, 0);
camMatrix.put(0, 2, 640);
camMatrix.put(1, 0, 0);
camMatrix.put(1, 1, 1225);
camMatrix.put(1, 2, 360);
camMatrix.put(2, 0, 0);
camMatrix.put(2, 1, 0);
camMatrix.put(2, 2, 1.0f);

        distCoeffs = new MatOfDouble(0.6983763633559256, -7.92845301235481, 66.48311646581958, -176.59573201741406);

    void UndistortImage(byte[] tempByteArray)
    {
        Mat buff = new Mat(1, tempByteArray.Length, CvType.CV_8UC1);
        Utils.copyToMat<byte>(tempByteArray, buff);
        
        Mat img_result = Imgcodecs.imdecode(buff, Imgcodecs.IMREAD_COLOR);
        Mat outMat = new Mat();
        Mat eye = Mat.eye(3, 3, CvType.CV_64FC1);
        Mat Map1 = new Mat();
        Mat Map2 = new Mat();
        OpenCVForUnity.Calib3dModule.Calib3d.initUndistortRectifyMap(camMatrix, distCoeffs, eye, camMatrix, new Size(1280, 720), CvType.CV_16SC2, Map1, Map2);
        Debug.Log(Map1.dump());
        OpenCVForUnity.ImgprocModule.Imgproc.remap(img_result, outMat, Map1, Map2, Imgproc.INTER_LINEAR, Core.BORDER_CONSTANT);
        OpenCVForUnity.UnityUtils.Utils.matToTexture2D(outMat, rawVideoTexturesRGBA, false, 0, true);
    }`

I have also tried OpenCVForUnity.Calib3dModule.Calib3d.undistort with the same result.

[EnoxSoftware]

I had already created a ported example of the python code for the tutorial you tried, as I had a similar question earlier.
The attached scene is an example of correcting distortion of a fisheye lens camera image.
Please try it.
FisheyeUndistortImageExample.zip

[matthew-corbett]

Thank you! I completely missed the APIs for the fisheye package. I was using Calib3D.undistort as shown above. I didn't even see the "fisheye_" APIs in the docs. For anyone else with this problem, the code below fixed it!

OpenCVForUnity.Calib3dModule.Calib3d.fisheye_undistortImage(img_result, outMat, camMatrix, distCoeffs, camMatrix, size);