Python fisheye.stereoCalibrate: Assertion failed objectPoints.type() #11085

philipptrenz · 2018-03-15T17:56:40Z

System information

opencv-python '3.4.0.12' (installed via pip)
opencv-contrib-python '3.4.0.12' (installed via pip)
Windows 64 Bit
Python 3.6.4 :: Anaconda, Inc.

Detailed description

While cv2.fisheye.calibrate() works with both versions of objp as intented, cv2.fisheye.stereoCalibrate() throws errors. Curiously different ones, in function cv::internal::projectPoints and in function cv::fisheye::stereoCalibrate.

This could be related to #5534

Steps to reproduce

Case 1

objp = np.zeros((1, CHECKERBOARD[0]*CHECKERBOARD[1], 3), np.float32)
objp[0,:,:2] = np.mgrid[0:CHECKERBOARD[0], 0:CHECKERBOARD[1]].T.reshape(-1, 2)
...

K_left = np.zeros((3, 3))
D_left = np.zeros((4, 1))

K_right = np.zeros((3, 3))
D_right = np.zeros((4, 1))

R = np.zeros((1, 1, 3), dtype=np.float64)
T = np.zeros((1, 1, 3), dtype=np.float64)

rms, _, _, _, _, _, _ = \
        cv2.fisheye.stereoCalibrate(
            [objp]*len(imgpoints_left),
            imgpoints_left,
            imgpoints_right,
            K_left,
            D_left,
            K_right,
            D_right,
            (1280,800),
            R,
            T,
            flags,
            (cv2.TERM_CRITERIA_EPS+cv2.TERM_CRITERIA_MAX_ITER, 30, 1e-6)
        )

OpenCV Error: Assertion failed (!objectPoints.empty() && objectPoints.type() == (((6) & ((1 << 3) - 1)) + (((3)-1) << 3))) in cv::internal::projectPoints, file C:\projects\opencv-python\opencv\modules\calib3d\src\fisheye.cpp, line 1168
Traceback (most recent call last):
  File "C:/Users/treph/pycamdriver/stereo_calibration.py", line 239, in <module>
    calibrate(reader, cam_ids, CHECKERBOARD, SQUARE_SIZE)
  File "C:/Users/treph/pycamdriver/stereo_calibration.py", line 206, in calibrate
    (cv2.TERM_CRITERIA_EPS+cv2.TERM_CRITERIA_MAX_ITER, 30, 1e-6)
cv2.error: C:\projects\opencv-python\opencv\modules\calib3d\src\fisheye.cpp:1168: error: (-215) !objectPoints.empty() && objectPoints.type() == (((6) & ((1 << 3) - 1)) + (((3)-1) << 3)) in function cv::internal::projectPoints

Case 2

objp = np.zeros((CHECKERBOARD[0]*CHECKERBOARD[1],3), np.float32)
objp[:,:2] = np.mgrid[0:CHECKERBOARD[0],0:CHECKERBOARD[1]].T.reshape(-1,2)

OpenCV Error: Assertion failed (objectPoints.type() == (((5) & ((1 << 3) - 1)) + (((3)-1) << 3)) || objectPoints.type() == (((6) & ((1 << 3) - 1)) + (((3)-1) << 3))) in cv::fisheye::stereoCalibrate, file C:\projects\opencv-python\opencv\modules\calib3d\src\fisheye.cpp, line 844
Traceback (most recent call last):
  File "C:/Users/treph/pycamdriver/stereo_calibration.py", line 239, in <module>
    calibrate(reader, cam_ids, CHECKERBOARD, SQUARE_SIZE)
  File "C:/Users/treph/pycamdriver/stereo_calibration.py", line 206, in calibrate
    (cv2.TERM_CRITERIA_EPS+cv2.TERM_CRITERIA_MAX_ITER, 30, 1e-6)
cv2.error: C:\projects\opencv-python\opencv\modules\calib3d\src\fisheye.cpp:844: error: (-215) objectPoints.type() == (((5) & ((1 << 3) - 1)) + (((3)-1) << 3)) || objectPoints.type() == (((6) & ((1 << 3) - 1)) + (((3)-1) << 3)) in function cv::fisheye::stereoCalibrate

The text was updated successfully, but these errors were encountered:

philipptrenz · 2018-03-16T07:23:55Z

Meanwhile also tested with OpenCV 3.4.1 via conda install -c conda-forge opencv

Case 1

OpenCV(3.4.1) Error: Assertion failed (!objectPoints.empty() && objectPoints.type() == (((6) & ((1 << 3) - 1)) + (((3)-1) << 3))) in cv::internal::projectPoints, file C:\bld\opencv_1520732670222\work\opencv-3.4.1\modules\calib3d\src\fisheye.cpp, line 1176
Traceback (most recent call last):
  File "C:/Users/treph/pycamdriver/stereo_calibration.py", line 244, in <module>
    calibrate(reader, cam_ids, CHECKERBOARD, SQUARE_SIZE)
  File "C:/Users/treph/pycamdriver/stereo_calibration.py", line 208, in calibrate
    (cv2.TERM_CRITERIA_EPS+cv2.TERM_CRITERIA_MAX_ITER, 30, 1e-6)
cv2.error: OpenCV(3.4.1) C:\bld\opencv_1520732670222\work\opencv-3.4.1\modules\calib3d\src\fisheye.cpp:1176: error: (-215) !objectPoints.empty() && objectPoints.type() == (((6) & ((1 << 3) - 1)) + (((3)-1) << 3)) in function cv::internal::projectPoints

Case 2

OpenCV(3.4.1) Error: Assertion failed (objectPoints.type() == (((5) & ((1 << 3) - 1)) + (((3)-1) << 3)) || objectPoints.type() == (((6) & ((1 << 3) - 1)) + (((3)-1) << 3))) in cv::fisheye::stereoCalibrate, file C:\bld\opencv_1520732670222\work\opencv-3.4.1\modules\calib3d\src\fisheye.cpp, line 852
Traceback (most recent call last):
  File "C:/Users/treph/pycamdriver/stereo_calibration.py", line 244, in <module>
    calibrate(reader, cam_ids, CHECKERBOARD, SQUARE_SIZE)
  File "C:/Users/treph/pycamdriver/stereo_calibration.py", line 208, in calibrate
    (cv2.TERM_CRITERIA_EPS+cv2.TERM_CRITERIA_MAX_ITER, 30, 1e-6)
cv2.error: OpenCV(3.4.1) C:\bld\opencv_1520732670222\work\opencv-3.4.1\modules\calib3d\src\fisheye.cpp:852: error: (-215) objectPoints.type() == (((5) & ((1 << 3) - 1)) + (((3)-1) << 3)) || objectPoints.type() == (((6) & ((1 << 3) - 1)) + (((3)-1) << 3)) in function cv::fisheye::stereoCalibrate

philipptrenz · 2018-03-16T10:01:39Z

I found a workaround (similar to #5534)

K_left = np.zeros((3, 3))
D_left = np.zeros((4, 1))

K_right = np.zeros((3, 3))
D_right = np.zeros((4, 1))

R = np.zeros((1, 1, 3), dtype=np.float64)
T = np.zeros((1, 1, 3), dtype=np.float64)

N_OK = len(imgpoints_left)

objp = np.zeros( (CHECKERBOARD[0]*CHECKERBOARD[1], 1, 3) , np.float64)
objp[:,0, :2] = np.mgrid[0:CHECKERBOARD[0], 0:CHECKERBOARD[1]].T.reshape(-1, 2)

objpoints = np.array([objp]*len(imgpoints_left), dtype=np.float64)
imgpoints_left = np.asarray(imgpoints_left, dtype=np.float64)
imgpoints_right = np.asarray(imgpoints_right, dtype=np.float64)

objpoints = np.reshape(objpoints, (N_OK, 1, CHECKERBOARD[0]*CHECKERBOARD[1], 3))
imgpoints_left = np.reshape(imgpoints_left, (N_OK, 1, CHECKERBOARD[0]*CHECKERBOARD[1], 2))
imgpoints_right = np.reshape(imgpoints_right, (N_OK, 1, CHECKERBOARD[0]*CHECKERBOARD[1], 2))	

(rms, K_left, D_left, K_right, D_right, R, T) = \
        cv2.fisheye.stereoCalibrate(
            objpoints,
            imgpoints_left,
            imgpoints_right,
            K_left,
            D_left,
            K_right,
            D_right,
            img_size,
            R,
            T,
            calibration_flags
        )

Now the arrays have the format

objpoints shape: (<num of calibration images>, 1, <num points in set>, 3)
imgpoints_left shape: (<num of calibration images>, 1, <num points in set>, 2)
imgpoints_right shape: (<num of calibration images>, 1, <num points in set>, 2)

and cv2.fisheye.stereoCalibrate() returns a valid result

set_cammodel of StereoCalibrator need to override the method of parent class fix related to opencv/opencv#11085 unlike cv2.calibrate, the cv2.fisheye.calibrate method expects float64 points and in an array with an extra dimension. The same for cv2.stereoCalibrate vs cv2.fisheye.stereoCalibrate

set_cammodel of StereoCalibrator need to override the method of parent class fix related to opencv/opencv#11085: unlike cv2.calibrate, the cv2.fisheye.calibrate method expects float64 points and in an array with an extra dimension. The same for cv2.stereoCalibrate vs cv2.fisheye.stereoCalibrate Trimmed whitespace at line endings

* fix #503: set_cammodel of StereoCalibrator need to override the method of parent class fix related to opencv/opencv#11085 unlike cv2.calibrate, the cv2.fisheye.calibrate method expects float64 points and in an array with an extra dimension. The same for cv2.stereoCalibrate vs cv2.fisheye.stereoCalibrate * removed commented line; trimmed whitespace at line endings

* fix ros-perception#503: set_cammodel of StereoCalibrator need to override the method of parent class fix related to opencv/opencv#11085 unlike cv2.calibrate, the cv2.fisheye.calibrate method expects float64 points and in an array with an extra dimension. The same for cv2.stereoCalibrate vs cv2.fisheye.stereoCalibrate * removed commented line; trimmed whitespace at line endings

rajattiwari01 · 2020-08-29T09:33:32Z

how to define imgpoints_left and right please tell??

set_cammodel of StereoCalibrator need to override the method of parent class fix related to opencv/opencv#11085: unlike cv2.calibrate, the cv2.fisheye.calibrate method expects float64 points and in an array with an extra dimension. The same for cv2.stereoCalibrate vs cv2.fisheye.stereoCalibrate Trimmed whitespace at line endings

AleksandrPanov · 2021-08-24T17:11:44Z

@philipptrenz, as I understand it, a modified example is used from here. The images you are using are missing, as well as the complete code.
Could you provide full code to reproduce the issue?

AleksandrPanov · 2021-08-27T17:18:58Z

reproducer.py.txt
I made a a working reproducer, now I need to check for the bugs in this issue)
Photos from here.

photos

AleksandrPanov · 2021-08-30T16:16:45Z

As I understand it, the problem is the extra dimension of the arrays objpoints, imgpoints_left and imgpoints_right. You can try using cv.Mat wrapper #20558. Then you don't have to use extra dimension. The code will look like this:

objPoints = cv.Mat(objPoints, wrap_channels=3)
imgPointsLeft = cv.Mat(imgPointsLeft, wrap_channels=2)
...

Also, I noticed that regardless of the input data type, stereoCalibrate works with double (np.float64).

AleksandrPanov · 2021-09-02T17:46:55Z

checked cv.Mat wrapper #20558
You can write code like this:

objPoints = np.array([objPoint]*N_OK, dtype=np.float64)
objPoints = cv.Mat(np.reshape(objPoints, (N_OK, -1, 3)), wrap_channels=3)

imgPointsLeft = np.asarray(imgPointsLeft, dtype=np.float64)
imgPointsLeft = cv.Mat(np.reshape(imgPointsLeft, (N_OK, -1, 2)), wrap_channels=2)

imgPointsRight = np.asarray(imgPointsRight, dtype=np.float64)
imgPointsRight = cv.Mat(np.reshape(imgPointsRight, (N_OK, -1, 2)), wrap_channels=2)

Here is all the code:
repr_cv_mat.py.txt

set_cammodel of StereoCalibrator need to override the method of parent class fix related to opencv/opencv#11085: unlike cv2.calibrate, the cv2.fisheye.calibrate method expects float64 points and in an array with an extra dimension. The same for cv2.stereoCalibrate vs cv2.fisheye.stereoCalibrate Trimmed whitespace at line endings

philipptrenz mentioned this issue Mar 16, 2018

python calls to fisheye calibration fail with exception (matrix dimension errors) #5534

Closed

qenops mentioned this issue Jul 19, 2018

Python stereoCalibrate fails assertion when adding CALIB_USE_EXTRINSIC_GUESS flag #12012

Closed

soeroesg mentioned this issue Jun 10, 2020

fixes for fisheye calibration ros-perception/image_pipeline#542

Merged

soeroesg mentioned this issue Jun 12, 2020

fixes for fisheye calibration ros-perception/image_pipeline#545

Merged

asmorkalov assigned AleksandrPanov Aug 23, 2021

AleksandrPanov added the incomplete label Aug 24, 2021

AleksandrPanov removed the incomplete label Aug 30, 2021

AleksandrPanov linked a pull request Sep 9, 2021 that will close this issue

python: cv.Mat wrapper over numpy.ndarray #20558

Merged

4 tasks

opencv-pushbot closed this as completed in #20558 Sep 26, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python fisheye.stereoCalibrate: Assertion failed objectPoints.type() #11085

Python fisheye.stereoCalibrate: Assertion failed objectPoints.type() #11085

philipptrenz commented Mar 15, 2018 •

edited

philipptrenz commented Mar 16, 2018 •

edited

philipptrenz commented Mar 16, 2018 •

edited

rajattiwari01 commented Aug 29, 2020

AleksandrPanov commented Aug 24, 2021

AleksandrPanov commented Aug 27, 2021

AleksandrPanov commented Aug 30, 2021 •

edited

AleksandrPanov commented Sep 2, 2021

Python fisheye.stereoCalibrate: Assertion failed objectPoints.type() #11085

Python fisheye.stereoCalibrate: Assertion failed objectPoints.type() #11085

Comments

philipptrenz commented Mar 15, 2018 • edited

System information

Detailed description

Steps to reproduce

philipptrenz commented Mar 16, 2018 • edited

Case 1

Case 2

philipptrenz commented Mar 16, 2018 • edited

rajattiwari01 commented Aug 29, 2020

AleksandrPanov commented Aug 24, 2021

AleksandrPanov commented Aug 27, 2021

AleksandrPanov commented Aug 30, 2021 • edited

AleksandrPanov commented Sep 2, 2021

philipptrenz commented Mar 15, 2018 •

edited

philipptrenz commented Mar 16, 2018 •

edited

philipptrenz commented Mar 16, 2018 •

edited

AleksandrPanov commented Aug 30, 2021 •

edited