Linear solver failure. Failed to compute a step: CHOLMOD warning: Matrix not positive definite.

[hnzhw]

windows 10
glomap: 1.0.0
datasets: https://demuc.de/colmap/datasets/south-building.zip
Use database.db in south-building.zip and run like below, everything is OK.

glomap mapper
--database_path ./data/gerrard-hall/database.db
--image_path ./data/gerrard-hall/images
--output_path ./output/gerrard-hall/sparse

Output from glomap mapper:

Loading Images 128 / 128
Loading Image Pair 4210 / 4210
I20240822 17:35:30.230699 11584 colmap_converter.cc:303] Pairs read done. 2323 / 4210 are invalid
I20240822 17:35:30.233300 11584 global_mapper.cc:80] Loaded database

Running preprocessing ...

I20240822 17:35:30.234301 11584 view_graph_manipulation.cc:246] Decompose relative pose for 1887 pairs
I20240822 17:35:30.244260 11584 view_graph_manipulation.cc:295] Decompose relative pose done. 625 pairs are pure rotation
I20240822 17:35:30.244423 11584 timer.cc:87] Elapsed time: 0.01060 [seconds]

Running view graph calibration ...

I20240822 17:35:30.244914 11584 view_graph_calibration.cc:16] Start ViewGraphCalibrator
I20240822 17:35:30.248474 11584 view_graph_calibration.cc:33] No cameras to optimize

Running relative pose estimation ...

I20240822 17:35:30.251966 11584 image_undistorter.cc:17] Undistorting images..
I20240822 17:35:30.268700 11584 image_undistorter.cc:37] Image undistortion done
I20240822 17:35:30.269020 11584 relpose_estimation.cc:24] Estimating relative pose for 1887 pairs
Estimating relative pose: 100%
I20240822 17:35:37.272276 11584 relpose_estimation.cc:69] Estimating relative pose done
I20240822 17:35:37.343369 11584 relpose_filter.cc:46] Filtered 1056 relative poses with inlier number < 30
I20240822 17:35:37.343642 11584 relpose_filter.cc:63] Filtered 1 relative poses with inlier ratio < 0.25
I20240822 17:35:37.346379 11584 timer.cc:87] Elapsed time: 7.09444 [seconds]

Running rotation averaging ...

I20240822 17:35:37.357534 11584 relpose_filter.cc:31] Filtered 0 relative rotation with angle > 10 degrees
I20240822 17:35:37.364138 11584 relpose_filter.cc:31] Filtered 0 relative rotation with angle > 10 degrees
I20240822 17:35:37.364799 11584 global_mapper.cc:95] 125 / 128 images are within the connected component.
I20240822 17:35:37.365284 11584 timer.cc:87] Elapsed time: 0.01402 [seconds]

Running track establishment ...

Initializing pairs 4210 / 4210
Establishing pairs 4210 / 4210
Establishing tracks 80627 / 80627
I20240822 17:35:38.067952 11584 track_establishment.cc:149] Discarded 104 tracks due to inconsistency
I20240822 17:35:38.193388 11584 global_mapper.cc:115] Before filtering: 80627, after filtering: 38639
I20240822 17:35:38.193712 11584 timer.cc:87] Elapsed time: 0.82736 [seconds]

Running global positioning ...

I20240822 17:35:38.233830 11584 image_undistorter.cc:17] Undistorting images..
I20240822 17:35:38.234965 11584 image_undistorter.cc:37] Image undistortion done
I20240822 17:35:38.235684 11584 global_positioning.cc:43] Setting up the global positioner problem
I20240822 17:35:38.868376 11584 global_positioning.cc:68] Solving the global positioner problem
I20240822 17:36:17.080872 11584 global_positioning.cc:77] Ceres Solver Report: Iterations: 45, Initial cost: 2.820551e+06, Final cost: 3.033326e+00, Termination: CONVERGENCE
I20240822 17:36:17.127746 11584 track_filter.cc:86] Filtered 126 / 38639 tracks by angle error
I20240822 17:36:17.128000 11584 timer.cc:87] Elapsed time: 38.89250 [seconds]

Running bundle adjustment ...

I20240822 17:36:17.355226 11584 global_mapper.cc:167] Bundle adjustment start
I20240822 17:36:27.161768 11584 bundle_adjustment.cc:48] Ceres Solver Report: Iterations: 21, Initial cost: 2.295280e+05, Final cost: 1.672357e+05, Termination: CONVERGENCE
I20240822 17:36:27.162060 11584 global_mapper.cc:183] Global bundle adjustment iteration 1 / 3, stage 1 finished (position only)
I20240822 17:36:27.163555 11584 timer.cc:87] Elapsed time: 9.80691 [seconds]
I20240822 17:36:33.258206 11584 bundle_adjustment.cc:48] Ceres Solver Report: Iterations: 12, Initial cost: 1.672357e+05, Final cost: 4.636266e+04, Termination: CONVERGENCE
I20240822 17:36:33.258443 11584 global_mapper.cc:195] Global bundle adjustment iteration 1 / 3, stage 2 finished
I20240822 17:36:33.259880 11584 timer.cc:87] Elapsed time: 15.90297 [seconds]
I20240822 17:36:33.260977 11584 image_undistorter.cc:17] Undistorting images..
I20240822 17:36:33.274364 11584 image_undistorter.cc:37] Image undistortion done
I20240822 17:36:33.274473 11584 global_mapper.cc:206] Filtering tracks by reprojection ...
I20240822 17:36:33.353821 11584 track_filter.cc:48] Filtered 13 / 38639 tracks by reprojection error
I20240822 17:36:33.409078 11584 track_filter.cc:48] Filtered 38 / 38639 tracks by reprojection error
I20240822 17:36:35.530746 11584 bundle_adjustment.cc:48] Ceres Solver Report: Iterations: 3, Initial cost: 4.545477e+04, Final cost: 4.544126e+04, Termination: CONVERGENCE
I20240822 17:36:35.531020 11584 global_mapper.cc:183] Global bundle adjustment iteration 3 / 3, stage 1 finished (position only)
I20240822 17:36:35.532505 11584 timer.cc:87] Elapsed time: 18.17550 [seconds]
I20240822 17:36:37.565226 11584 bundle_adjustment.cc:48] Ceres Solver Report: Iterations: 2, Initial cost: 4.544126e+04, Final cost: 4.544079e+04, Termination: CONVERGENCE
I20240822 17:36:37.565523 11584 global_mapper.cc:195] Global bundle adjustment iteration 3 / 3, stage 2 finished
I20240822 17:36:37.569686 11584 image_undistorter.cc:17] Undistorting images..
I20240822 17:36:37.584867 11584 image_undistorter.cc:37] Image undistortion done
I20240822 17:36:37.585013 11584 global_mapper.cc:206] Filtering tracks by reprojection ...
I20240822 17:36:37.634968 11584 track_filter.cc:48] Filtered 91 / 38639 tracks by reprojection error
I20240822 17:36:37.767502 11584 image_undistorter.cc:17] Undistorting images..
I20240822 17:36:37.781028 11584 image_undistorter.cc:37] Image undistortion done
I20240822 17:36:37.781199 11584 global_mapper.cc:232] Filtering tracks by reprojection ...
I20240822 17:36:37.828873 11584 track_filter.cc:48] Filtered 0 / 38639 tracks by reprojection error
I20240822 17:36:37.844119 11584 track_filter.cc:123] Filtered 162 / 38639 tracks by too small triangulation angle
I20240822 17:36:37.844235 11584 timer.cc:87] Elapsed time: 20.48712 [seconds]

Running retriangulation ...

I20240822 17:36:37.846989 11584 database_cache.cc:65] Loading cameras...
I20240822 17:36:37.847944 11584 database_cache.cc:75] 1 in 0.001s
I20240822 17:36:37.851057 11584 database_cache.cc:83] Loading matches...
I20240822 17:36:37.874086 11584 database_cache.cc:89] 1887 in 0.023s
I20240822 17:36:37.874201 11584 database_cache.cc:105] Loading images...
I20240822 17:36:37.967951 11584 database_cache.cc:155] 128 in 0.094s (connected 128)
I20240822 17:36:37.968143 11584 database_cache.cc:166] Building correspondence graph...
I20240822 17:36:38.187888 11584 database_cache.cc:195] in 0.220s (ignored 0)
Triangulating image 125 / 125
Global bundle adjustment iteration 5 / 5
I20240822 17:36:49.732669 11584 timer.cc:87] Elapsed time: 11.88522 [seconds]

Running bundle adjustment ...

I20240822 17:36:49.738017 11584 global_mapper.cc:263] Bundle adjustment start
I20240822 17:36:57.255989 11584 bundle_adjustment.cc:48] Ceres Solver Report: Iterations: 10, Initial cost: 8.329698e+04, Final cost: 6.548217e+04, Termination: CONVERGENCE
I20240822 17:36:57.256411 11584 image_undistorter.cc:17] Undistorting images..
I20240822 17:36:57.275769 11584 image_undistorter.cc:37] Image undistortion done
I20240822 17:36:57.275920 11584 global_mapper.cc:271] Filtering tracks by reprojection ...
I20240822 17:36:57.349538 11584 track_filter.cc:48] Filtered 3 / 59565 tracks by reprojection error
I20240822 17:36:59.761294 11584 bundle_adjustment.cc:48] Ceres Solver Report: Iterations: 1, Initial cost: 6.548217e+04, Final cost: 6.548217e+04, Termination: CONVERGENCE
I20240822 17:36:59.761617 11584 timer.cc:87] Elapsed time: 21.91381 [seconds]
I20240822 17:36:59.916997 11584 image_undistorter.cc:17] Undistorting images..
I20240822 17:36:59.929033 11584 image_undistorter.cc:37] Image undistortion done
I20240822 17:36:59.929224 11584 global_mapper.cc:286] Filtering tracks by reprojection ...
I20240822 17:37:00.008214 11584 track_filter.cc:48] Filtered 0 / 59565 tracks by reprojection error
I20240822 17:37:00.029527 11584 track_filter.cc:123] Filtered 3 / 59565 tracks by too small triangulation angle
I20240822 17:37:00.029670 11584 global_mapper.cc:86] Reconstruction done in 89.7928 seconds
I20240822 17:37:00.141283 11584 colmap_io.cc:27] Extracting colors ...
I20240822 17:37:06.192611 11584 global_mapper.cc:91] Export to COLMAP reconstruction done

/////////////////////////////////////////////////////////////////////////////////////////////////////////////
But, when I use colmap3.8 or colmap 3.10 like below, Reconstruction failed.

colmap feature_extractor
--image_path ./data/south-building/images
--database_path ./data/south-building/database.db
colmap exhaustive_matcher
--database_path ./data/south-building/database.db
glomap mapper
--database_path ./data/south-building/database.db
--image_path ./data/south-building/images
--output_path ./output/south-building/sparse

Output from glomap mapper:

Loading Images 128 / 128
Loading Image Pair 6043 / 6043
I20240822 17:33:52.184539 14184 colmap_converter.cc:303] Pairs read done. 3477 / 6043 are invalid
I20240822 17:33:52.188060 14184 global_mapper.cc:80] Loaded database

Running preprocessing ...

I20240822 17:33:52.193102 14184 view_graph_manipulation.cc:246] Decompose relative pose for 2566 pairs
I20240822 17:33:52.204991 14184 view_graph_manipulation.cc:295] Decompose relative pose done. 603 pairs are pure rotation
I20240822 17:33:52.205195 14184 timer.cc:87] Elapsed time: 0.01303 [seconds]

Running view graph calibration ...

I20240822 17:33:52.208536 14184 view_graph_calibration.cc:16] Start ViewGraphCalibrator
I20240822 17:33:52.215678 14184 view_graph_calibration.cc:33] No cameras to optimize

Running relative pose estimation ...

I20240822 17:33:52.217962 14184 image_undistorter.cc:17] Undistorting images..
I20240822 17:33:52.232467 14184 image_undistorter.cc:37] Image undistortion done
I20240822 17:33:52.232938 14184 relpose_estimation.cc:24] Estimating relative pose for 2566 pairs
Estimating relative pose: 100%
I20240822 17:34:03.979320 14184 relpose_estimation.cc:69] Estimating relative pose done
I20240822 17:34:04.119572 14184 relpose_filter.cc:46] Filtered 990 relative poses with inlier number < 30
I20240822 17:34:04.120230 14184 relpose_filter.cc:63] Filtered 1 relative poses with inlier ratio < 0.25
I20240822 17:34:04.123964 14184 timer.cc:87] Elapsed time: 11.90602 [seconds]

Running rotation averaging ...

I20240822 17:34:04.153754 14184 relpose_filter.cc:31] Filtered 3 relative rotation with angle > 10 degrees
I20240822 17:34:04.186612 14184 relpose_filter.cc:31] Filtered 0 relative rotation with angle > 10 degrees
I20240822 17:34:04.187839 14184 global_mapper.cc:95] 128 / 128 images are within the connected component.
I20240822 17:34:04.187935 14184 timer.cc:87] Elapsed time: 0.06126 [seconds]

Running track establishment ...

Initializing pairs 6043 / 6043
Establishing pairs 6043 / 6043
Establishing tracks 129965 / 129965
I20240822 17:34:05.852708 14184 track_establishment.cc:149] Discarded 224 tracks due to inconsistency
I20240822 17:34:06.114772 14184 global_mapper.cc:115] Before filtering: 129965, after filtering: 71576
I20240822 17:34:06.115021 14184 timer.cc:87] Elapsed time: 1.92610 [seconds]

Running global positioning ...

I20240822 17:34:06.178141 14184 image_undistorter.cc:17] Undistorting images..
I20240822 17:34:06.179424 14184 image_undistorter.cc:37] Image undistortion done
I20240822 17:34:06.180182 14184 global_positioning.cc:43] Setting up the global positioner problem
I20240822 17:34:07.567950 14184 global_positioning.cc:68] Solving the global positioner problem
W20240822 17:34:12.603970 14184 levenberg_marquardt_strategy.cc:115] Linear solver failure. Failed to compute a step: CHOLMOD warning: Matrix not positive definite.
W20240822 17:34:13.179074 14184 levenberg_marquardt_strategy.cc:115] Linear solver failure. Failed to compute a step: CHOLMOD warning: Matrix not positive definite.
W20240822 17:34:13.764874 14184 levenberg_marquardt_strategy.cc:115] Linear solver failure. Failed to compute a step: CHOLMOD warning: Matrix not positive definite.
W20240822 17:34:14.358298 14184 levenberg_marquardt_strategy.cc:115] Linear solver failure. Failed to compute a step: CHOLMOD warning: Matrix not positive definite.
W20240822 17:34:14.939368 14184 levenberg_marquardt_strategy.cc:115] Linear solver failure. Failed to compute a step: CHOLMOD warning: Matrix not positive definite.
E20240822 17:34:14.939627 14184 trust_region_minimizer.cc:95] Terminating: Number of consecutive invalid steps more than Solver::Options::max_num_consecutive_invalid_steps: 5
I20240822 17:34:15.318360 14184 global_positioning.cc:77] Ceres Solver Report: Iterations: 5, Initial cost: 6.013900e+06, Final cost: 6.013900e+06, Termination: FAILURE
I20240822 17:34:15.832611 14184 global_mapper.cc:86] Reconstruction done in 23.6444 seconds
I20240822 17:34:15.948935 14184 colmap_io.cc:27] Extracting colors ...
I20240822 17:34:22.248869 14184 global_mapper.cc:91] Export to COLMAP reconstruction done

Is there anything that I should pay extra attention to?

[lpanaf]

Hi, thanks for your interest in the work. Can you maybe try starting all the steps from scratch (after deleting the database)? If this does not work for you, you can maybe try reinstalling suitesparse (this seems to be the most possible problem) or changing the random seed specified at https://github.com/colmap/glomap/blob/main/glomap/estimators/global_positioning.h#L36

[jonstephens85]

I ran into this issue too. I tried 8 different datasets with the same error as mentioned here. When I tried the example dataset from scratch it worked without issue.

I am using the windows binaries and COLMAP 3.8. I thought perhaps trying COLMAP 3.9 but was unsure if that was related to the issue considering the example dataset worked just fine with COLMAP 3.8.

[hnzhw]

Hi, thanks for your interest in the work. Can you maybe try starting all the steps from scratch (after deleting the database)? If this does not work for you, you can maybe try reinstalling suitesparse (this seems to be the most possible problem) or changing the random seed specified at https://github.com/colmap/glomap/blob/main/glomap/estimators/global_positioning.h#L36

Thank you very much for your reply.
I start all the steps from scratch, delete all files except for the image datasets, but it is still not working.
When running glomap, the CPU usage reaches 100%, is it normal?
Additionally, I am using windows binaries and colmap3.8, how to reinstalling suitesparse?

D:\mygs\glomap\colmap38\bin\colmap feature_extractor --image_path D:\mygs\glomap\data\south-building\images --database_path D:\mygs\glomap\data\south-building\database.db --ImageReader.camera_model OPENCV --SiftExtraction.use_gpu 1

output logs:

Feature extraction

Processed file [1/128]
Name: P1180141.JPG
Dimensions: 3072 x 2304
Camera: #1 - OPENCV
Focal Length: 2457.60px (Prior)
Features: 12923
Processed file [2/128]
Name: P1180142.JPG
Dimensions: 3072 x 2304
Camera: #1 - OPENCV
Focal Length: 2457.60px (Prior)
Features: 8620
... ...
Processed file [127/128]
Name: P1180346.JPG
Dimensions: 3072 x 2304
Camera: #1 - OPENCV
Focal Length: 2457.60px (Prior)
Features: 10936
Processed file [128/128]
Name: P1180347.JPG
Dimensions: 3072 x 2304
Camera: #1 - OPENCV
Focal Length: 2457.60px (Prior)
Features: 14988
Elapsed time: 0.388 [minutes]

D:\mygs\glomap\colmap38\bin\colmap exhaustive_matcher --database_path D:\mygs\glomap\data\south-building\database.db --SiftMatching.use_gpu 1

output logs:

Exhaustive feature matching

Matching block [1/3, 1/3] in 5.933s
Matching block [1/3, 2/3] in 8.884s
Matching block [1/3, 3/3] in 2.078s
Matching block [2/3, 1/3] in 16.031s
Matching block [2/3, 2/3] in 9.626s
Matching block [2/3, 3/3] in 3.254s
Matching block [3/3, 1/3] in 7.076s
Matching block [3/3, 2/3] in 6.718s
Matching block [3/3, 3/3] in 1.723s
Elapsed time: 1.030 [minutes]

D:\mygs\glomap\bin\glomap mapper --database_path D:\mygs\glomap\data\south-building\database.db --image_path D:\mygs\glomap\data\south-building\images --output_path D:\mygs\glomap\data\south-building\sparse

.... ...

[hnzhw]

I ran into this issue too. I tried 8 different datasets with the same error as mentioned here. When I tried the example dataset from scratch it worked without issue.

I am using the windows binaries and COLMAP 3.8. I thought perhaps trying COLMAP 3.9 but was unsure if that was related to the issue considering the example dataset worked just fine with COLMAP 3.8.

Same envirment, but it is not working.
"When I tried the example dataset from scratch it worked without issue."
Could you please to tell me which dataset you are using? what's the steps?
Thanks.

[ahojnnes]

Using --ImageReader.camera_model OPENCV without having manually calibrated the camera using a checkerboard pattern or, at least, sharing intrinsics for multiple images, rarely ever makes sense. If you really deal with uncalibrated cameras that vary by image, I recommend using a simpler camera model, e.g., SIMPLE_RADIAL. The same advice also applies to COLMAP and even more so to GLOMAP.

[hnzhw]

Using --ImageReader.camera_model OPENCV without having manually calibrated the camera using a checkerboard pattern or, at least, sharing intrinsics for multiple images, rarely ever makes sense. If you really deal with uncalibrated cameras that vary by image, I recommend using a simpler camera model, e.g., SIMPLE_RADIAL. The same advice also applies to COLMAP and even more so to GLOMAP.无需 --ImageReader.camera_model OPENCV 使用棋盘格图案手动校准相机，或者至少共享多个图像的内部函数，很少有意义。如果您真的要处理因图像而异的未校准相机，我建议使用更简单的相机型号，例如 SIMPLE_RADIAL。同样的建议也适用于 COLMAP，更适用于 GLOMAP。

windows10 x64 / 128G / Intel(R) Xeon(R) Gold 6226R / RTX4090 24G
colmap 3.8 / glomap 1.0.0 for windows

Use SIMPLE_RADIAL, or without camera_model, same result.

[hnzhw]

Resolved.
#69