Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"GTSAM exception caught: Indeterminant linear system detected while working near variable" then crash #1057

Closed
ApoloM opened this issue Oct 27, 2023 · 16 comments
Labels

Comments

@ApoloM
Copy link

ApoloM commented Oct 27, 2023

Hi everyone, I'm working in a project with ROS1 in which we use Rtabmap for localization in a previously mapped environment. I'm using the localization_pose to feed my localization stack. Recently I have experienced problems with GTSAM which warns for "inderteminant linear systems". Eventually, it also crashes with the following FATAL ERROR.

FATAL] (2023-10-27 09:25:31.539) Transform.cpp:184::inverse() Condition (invertible) not met! [This transform is not invertible! xyz=inf,inf,0.000000 rpy=0.000000,0.000000,-1.267575 
[0.298596 0.954380 0.000000 inf;
 -0.954380 0.298596 -0.000000 inf;
 -0.000000 0.000000 1.000000 0.000000;
 0 0 0 1]]
terminate called after throwing an instance of 'UException'
  what():  [FATAL] (2023-10-27 09:25:31.539) Transform.cpp:184::inverse() Condition (invertible) not met! [This transform is not invertible! xyz=inf,inf,0.000000 rpy=0.000000,0.000000,-1.267575 
[0.298596 0.954380 0.000000 inf;
 -0.954380 0.298596 -0.000000 inf;
 -0.000000 0.000000 1.000000 0.000000;
 0 0 0 1]]

By setting the "--udebug --logconsole" arguments I was able to see some information about the problem. However something very strange took my attemption, which are the following debug messages:

[DEBUG] (2023-10-27 09:25:31.535) Rtabmap.cpp:2946::process() Constraint -8->-8 (type=PosePrior)
[DEBUG] (2023-10-27 09:25:31.535) Rtabmap.cpp:2948::process() Constraint 1058->-8 (type=Landmark, var = 0.001000 0.010000)
[DEBUG] (2023-10-27 09:25:31.535) Rtabmap.cpp:2946::process() Constraint 875->875 (type=PosePrior)
[DEBUG] (2023-10-27 09:25:31.535) Rtabmap.cpp:2948::process() Constraint 1058->875 (type=LocalSpaceClosure, var = 0.000003 0.002736)
[DEBUG] (2023-10-27 09:25:31.535) Rtabmap.cpp:2946::process() Constraint 876->876 (type=PosePrior)
[DEBUG] (2023-10-27 09:25:31.535) Rtabmap.cpp:2948::process() Constraint 1058->876 (type=GlobalClosure, var = 0.000002 0.001697)
[DEBUG] (2023-10-27 09:25:31.535) Rtabmap.cpp:2946::process() Constraint 1033->1033 (type=PosePrior)
[DEBUG] (2023-10-27 09:25:31.535) Rtabmap.cpp:2948::process() Constraint 1058->1033 (type=LocalSpaceClosure, var = 0.000027 0.007702)
[DEBUG] (2023-10-27 09:25:31.535) Rtabmap.cpp:2952::process() Pose -8 xyz=0.851523,-0.041897,0.404141 rpy=0.085773,0.006991,1.561444
[DEBUG] (2023-10-27 09:25:31.535) Rtabmap.cpp:2952::process() Pose 875 xyz=0.319759,-0.040769,0.000000 rpy=0.000000,-0.000000,0.013201
[DEBUG] (2023-10-27 09:25:31.536) Rtabmap.cpp:2952::process() Pose 876 xyz=0.297985,-0.013632,0.000000 rpy=0.000000,0.000000,-0.039805
[DEBUG] (2023-10-27 09:25:31.536) Rtabmap.cpp:2952::process() Pose 1033 xyz=0.240514,-0.004088,0.000000 rpy=0.000000,-0.000000,0.023084
[DEBUG] (2023-10-27 09:25:31.536) Rtabmap.cpp:2952::process() Pose 1058 xyz=-197.406799,29.594700,0.000000 rpy=0.000000,0.000000,-3.107898

The above logs, show the constraints used in the optmization. However the last line took my attemption. In the referred line we can see that the pose of the node 1058 is very distant (in terms of xyz) from the other nodes of the graph. Then, I was looking in the code and found these debug messages in the Rtabmap library. I noticed that the odom pose is not being transformed to the same frame of the graph nodes before optimization, as shown in the following lines:

https://github.com/introlab/rtabmap_ros/blob/3293f2b51a42cdcd4f531576e3d787f149ac2e1b/rtabmap_slam/src/CoreWrapper.cpp#L2026C6-L2030

Based in my knowledge of optimization, I thought it would be better for the optimization problem (in terms of speed and accuracy) if the initial guess was near to the minimum. I understand that the mapCorrection output is being used to generate the mapToOdom information. However, in my point of view, it would be better for the optimization problem, if the odom pose was transformed to the map frame with the last map->odom transform before passing it to the optimizer. Then the mapCorrection output could be used to correct the last map->odom TF from /tf.

I would appreciate any comments and clarifications on this.
Thank you!

@matlabbe
Copy link
Member

matlabbe commented Oct 29, 2023

It seems you are using rtabmap 0.20.18:
https://github.com/introlab/rtabmap/blob/4d8c1d8b90afc796bf3a3747516d68dd0e072409/corelib/src/Rtabmap.cpp#L2938-L2953

In that code, the poses fed to optimization are taken for _optimizedPoses map. The _optimizedPoses map should contain the latest pose already corrected by _mapCorrection before the graph optimization: https://github.com/introlab/rtabmap/blob/4d8c1d8b90afc796bf3a3747516d68dd0e072409/corelib/src/Rtabmap.cpp#L1538, which would mean that the large jump is coming from the odometry (or from a bad previous optimization).

For the Transform.cpp:184::inverse() error, does it appear before or after optimization? I fixed a similar error some time ago, not sure it is the same. You may try latest master version.

@ApoloM
Copy link
Author

ApoloM commented Oct 30, 2023

Thanks for the response.

Yes, I'm using rtabmap 0.20.18. Ok, thanks for the clarification about the code.

In my case, sometimes the odometry is not at the origin when the rtabmap starts. Could it be the problem?

The Transform.cpp:184::inverse() error occurs after the optimization. I'm providing below some logs that could be useful:

Here is the log of a local loop closure that doesn't crash however warns about "Indeterminant linear system".

[DEBUG] (2023-10-30 09:20:49.995) Memory.cpp:3298::addLink() Add link between 1033 and 1053
[DEBUG] (2023-10-30 09:20:49.995) Signature.cpp:121::addLink() Add link 1053 to 1033 (type=2/LocalSpaceClosure var=0.000050,0.008785)
[DEBUG] (2023-10-30 09:20:49.995) Signature.cpp:121::addLink() Add link 1033 to 1053 (type=2/LocalSpaceClosure var=0.000050,0.008785)
[ INFO] (2023-10-30 09:20:49.995) Rtabmap.cpp:2569::process() timeProximityBySpaceVisualDetection=0.116715s
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2574::process() Proximity detection (local loop closure in SPACE with scan matching)
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2577::process() Proximity by scan matching is disabled (RGBD/ProximityPathMaxNeighbors=0).
[ INFO] (2023-10-30 09:20:49.995) Rtabmap.cpp:2759::process() timeProximityBySpaceDetection=0.000126s
[ INFO] (2023-10-30 09:20:49.995) Rtabmap.cpp:2823::process() timeAddLoopClosureLink=0.000015s
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2863::process() RGB-D SLAM mode: 1
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2864::process() Incremental: 0
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2865::process() Loop hyp: 0
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2866::process() Last prox: 1033
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2867::process() Reduced ids: 0
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2868::process() Has prior: 0 (prior ignored=1)
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2869::process() Has gravity: 0 (sigma=0.300000, odomGravity=0, refined=0)
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2870::process() Has virtual link: 0
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2871::process() Prox Time: 0
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2872::process() Landmarks: 1
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2873::process() Retrieved: 0
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2874::process() Not self ref links: 3
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2946::process() Constraint -8->-8 (type=PosePrior)
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2948::process() Constraint 1053->-8 (type=Landmark, var = 0.001000 0.010000)
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2946::process() Constraint 875->875 (type=PosePrior)
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2948::process() Constraint 1053->875 (type=LocalSpaceClosure, var = 0.000003 0.002840)
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2946::process() Constraint 876->876 (type=PosePrior)
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2948::process() Constraint 1053->876 (type=GlobalClosure, var = 0.000004 0.002840)
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2946::process() Constraint 1033->1033 (type=PosePrior)
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2948::process() Constraint 1053->1033 (type=LocalSpaceClosure, var = 0.000050 0.008785)
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2952::process() Pose -8 xyz=0.851523,-0.041897,0.404141 rpy=0.085773,0.006991,1.561444
[DEBUG] (2023-10-30 09:20:49.995) Rtabmap.cpp:2952::process() Pose 875 xyz=0.319759,-0.040769,0.000000 rpy=0.000000,-0.000000,0.013201
[DEBUG] (2023-10-30 09:20:49.996) Rtabmap.cpp:2952::process() Pose 876 xyz=0.297985,-0.013632,0.000000 rpy=0.000000,0.000000,-0.039805
[DEBUG] (2023-10-30 09:20:49.996) Rtabmap.cpp:2952::process() Pose 1033 xyz=0.240514,-0.004088,0.000000 rpy=0.000000,-0.000000,0.023084
[DEBUG] (2023-10-30 09:20:49.996) Rtabmap.cpp:2952::process() Pose 1053 xyz=-197.406799,29.594700,0.000000 rpy=0.000000,0.000000,-3.107898
[DEBUG] (2023-10-30 09:20:49.996) Rtabmap.cpp:2959::process() priorsIgnored was true
[DEBUG] (2023-10-30 09:20:49.996) Optimizer.cpp:196::getConnectedGraph() IN: fromId=1053 poses=5 links=8 priorsIgnored=0 landmarksIgnored=0
[DEBUG] (2023-10-30 09:20:49.996) Optimizer.cpp:301::getConnectedGraph() OUT: poses=5 links=8
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:103::optimize() Optimizing graph...
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:165::optimize() fill poses to gtsam... rootId=0 (priorsIgnored=0 landmarksIgnored=0)
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:225::optimize() fill edges to gtsam...
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:481::optimize() create optimizer
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:506::optimize() GTSAM optimizing begin (max iterations=20, robust=0)
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:586::optimize() iteration 1 error =49433668062.280464
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:586::optimize() iteration 2 error =1488168922289.005127
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-1438735254226.724609 < 0.000010)
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:586::optimize() iteration 3 error =797369678456255.375000
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-795881509533966.375000 < 0.000010)
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:586::optimize() iteration 4 error =205157881703003193344.000000
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-205157084333324730368.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:586::optimize() iteration 5 error =73079294581687113972973568.000000
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-73079089423805409143029760.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:586::optimize() iteration 6 error =39387139691086349379778444670271488.000000
[DEBUG] (2023-10-30 09:20:49.996) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-39387139618007054751211994589691904.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:586::optimize() iteration 7 error =2804363978864505529019133005520653641471492096.000000
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-2804363978825118357219441689786749576177254400.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:586::optimize() iteration 8 error =12138778478574983438982965529310933274210753450150285279232.000000
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-12138778478572179120274250303450885157637967918690908241920.000000 < 0.000010)
[ WARN] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:577::optimize() GTSAM exception caught: 
Indeterminant linear system detected while working near variable
1053 (Symbol: 1053).

Thrown when a linear system is ill-posed.  The most common cause for this
error is having underconstrained variables.  Mathematically, the system is
underdetermined.  See the GTSAM Doxygen documentation at
http://borg.cc.gatech.edu/ on gtsam::IndeterminantLinearSystemException for
more information.
 Graph has 8 edges and 5 vertices
[ WARN] (2023-10-30 09:20:49.997) Rtabmap.cpp:2972::process() Optimization failed, rejecting localization!
[ WARN] (2023-10-30 09:20:49.997) Rtabmap.cpp:3071::process() Global and loop closures seem not tallying together, try again to optimize without local loop closures...
[DEBUG] (2023-10-30 09:20:49.997) Rtabmap.cpp:3073::process() priorsIgnored was true
[DEBUG] (2023-10-30 09:20:49.997) Optimizer.cpp:196::getConnectedGraph() IN: fromId=1053 poses=5 links=6 priorsIgnored=0 landmarksIgnored=0
[DEBUG] (2023-10-30 09:20:49.997) Optimizer.cpp:301::getConnectedGraph() OUT: poses=3 links=4
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:103::optimize() Optimizing graph...
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:165::optimize() fill poses to gtsam... rootId=0 (priorsIgnored=0 landmarksIgnored=0)
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:225::optimize() fill edges to gtsam...
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:481::optimize() create optimizer
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:506::optimize() GTSAM optimizing begin (max iterations=20, robust=0)
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:586::optimize() iteration 1 error =42097696798.959717
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:586::optimize() iteration 2 error =15738131607.868059
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:586::optimize() iteration 3 error =143002741.243694
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:586::optimize() iteration 4 error =29923.925212
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:586::optimize() iteration 5 error =0.144665
[DEBUG] (2023-10-30 09:20:49.997) OptimizerGTSAM.cpp:586::optimize() iteration 6 error =0.005419
[DEBUG] (2023-10-30 09:20:49.998) OptimizerGTSAM.cpp:586::optimize() iteration 7 error =0.005419
[DEBUG] (2023-10-30 09:20:49.998) OptimizerGTSAM.cpp:596::optimize() Stop optimizing, not enough improvement (0.000001 < 0.000010)
[DEBUG] (2023-10-30 09:20:49.998) OptimizerGTSAM.cpp:615::optimize() GTSAM optimizing end (7 iterations done, error=0.005419 (initial=5337985244.923881 final=0.005419), time=0.000484 s)
[DEBUG] (2023-10-30 09:20:49.998) OptimizerGTSAM.cpp:675::optimize() Computing marginals...
[DEBUG] (2023-10-30 09:20:49.998) OptimizerGTSAM.cpp:679::optimize() Computed marginals = 0.000080s (key=1053)
[DEBUG] (2023-10-30 09:20:49.998) OptimizerGTSAM.cpp:725::optimize() Optimizing graph...end!
[DEBUG] (2023-10-30 09:20:49.998) Rtabmap.cpp:3081::process() Opt2  -8 xyz=0.851525,-0.041899,0.404141 rpy=0.085773,0.006991,1.561444
[DEBUG] (2023-10-30 09:20:49.998) Rtabmap.cpp:3081::process() Opt2  876 xyz=0.297983,-0.013630,0.000000 rpy=0.000000,0.000000,-0.039804
[DEBUG] (2023-10-30 09:20:49.998) Rtabmap.cpp:3081::process() Opt2  1053 xyz=0.306213,-0.044032,0.000000 rpy=0.000000,0.000000,0.020466
[ INFO] (2023-10-30 09:20:49.998) Rtabmap.cpp:3091::process() Compute max graph errors...
[DEBUG] (2023-10-30 09:20:49.998) Graph.cpp:909::computeMaxGraphErrors() poses=3 links=4
[ INFO] (2023-10-30 09:20:49.998) Rtabmap.cpp:3112::process() Max optimization linear error = 0.000008 m (link 1053->876, var=0.000004, ratio error/std=0.004070, thr=3.000000)

Here is the log of the local loop closure made just before the loop closure that crashes:

[DEBUG] (2023-10-30 09:20:51.055) Memory.cpp:3298::addLink() Add link between 875 and 1054
[DEBUG] (2023-10-30 09:20:51.055) Signature.cpp:121::addLink() Add link 1054 to 875 (type=2/LocalSpaceClosure var=0.000002,0.002147)
[DEBUG] (2023-10-30 09:20:51.055) Signature.cpp:121::addLink() Add link 875 to 1054 (type=2/LocalSpaceClosure var=0.000002,0.002147)
[DEBUG] (2023-10-30 09:20:51.055) Graph.cpp:2185::findNearestNodes() found nodes=3
[DEBUG] (2023-10-30 09:20:51.055) Graph.cpp:2218::findNearestPoses() found nodes=3
[DEBUG] (2023-10-30 09:20:51.055) Memory.cpp:2776::computeTransform() 
[DEBUG] (2023-10-30 09:20:51.055) SensorData.cpp:506::uncompressData() 1033 data(0,0,0,0,0,0,0)
[DEBUG] (2023-10-30 09:20:51.055) SensorData.cpp:506::uncompressData() 1054 data(0,0,0,0,0,0,0)
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:282::computeTransformationImpl() Vis/MinInliers=30
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:283::computeTransformationImpl() Vis/InlierDistance=0.100000
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:284::computeTransformationImpl() Vis/Iterations=300
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:285::computeTransformationImpl() Vis/EstimationType=1
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:286::computeTransformationImpl() Vis/ForwardEstOnly=1
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:287::computeTransformationImpl() Vis/EpipolarGeometryVar=0.100000
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:288::computeTransformationImpl() Vis/PnPReprojError=2.000000
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:289::computeTransformationImpl() Vis/PnPFlags=0
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:290::computeTransformationImpl() Vis/CorType=0
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:291::computeTransformationImpl() Vis/CorFlowWinSize=16
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:292::computeTransformationImpl() Vis/CorFlowIterations=30
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:293::computeTransformationImpl() Vis/CorFlowEps=0.010000
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:294::computeTransformationImpl() Vis/CorFlowMaxLevel=3
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:295::computeTransformationImpl() Vis/CorNNDR=0.800000
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:296::computeTransformationImpl() Vis/CorNNType=1
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:297::computeTransformationImpl() Vis/CorGuessWinSize=40
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:298::computeTransformationImpl() Vis/CorGuessMatchToProjection=0
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:299::computeTransformationImpl() Feature Detector = 11
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:300::computeTransformationImpl() guess=xyz=[null] rpy=[null]
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:302::computeTransformationImpl() Input(1033): from=360 words, 360 3D words, 360 words descriptors,  0 kpts, 0 kpts3D, 0 descriptors, image=0x0 models=1 stereo=0
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:315::computeTransformationImpl() Input(1054): to=348 words, 348 3D words, 348 words descriptors, 0 kpts, 0 kpts3D, 0 descriptors, image=848x480 models=1 stereo=0
[DEBUG] (2023-10-30 09:20:51.055) RegistrationVis.cpp:346::computeTransformationImpl() 
[DEBUG] (2023-10-30 09:20:51.056) RegistrationVis.cpp:438::computeTransformationImpl() IDs are not unique, IDs will be regenerated!
[DEBUG] (2023-10-30 09:20:51.056) RegistrationVis.cpp:599::computeTransformationImpl() 
[DEBUG] (2023-10-30 09:20:51.056) RegistrationVis.cpp:649::computeTransformationImpl() kptsFrom=360 kptsFromSource=2
[DEBUG] (2023-10-30 09:20:51.056) RegistrationVis.cpp:650::computeTransformationImpl() kptsTo=348 kptsToSource=2
[DEBUG] (2023-10-30 09:20:51.056) RegistrationVis.cpp:783::computeTransformationImpl() descriptorsFrom=360
[DEBUG] (2023-10-30 09:20:51.056) RegistrationVis.cpp:784::computeTransformationImpl() descriptorsTo=348
[DEBUG] (2023-10-30 09:20:51.056) RegistrationVis.cpp:785::computeTransformationImpl() orignalWordsFromIds=0
[DEBUG] (2023-10-30 09:20:51.056) RegistrationVis.cpp:1165::computeTransformationImpl() 
[DEBUG] (2023-10-30 09:20:51.056) RegistrationVis.cpp:1265::computeTransformationImpl() VWDictionary knn matching
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:470::update() incremental=1
[DEBUG] (2023-10-30 09:20:51.056) FlannIndex.cpp:52::release() 
[DEBUG] (2023-10-30 09:20:51.056) FlannIndex.cpp:80::release() 
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:680::update() Dictionary updated! (size=0 added=0 removed=0)
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:689::update() 
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:763::addNewWords() id=1 descriptors=360
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:1059::addNewWords() naive search and add ref/words time = 0.000406 s
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:1061::addNewWords() 360 new words added...
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:1062::addNewWords() 0 duplicated words added (from current image = 0)...
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:1064::addNewWords() total time 0.000453s
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:470::update() incremental=1
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:491::update() Incremental FLANN: Removing 0 words...
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:500::update() Incremental FLANN: Removing 0 words... done!
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:504::update() Incremental FLANN: Inserting 360 words...
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:531::update() Building FLANN index...
[DEBUG] (2023-10-30 09:20:51.056) FlannIndex.cpp:208::buildKDTreeIndex() 
[DEBUG] (2023-10-30 09:20:51.056) FlannIndex.cpp:52::release() 
[DEBUG] (2023-10-30 09:20:51.056) FlannIndex.cpp:80::release() 
[DEBUG] (2023-10-30 09:20:51.056) FlannIndex.cpp:260::buildKDTreeIndex() 
[DEBUG] (2023-10-30 09:20:51.056) VWDictionary.cpp:549::update() Building FLANN index... done!
[DEBUG] (2023-10-30 09:20:51.056) FlannIndex.cpp:433::addPoints() Rebuilding FLANN index: 1 -> 3
[DEBUG] (2023-10-30 09:20:51.056) FlannIndex.cpp:433::addPoints() Rebuilding FLANN index: 3 -> 7
[DEBUG] (2023-10-30 09:20:51.056) FlannIndex.cpp:433::addPoints() Rebuilding FLANN index: 7 -> 15
[DEBUG] (2023-10-30 09:20:51.057) FlannIndex.cpp:433::addPoints() Rebuilding FLANN index: 15 -> 31
[DEBUG] (2023-10-30 09:20:51.057) FlannIndex.cpp:433::addPoints() Rebuilding FLANN index: 31 -> 63
[DEBUG] (2023-10-30 09:20:51.058) FlannIndex.cpp:433::addPoints() Rebuilding FLANN index: 63 -> 127
[DEBUG] (2023-10-30 09:20:51.059) FlannIndex.cpp:433::addPoints() Rebuilding FLANN index: 127 -> 255
[DEBUG] (2023-10-30 09:20:51.061) VWDictionary.cpp:564::update() Incremental FLANN: Inserting 360 words... done!
[DEBUG] (2023-10-30 09:20:51.061) VWDictionary.cpp:680::update() Dictionary updated! (size=0 added=360 removed=0)
[DEBUG] (2023-10-30 09:20:51.061) VWDictionary.cpp:689::update() 
[DEBUG] (2023-10-30 09:20:51.061) VWDictionary.cpp:763::addNewWords() id=2 descriptors=348
[DEBUG] (2023-10-30 09:20:51.061) VWDictionary.cpp:857::addNewWords() newPts.total()=348 
[DEBUG] (2023-10-30 09:20:51.067) VWDictionary.cpp:924::addNewWords() Time to find nn = 0.005433 s
[DEBUG] (2023-10-30 09:20:51.067) VWDictionary.cpp:1059::addNewWords() naive search and add ref/words time = 0.000454 s
[DEBUG] (2023-10-30 09:20:51.067) VWDictionary.cpp:1061::addNewWords() 105 new words added...
[DEBUG] (2023-10-30 09:20:51.067) VWDictionary.cpp:1062::addNewWords() 243 duplicated words added (from current image = 0)...
[DEBUG] (2023-10-30 09:20:51.067) VWDictionary.cpp:1064::addNewWords() total time 0.005917s
[DEBUG] (2023-10-30 09:20:51.067) VWDictionary.cpp:694::clear() 
[DEBUG] (2023-10-30 09:20:51.067) FlannIndex.cpp:52::release() 
[DEBUG] (2023-10-30 09:20:51.067) FlannIndex.cpp:80::release() 
[DEBUG] (2023-10-30 09:20:51.067) VWDictionary.cpp:694::clear() 
[DEBUG] (2023-10-30 09:20:51.067) FlannIndex.cpp:52::release() 
[DEBUG] (2023-10-30 09:20:51.067) FlannIndex.cpp:80::release() 
[DEBUG] (2023-10-30 09:20:51.067) FlannIndex.cpp:52::release() 
[DEBUG] (2023-10-30 09:20:51.067) FlannIndex.cpp:80::release() 
[DEBUG] (2023-10-30 09:20:51.068) RegistrationVis.cpp:1481::computeTransformationImpl() 
[DEBUG] (2023-10-30 09:20:51.068) RegistrationVis.cpp:1494::computeTransformationImpl() words from3D=360 to2D=308
[DEBUG] (2023-10-30 09:20:51.069) util3d_motion_estimation.cpp:97::estimateMotion3DTo2D() words3A=360 words2B=308 matches=203 words3B=308 guess=xyz=0.000000,0.000000,0.000000 rpy=0.000000,-0.000000,0.000000 reprojError=2.000000 iterations=300
[DEBUG] (2023-10-30 09:20:51.071) RegistrationVis.cpp:1537::computeTransformationImpl() inliers: 176/203
[DEBUG] (2023-10-30 09:20:51.071) RegistrationVis.cpp:1658::computeTransformationImpl() Refine with bundle adjustment
[DEBUG] (2023-10-30 09:20:51.071) OptimizerG2O.cpp:1353::optimizeBA() Optimizing graph...
[DEBUG] (2023-10-30 09:20:51.071) OptimizerG2O.cpp:1433::optimizeBA() fill poses to g2o...
[DEBUG] (2023-10-30 09:20:51.071) OptimizerG2O.cpp:1486::optimizeBA() fill edges to g2o...
[DEBUG] (2023-10-30 09:20:51.071) OptimizerG2O.cpp:1580::optimizeBA() fill 3D points to g2o...
[DEBUG] (2023-10-30 09:20:51.071) OptimizerG2O.cpp:1587::optimizeBA() stepVertexId=3, negVertexOffset=349
[DEBUG] (2023-10-30 09:20:51.072) OptimizerG2O.cpp:1713::optimizeBA() Initial optimization...
[DEBUG] (2023-10-30 09:20:51.072) OptimizerG2O.cpp:1718::optimizeBA() g2o optimizing begin (max iterations=20, robustKernel=8.000000)
[DEBUG] (2023-10-30 09:20:51.073) OptimizerG2O.cpp:1738::optimizeBA() iteration 0: 178 nodes, 353 edges, chi2: 237.856292
[DEBUG] (2023-10-30 09:20:51.074) OptimizerG2O.cpp:1793::optimizeBA() outliers=0 outliersCountFar=0
[DEBUG] (2023-10-30 09:20:51.074) OptimizerG2O.cpp:1738::optimizeBA() iteration 1: 178 nodes, 353 edges, chi2: 237.856292
[DEBUG] (2023-10-30 09:20:51.075) OptimizerG2O.cpp:1793::optimizeBA() outliers=0 outliersCountFar=0
[DEBUG] (2023-10-30 09:20:51.075) OptimizerG2O.cpp:1796::optimizeBA() g2o optimizing end (7 iterations done, error=237.856292, outliers=0/352 (delta=8.000000) time = 0.002376 s)
[DEBUG] (2023-10-30 09:20:51.075) OptimizerG2O.cpp:1891::optimizeBA() Optimizing graph...end!
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:1800::computeTransformationImpl() Pose optimization: xyz=0.068028,-0.032343,0.000000 rpy=0.000000,0.000000,-0.022635 -> xyz=0.065436,-0.037246,0.000000 rpy=0.000000,0.000000,-0.013245
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:1981::computeTransformationImpl() inliers=176/203
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:1982::computeTransformationImpl() transform=xyz=0.065436,-0.037246,0.000000 rpy=0.000000,0.000000,-0.013245
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:282::computeTransformationImpl() Vis/MinInliers=30
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:283::computeTransformationImpl() Vis/InlierDistance=0.100000
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:284::computeTransformationImpl() Vis/Iterations=300
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:285::computeTransformationImpl() Vis/EstimationType=1
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:286::computeTransformationImpl() Vis/ForwardEstOnly=1
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:287::computeTransformationImpl() Vis/EpipolarGeometryVar=0.100000
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:288::computeTransformationImpl() Vis/PnPReprojError=2.000000
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:289::computeTransformationImpl() Vis/PnPFlags=0
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:290::computeTransformationImpl() Vis/CorType=0
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:291::computeTransformationImpl() Vis/CorFlowWinSize=16
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:292::computeTransformationImpl() Vis/CorFlowIterations=30
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:293::computeTransformationImpl() Vis/CorFlowEps=0.010000
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:294::computeTransformationImpl() Vis/CorFlowMaxLevel=3
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:295::computeTransformationImpl() Vis/CorNNDR=0.800000
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:296::computeTransformationImpl() Vis/CorNNType=1
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:297::computeTransformationImpl() Vis/CorGuessWinSize=40
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:298::computeTransformationImpl() Vis/CorGuessMatchToProjection=0
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:299::computeTransformationImpl() Feature Detector = 11
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:300::computeTransformationImpl() guess=xyz=0.065436,-0.037246,0.000000 rpy=0.000000,0.000000,-0.013245
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:302::computeTransformationImpl() Input(1033): from=360 words, 360 3D words, 360 words descriptors,  360 kpts, 360 kpts3D, 360 descriptors, image=0x0 models=1 stereo=0
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:315::computeTransformationImpl() Input(1054): to=308 words, 308 3D words, 308 words descriptors, 348 kpts, 348 kpts3D, 348 descriptors, image=848x480 models=1 stereo=0
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:346::computeTransformationImpl() 
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:599::computeTransformationImpl() 
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:649::computeTransformationImpl() kptsFrom=360 kptsFromSource=2
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:650::computeTransformationImpl() kptsTo=308 kptsToSource=2
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:783::computeTransformationImpl() descriptorsFrom=360
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:784::computeTransformationImpl() descriptorsTo=308
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:785::computeTransformationImpl() orignalWordsFromIds=360
[DEBUG] (2023-10-30 09:20:51.075) RegistrationVis.cpp:806::computeTransformationImpl() 
[DEBUG] (2023-10-30 09:20:51.076) RegistrationVis.cpp:828::computeTransformationImpl() Projected points=360
[DEBUG] (2023-10-30 09:20:51.076) RegistrationVis.cpp:846::computeTransformationImpl() corners in frame=324
[DEBUG] (2023-10-30 09:20:51.076) RegistrationVis.cpp:851::computeTransformationImpl() guessMatchToProjection=0, cornersProjected=324
[DEBUG] (2023-10-30 09:20:51.076) RegistrationVis.cpp:998::computeTransformationImpl() match projected to frame
[DEBUG] (2023-10-30 09:20:51.079) RegistrationVis.cpp:1016::computeTransformationImpl() radius search done for guess
[DEBUG] (2023-10-30 09:20:51.082) RegistrationVis.cpp:1108::computeTransformationImpl() bruteForceDescCopy=0.000848s, bruteForceTotalTime=0.000927s
[DEBUG] (2023-10-30 09:20:51.082) RegistrationVis.cpp:1144::computeTransformationImpl() 
[DEBUG] (2023-10-30 09:20:51.082) RegistrationVis.cpp:1481::computeTransformationImpl() 
[DEBUG] (2023-10-30 09:20:51.082) RegistrationVis.cpp:1494::computeTransformationImpl() words from3D=360 to2D=308
[DEBUG] (2023-10-30 09:20:51.083) util3d_motion_estimation.cpp:97::estimateMotion3DTo2D() words3A=360 words2B=308 matches=224 words3B=308 guess=xyz=0.065436,-0.037246,0.000000 rpy=0.000000,0.000000,-0.013245 reprojError=2.000000 iterations=300
[DEBUG] (2023-10-30 09:20:51.086) RegistrationVis.cpp:1537::computeTransformationImpl() inliers: 181/224
[DEBUG] (2023-10-30 09:20:51.086) RegistrationVis.cpp:1658::computeTransformationImpl() Refine with bundle adjustment
[DEBUG] (2023-10-30 09:20:51.086) OptimizerG2O.cpp:1353::optimizeBA() Optimizing graph...
[DEBUG] (2023-10-30 09:20:51.087) OptimizerG2O.cpp:1433::optimizeBA() fill poses to g2o...
[DEBUG] (2023-10-30 09:20:51.087) OptimizerG2O.cpp:1486::optimizeBA() fill edges to g2o...
[DEBUG] (2023-10-30 09:20:51.087) OptimizerG2O.cpp:1580::optimizeBA() fill 3D points to g2o...
[DEBUG] (2023-10-30 09:20:51.087) OptimizerG2O.cpp:1587::optimizeBA() stepVertexId=3, negVertexOffset=347
[DEBUG] (2023-10-30 09:20:51.087) OptimizerG2O.cpp:1713::optimizeBA() Initial optimization...
[DEBUG] (2023-10-30 09:20:51.088) OptimizerG2O.cpp:1718::optimizeBA() g2o optimizing begin (max iterations=20, robustKernel=8.000000)
[DEBUG] (2023-10-30 09:20:51.089) OptimizerG2O.cpp:1738::optimizeBA() iteration 0: 183 nodes, 363 edges, chi2: 242.142814
[DEBUG] (2023-10-30 09:20:51.089) OptimizerG2O.cpp:1793::optimizeBA() outliers=0 outliersCountFar=0
[DEBUG] (2023-10-30 09:20:51.091) OptimizerG2O.cpp:1738::optimizeBA() iteration 1: 183 nodes, 363 edges, chi2: 242.142814
[DEBUG] (2023-10-30 09:20:51.091) OptimizerG2O.cpp:1793::optimizeBA() outliers=0 outliersCountFar=0
[DEBUG] (2023-10-30 09:20:51.091) OptimizerG2O.cpp:1796::optimizeBA() g2o optimizing end (9 iterations done, error=242.142814, outliers=0/362 (delta=8.000000) time = 0.003353 s)
[DEBUG] (2023-10-30 09:20:51.091) OptimizerG2O.cpp:1891::optimizeBA() Optimizing graph...end!
[DEBUG] (2023-10-30 09:20:51.091) RegistrationVis.cpp:1800::computeTransformationImpl() Pose optimization: xyz=0.054758,-0.037343,0.000000 rpy=0.000000,0.000000,-0.014358 -> xyz=0.062884,-0.042192,0.000000 rpy=0.000000,0.000000,-0.004366
[DEBUG] (2023-10-30 09:20:51.092) RegistrationVis.cpp:1981::computeTransformationImpl() inliers=181/224
[DEBUG] (2023-10-30 09:20:51.092) RegistrationVis.cpp:1982::computeTransformationImpl() transform=xyz=0.062884,-0.042192,0.000000 rpy=0.000000,0.000000,-0.004366
[ INFO] (2023-10-30 09:20:51.092) Rtabmap.cpp:2516::process() [Visual] Add local loop closure in SPACE (1054->1033) xyz=-0.063067,0.041917,-0.000000 rpy=0.000000,-0.000000,0.004366
[ INFO] (2023-10-30 09:20:51.092) Memory.cpp:3286::addLink() to=1033, from=1054 transform: xyz=-0.063067,0.041917,-0.000000 rpy=0.000000,-0.000000,0.004366 var=0.000027

Here is the log of the local loop closure that crashes:

[DEBUG] (2023-10-30 09:20:51.092) Memory.cpp:3298::addLink() Add link between 1033 and 1054
[DEBUG] (2023-10-30 09:20:51.092) Signature.cpp:121::addLink() Add link 1054 to 1033 (type=2/LocalSpaceClosure var=0.000027,0.006703)
[DEBUG] (2023-10-30 09:20:51.092) Signature.cpp:121::addLink() Add link 1033 to 1054 (type=2/LocalSpaceClosure var=0.000027,0.006703)
[ INFO] (2023-10-30 09:20:51.092) Rtabmap.cpp:2569::process() timeProximityBySpaceVisualDetection=0.118137s
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2574::process() Proximity detection (local loop closure in SPACE with scan matching)
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2577::process() Proximity by scan matching is disabled (RGBD/ProximityPathMaxNeighbors=0).
[ INFO] (2023-10-30 09:20:51.092) Rtabmap.cpp:2759::process() timeProximityBySpaceDetection=0.000138s
[ INFO] (2023-10-30 09:20:51.092) Rtabmap.cpp:2823::process() timeAddLoopClosureLink=0.000021s
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2863::process() RGB-D SLAM mode: 1
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2864::process() Incremental: 0
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2865::process() Loop hyp: 0
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2866::process() Last prox: 1033
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2867::process() Reduced ids: 0
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2868::process() Has prior: 0 (prior ignored=1)
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2869::process() Has gravity: 0 (sigma=0.300000, odomGravity=0, refined=0)
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2870::process() Has virtual link: 0
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2871::process() Prox Time: 0
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2872::process() Landmarks: 1
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2873::process() Retrieved: 0
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2874::process() Not self ref links: 3
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2946::process() Constraint -8->-8 (type=PosePrior)
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2948::process() Constraint 1054->-8 (type=Landmark, var = 0.001000 0.010000)
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2946::process() Constraint 875->875 (type=PosePrior)
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2948::process() Constraint 1054->875 (type=LocalSpaceClosure, var = 0.000002 0.002147)
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2946::process() Constraint 876->876 (type=PosePrior)
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2948::process() Constraint 1054->876 (type=GlobalClosure, var = 0.000003 0.002008)
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2946::process() Constraint 1033->1033 (type=PosePrior)
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2948::process() Constraint 1054->1033 (type=LocalSpaceClosure, var = 0.000027 0.006703)
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2952::process() Pose -8 xyz=0.851523,-0.041897,0.404141 rpy=0.085773,0.006991,1.561444
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2952::process() Pose 875 xyz=0.319759,-0.040769,0.000000 rpy=0.000000,-0.000000,0.013201
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2952::process() Pose 876 xyz=0.297985,-0.013632,0.000000 rpy=0.000000,0.000000,-0.039805
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2952::process() Pose 1033 xyz=0.240514,-0.004088,0.000000 rpy=0.000000,-0.000000,0.023084
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2952::process() Pose 1054 xyz=-197.406799,29.594700,0.000000 rpy=0.000000,0.000000,-3.107898
[DEBUG] (2023-10-30 09:20:51.092) Rtabmap.cpp:2959::process() priorsIgnored was true
[DEBUG] (2023-10-30 09:20:51.093) Optimizer.cpp:196::getConnectedGraph() IN: fromId=1054 poses=5 links=8 priorsIgnored=0 landmarksIgnored=0
[DEBUG] (2023-10-30 09:20:51.093) Optimizer.cpp:301::getConnectedGraph() OUT: poses=5 links=8
[DEBUG] (2023-10-30 09:20:51.093) OptimizerGTSAM.cpp:103::optimize() Optimizing graph...
[DEBUG] (2023-10-30 09:20:51.093) OptimizerGTSAM.cpp:165::optimize() fill poses to gtsam... rootId=0 (priorsIgnored=0 landmarksIgnored=0)
[DEBUG] (2023-10-30 09:20:51.093) OptimizerGTSAM.cpp:225::optimize() fill edges to gtsam...
[DEBUG] (2023-10-30 09:20:51.093) OptimizerGTSAM.cpp:481::optimize() create optimizer
[DEBUG] (2023-10-30 09:20:51.093) OptimizerGTSAM.cpp:506::optimize() GTSAM optimizing begin (max iterations=20, robust=0)
[DEBUG] (2023-10-30 09:20:51.093) OptimizerGTSAM.cpp:586::optimize() iteration 1 error =116755840167.375732
[DEBUG] (2023-10-30 09:20:51.093) OptimizerGTSAM.cpp:586::optimize() iteration 2 error =106854122897.797043
[DEBUG] (2023-10-30 09:20:51.093) OptimizerGTSAM.cpp:586::optimize() iteration 3 error =140041458973.719299
[DEBUG] (2023-10-30 09:20:51.093) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-33187336075.922256 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.093) OptimizerGTSAM.cpp:586::optimize() iteration 4 error =237064370786.200806
[DEBUG] (2023-10-30 09:20:51.093) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-97022911812.481506 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.093) OptimizerGTSAM.cpp:586::optimize() iteration 5 error =660572625122.000732
[DEBUG] (2023-10-30 09:20:51.093) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-423508254335.799927 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:586::optimize() iteration 6 error =8353987175376.525391
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-7693414550254.524414 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:586::optimize() iteration 7 error =1177088229069688.250000
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-1168734241894311.750000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:586::optimize() iteration 8 error =4303712320041369088.000000
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-4302535231812299264.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:586::optimize() iteration 9 error =341027594465994673045372928.000000
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-341027590162282342594379776.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:586::optimize() iteration 10 error =104239704971904607249312529500755460096.000000
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-104239704971563576831385615202003189760.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:586::optimize() iteration 11 error =2002174876782583377093932758402499248104242791055360.000000
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-2002174876782479032696263642506476299250010784006144.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:586::optimize() iteration 12 error =462248347965546868089978960768110695681945294388033405381194547200.000000
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-462248347965544903831778388034588629929530123703654987807856787456.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.094) OptimizerGTSAM.cpp:586::optimize() iteration 13 error =17391403043558547904467810400929903064855269978465714362059622440575400207712256.000000
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-17391403043558087163191760463238099673484153920324319196017245038213662597185536.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:586::optimize() iteration 14 error =10063112265575718969030346657411877486125346794332217103643198393522670850685210146527395184640.000000
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-10063112265575702294969496483879736491903505543273486232765406541281449168793113007036727033856.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:586::optimize() iteration 15 error =1562517372895668037238217827074627142754482619981478021166238627662852568042396396036627388564657449992192.000000
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-1562517372885604824885628423997700237498029240693613635867979694342678704705029583118134034819023194030080.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:586::optimize() iteration 16 error =6179402150745495536086924898086180611781173250555794901901578104500542873754946619590869181161311923783565280005849088.000000
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-6179402150743933314129255879649316795790476760671805054167256936449950131357024374760721018901908761732008334042595328.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:586::optimize() iteration 17 error =76383264323005122494975806642350008672142428458020328328900312592201852579877655104205761841700289181289504732841667014633319301120.000000
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-76383264322998946230504521807733483932606730946556382079769667133436651684886539587318079653936949995054051522314970575695838707712.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:586::optimize() iteration 18 error =111526861800517668649072414390829699869001195165070606705988634164297407287382807163611693563932347173436544232830781415602554230640529612210176.000000
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-111526861800441290263592180068994750740772378656177506103427890967078257982685563767273235470597667878468394262890379842088101816622803339706368.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:586::optimize() iteration 19 error =1875764391509334882195830764842771267272892743019005833080960615141279168978130297993596094944065695407778942469669628559998326933795831390983603153448992768.000000
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-1875764391509223227859253300765159596375879877085462822917898989586200832431600728423347780573131216686075133491697237289146450451800181213142534859064868864.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:586::optimize() iteration 20 error =9533322152532853640030056449926005768831342519803140304388757955517729352799655675363912919820522775307173340796235566347342387034234112245613383980468574289831449853952.000000
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:592::optimize() Negative improvement?! Ignore and continue optimizing... (-9533322152530978222895855729156702852084718477076059128094602594494020269642715093620206382437142613139717245179287423141178552297870943421255062953997316883176723841024.000000 < 0.000010)
[DEBUG] (2023-10-30 09:20:51.095) OptimizerGTSAM.cpp:615::optimize() GTSAM optimizing end (20 iterations done, error=9533322152532853640030056449926005768831342519803140304388757955517729352799655675363912919820522775307173340796235566347342387034234112245613383980468574289831449853952.000000 (initial=17789721126.488804 final=9533322152532853640030056449926005768831342519803140304388757955517729352799655675363912919820522775307173340796235566347342387034234112245613383980468574289831449853952.000000), time=0.002738 s)
[DEBUG] (2023-10-30 09:20:51.096) OptimizerGTSAM.cpp:675::optimize() Computing marginals...
[ WARN] (2023-10-30 09:20:51.096) OptimizerGTSAM.cpp:708::optimize() GTSAM exception caught: 
Indeterminant linear system detected while working near variable
1054 (Symbol: 1054).

Thrown when a linear system is ill-posed.  The most common cause for this
error is having underconstrained variables.  Mathematically, the system is
underdetermined.  See the GTSAM Doxygen documentation at
http://borg.cc.gatech.edu/ on gtsam::IndeterminantLinearSystemException for
more information.
[DEBUG] (2023-10-30 09:20:51.096) OptimizerGTSAM.cpp:725::optimize() Optimizing graph...end!
[DEBUG] (2023-10-30 09:20:51.096) Rtabmap.cpp:2967::process() Opt  -8 xyz=-inf,inf,0.404141 rpy=0.085773,0.006991,0.785367
[DEBUG] (2023-10-30 09:20:51.096) Rtabmap.cpp:2967::process() Opt  875 xyz=inf,-inf,0.000000 rpy=0.000000,-0.000000,0.465234
[DEBUG] (2023-10-30 09:20:51.096) Rtabmap.cpp:2967::process() Opt  876 xyz=inf,-inf,0.000000 rpy=0.000000,0.000000,-2.542567
[DEBUG] (2023-10-30 09:20:51.096) Rtabmap.cpp:2967::process() Opt  1033 xyz=-inf,inf,0.000000 rpy=0.000000,-0.000000,-2.975477
[DEBUG] (2023-10-30 09:20:51.096) Rtabmap.cpp:2967::process() Opt  1054 xyz=-inf,-inf,0.000000 rpy=0.000000,0.000000,-1.844153
[ INFO] (2023-10-30 09:20:51.096) Rtabmap.cpp:2977::process() Compute max graph errors...
[DEBUG] (2023-10-30 09:20:51.096) Graph.cpp:909::computeMaxGraphErrors() poses=5 links=8
[FATAL] (2023-10-30 09:20:51.096) Transform.cpp:184::inverse() Condition (invertible) not met! [This transform is not invertible! xyz=-inf,-inf,0.000000 rpy=0.000000,0.000000,-1.844153 
[-0.269965 0.962870 -0.000000 -inf;
 -0.962870 -0.269965 0.000000 -inf;
 -0.000000 0.000000 1.000000 0.000000;
 0 0 0 1]]
terminate called after throwing an instance of 'UException'
  what():  [FATAL] (2023-10-30 09:20:51.096) Transform.cpp:184::inverse() Condition (invertible) not met! [This transform is not invertible! xyz=-inf,-inf,0.000000 rpy=0.000000,0.000000,-1.844153 
[-0.269965 0.962870 -0.000000 -inf;
 -0.962870 -0.269965 0.000000 -inf;
 -0.000000 0.000000 1.000000 0.000000;
 0 0 0 1]]

Hope these are useful.
Thanks.

@matlabbe
Copy link
Member

For the inverse() error, that commit should fix that. The Indeterminant linear system detected while working near variable would still happen though, but at least rtabmap won't crash and only reject the localization. For GTSAM error, I would need to make some tests by artificially use large odometry values and see if I can reproduce the same error. I also saw this GTSAM error when the scale of the sensor changed (e..g, switching a camera to a new one with wrong calibration), make it difficult for the optimizer to correctly converge to a solution.

@ApoloM
Copy link
Author

ApoloM commented Nov 1, 2023

Hi, thanks for the response.
I tried adding this commit to the version 0.20.18 as a patch. However, it still crashing, as shown below:

[DEBUG] (2023-11-01 13:45:03.429) OptimizerGTSAM.cpp:615::optimize() GTSAM optimizing end (20 iterations done, error=370665897151686591118042194865150652730418506596897319548013708715507774486627007756496375995357581661849455872498980344507130287593126704789557833885327921244395869831168.000000 (initial=16453009441.994427 final=370665897151686591118042194865150652730418506596897319548013708715507774486627007756496375995357581661849455872498980344507130287593126704789557833885327921244395869831168.000000), time=0.002583 s)
[DEBUG] (2023-11-01 13:45:03.429) OptimizerGTSAM.cpp:675::optimize() Computing marginals...
[DEBUG] (2023-11-01 13:45:03.430) OptimizerGTSAM.cpp:679::optimize() Computed marginals = 0.000114s (key=1040)
[DEBUG] (2023-11-01 13:45:03.430) OptimizerGTSAM.cpp:725::optimize() Optimizing graph...end!
[DEBUG] (2023-11-01 13:45:03.430) Rtabmap.cpp:2967::process() Opt  -8 xyz=inf,inf,0.404141 rpy=0.085773,0.006991,-2.172289
[DEBUG] (2023-11-01 13:45:03.430) Rtabmap.cpp:2967::process() Opt  875 xyz=-inf,-inf,0.000000 rpy=0.000000,-0.000000,-0.380802
[DEBUG] (2023-11-01 13:45:03.430) Rtabmap.cpp:2967::process() Opt  876 xyz=inf,inf,0.000000 rpy=0.000000,0.000000,-2.488718
[DEBUG] (2023-11-01 13:45:03.430) Rtabmap.cpp:2967::process() Opt  1033 xyz=-inf,-inf,0.000000 rpy=0.000000,-0.000000,-1.994097
[DEBUG] (2023-11-01 13:45:03.430) Rtabmap.cpp:2967::process() Opt  1040 xyz=inf,-inf,0.000000 rpy=0.000000,0.000000,-0.070792
[ INFO] (2023-11-01 13:45:03.430) Rtabmap.cpp:2977::process() Compute max graph errors...
[DEBUG] (2023-11-01 13:45:03.430) Graph.cpp:909::computeMaxGraphErrors() poses=5 links=8
[ WARN] (2023-11-01 13:45:03.430) Graph.cpp:923::computeMaxGraphErrors() Poses are null or not invertible, aborting optimized graph max error check! (Pose 1040=xyz=inf,-inf,0.000000 rpy=0.000000,0.000000,-0.070792 Pose 875=xyz=-inf,-inf,0.000000 rpy=0.000000,-0.000000,-0.380802)
[FATAL] (2023-11-01 13:45:03.430) Transform.cpp:184::inverse() Condition (invertible) not met! [This transform is not invertible! xyz=inf,-inf,0.000000 rpy=0.000000,0.000000,-0.070792 
[0.997495 0.070733 0.000000 inf;
 -0.070733 0.997495 -0.000000 -inf;
 -0.000000 0.000000 1.000000 0.000000;
 0 0 0 1]]

I'm using ROS noetic, so the actual master branch is not compatible with the system I'm working on.
Then, I decided to change the solver used by the GTSAM from GaussNewton to Levenberg by changing the parameter:

GTSAM/Optimizer = "0"                               [0=Levenberg 1=GaussNewton 2=Dogleg]

As a result the Optimizer stopped crashing. It is rejecting the loop closures instead, as shown below:

[DEBUG] (2023-11-01 15:36:28.884) OptimizerGTSAM.cpp:615::optimize() GTSAM optimizing end (0 iterations done, error=10810002659.847393 (initial=10810002659.847393 final=10810002659.847393), time=0.000671 s)
[DEBUG] (2023-11-01 15:36:28.884) OptimizerGTSAM.cpp:675::optimize() Computing marginals...
[DEBUG] (2023-11-01 15:36:28.884) OptimizerGTSAM.cpp:679::optimize() Computed marginals = 0.000089s (key=1039)
[DEBUG] (2023-11-01 15:36:28.884) OptimizerGTSAM.cpp:725::optimize() Optimizing graph...end!
[DEBUG] (2023-11-01 15:36:28.884) Rtabmap.cpp:3081::process() Opt2  -8 xyz=0.851523,-0.041897,0.404141 rpy=0.085773,0.006991,1.561444
[DEBUG] (2023-11-01 15:36:28.884) Rtabmap.cpp:3081::process() Opt2  876 xyz=0.297985,-0.013632,0.000000 rpy=0.000000,0.000000,-0.039805
[DEBUG] (2023-11-01 15:36:28.884) Rtabmap.cpp:3081::process() Opt2  1039 xyz=-197.406799,29.594700,0.000000 rpy=0.000000,0.000000,-3.107898
[ INFO] (2023-11-01 15:36:28.884) Rtabmap.cpp:3091::process() Compute max graph errors...
[DEBUG] (2023-11-01 15:36:28.884) Graph.cpp:909::computeMaxGraphErrors() poses=3 links=4
[ INFO] (2023-11-01 15:36:28.884) Rtabmap.cpp:3112::process() Max optimization linear error = 196.587189 m (link 1039->876, var=0.000002, ratio error/std=144468.220376, thr=3.000000)
[ WARN] (2023-11-01 15:36:28.884) Rtabmap.cpp:3121::process() Rejecting localization (1039 <-> 876) in this iteration because a wrong loop closure has been detected after graph optimization, resulting in a maximum graph error ratio of 144468.218750 (edge 1039->876, type=1, abs error=196.587189 m, stddev=0.001361). The maximum error ratio parameter "RGBD/OptimizeMaxError" is 3.000000 of std deviation.

Do you have any thoughts about this behavior?
Thanks.

EDIT: I also noted that, this behavior occurs mostly when detected apriltags (landmarks).

@matlabbe
Copy link
Member

matlabbe commented Nov 1, 2023

The master branch should work on Noetic (this is the main platform I am developing on). Otherwise, please report the compatibility issue.

For:

[ WARN] (2023-11-01 13:45:03.430) Graph.cpp:923::computeMaxGraphErrors() Poses are null or not invertible, aborting optimized graph max error check! (Pose 1040=xyz=inf,-inf,0.000000 rpy=0.000000,0.000000,-0.070792 Pose 875=xyz=-inf,-inf,0.000000 rpy=0.000000,-0.000000,-0.380802)
[FATAL] (2023-11-01 13:45:03.430) Transform.cpp:184::inverse() Condition (invertible) not met! [This transform is not invertible! xyz=inf,-inf,0.000000 rpy=0.000000,0.000000,-0.070792 
[0.997495 0.070733 0.000000 inf;
 -0.070733 0.997495 -0.000000 -inf;
 -0.000000 0.000000 1.000000 0.000000;
 0 0 0 1]]

That update introlab/rtabmap#1030 is also required. I'll suggest to use latest master branch in case there are other small commits depending on that. Remove rtabmap binaries and rebuild rtabmap and rtabmap_ros from source:

sudo apt remove ros-noetic-rtabmap*

cd ~
git clone https://github.com/introlab/rtabmap.git rtabmap
cd rtabmap/build
cmake ..
make -j6
sudo make install

cd ~/catkin_ws
git clone https://github.com/introlab/rtabmap_ros.git src/rtabmap_ros
catkin_make -j4

Otherwise, it seems GTSAM really doesn't like the constraints you are providing. In your case, do you have Reg/Force3DoF=true? The poses look 3DoF while the tags are 6DoF. Are the tags detected with external node? Normally, rtabmap should handle 6DoF tag constraints even in 3DoF optimization mode. You could try with Reg/Force3DoF=false just to compare though.

@matlabbe
Copy link
Member

matlabbe commented Nov 1, 2023

Just confirming that even if odometry pose is large, GTSAM should be able to converge correctly. Here I reset odometry to x=10000 and y=8000:

[ INFO] (2023-11-01 14:31:20.381) Rtabmap.cpp:2979::process() timeAddLoopClosureLink=0.000003s
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3055::process() RGB-D SLAM mode: 1
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3056::process() Incremental: 0
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3057::process() Loop hyp: 9
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3058::process() Last prox: 9
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3059::process() Reduced ids: 0
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3060::process() Has prior: 0 (prior ignored=1)
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3061::process() Has gravity: 1 (sigma=0.300000, odomGravity=0, refined=0)
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3062::process() Has virtual link: 0
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3063::process() Prox Time: 0
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3064::process() Landmarks: 0
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3065::process() Retrieved: 0
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3066::process() Not self ref links: 1
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3150::process() Constraint 184->184 (type=Gravity, var = 1.000000 1.000000)
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3148::process() Constraint 9->9 (type=PosePrior)
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3150::process() Constraint 184->9 (type=GlobalClosure, var = 0.000223 0.003220)
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3154::process() Pose 9 xyz=-0.291190,-0.002876,-0.033898 rpy=0.183713,0.389845,-2.646930
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3154::process() Pose 184 xyz=10000.000000,8000.000000,0.000000 rpy=0.177814,0.359494,-2.979261
[DEBUG] (2023-11-01 14:31:20.381) Rtabmap.cpp:3161::process() priorsIgnored was true
[DEBUG] (2023-11-01 14:31:20.381) Optimizer.cpp:196::getConnectedGraph() IN: fromId=184 poses=2 links=3 priorsIgnored=0 landmarksIgnored=0
[DEBUG] (2023-11-01 14:31:20.381) Optimizer.cpp:301::getConnectedGraph() OUT: poses=2 links=3
[DEBUG] (2023-11-01 14:31:20.381) OptimizerGTSAM.cpp:101::optimize() Optimizing graph...
[DEBUG] (2023-11-01 14:31:20.381) OptimizerGTSAM.cpp:163::optimize() fill poses to gtsam... rootId=0 (priorsIgnored=0 landmarksIgnored=0)
[DEBUG] (2023-11-01 14:31:20.381) OptimizerGTSAM.cpp:223::optimize() fill edges to gtsam...
[DEBUG] (2023-11-01 14:31:20.381) OptimizerGTSAM.cpp:495::optimize() create optimizer
[DEBUG] (2023-11-01 14:31:20.381) OptimizerGTSAM.cpp:520::optimize() GTSAM optimizing begin (max iterations=20, robust=0)
[DEBUG] (2023-11-01 14:31:20.382) OptimizerGTSAM.cpp:604::optimize() iteration 1 error =3529.759225
[DEBUG] (2023-11-01 14:31:20.382) OptimizerGTSAM.cpp:604::optimize() iteration 2 error =0.318584
[DEBUG] (2023-11-01 14:31:20.382) OptimizerGTSAM.cpp:604::optimize() iteration 3 error =0.271166
[DEBUG] (2023-11-01 14:31:20.382) OptimizerGTSAM.cpp:604::optimize() iteration 4 error =0.271165
[DEBUG] (2023-11-01 14:31:20.382) OptimizerGTSAM.cpp:614::optimize() Stop optimizing, not enough improvement (0.000001 < 0.000010)
[DEBUG] (2023-11-01 14:31:20.382) OptimizerGTSAM.cpp:633::optimize() GTSAM optimizing end (4 iterations done, error=0.271165 (initial=84492982519949.375000 final=0.271165), time=0.000274 s)
[DEBUG] (2023-11-01 14:31:20.382) OptimizerGTSAM.cpp:697::optimize() Computing marginals...
[DEBUG] (2023-11-01 14:31:20.382) OptimizerGTSAM.cpp:701::optimize() Computed marginals = 0.000040s (key=184)
[DEBUG] (2023-11-01 14:31:20.382) OptimizerGTSAM.cpp:750::optimize() Optimizing graph...end!
[DEBUG] (2023-11-01 14:31:20.382) Rtabmap.cpp:3170::process() Opt  9 xyz=-0.291190,-0.002876,-0.033898 rpy=0.183711,0.389845,-2.646930
[DEBUG] (2023-11-01 14:31:20.382) Rtabmap.cpp:3170::process() Opt  184 xyz=-0.217603,0.092621,-0.115117 rpy=0.218768,0.354219,-2.363997
[ INFO] (2023-11-01 14:31:20.382) Rtabmap.cpp:3180::process() Compute max graph errors...

Maybe the tag detection is too wrong to correctly converge.

matlabbe added a commit to introlab/rtabmap that referenced this issue Nov 4, 2023
matlabbe added a commit to introlab/rtabmap that referenced this issue Nov 5, 2023
…er) (#1156)

* tuning localization priors (for introlab/rtabmap_ros#1057)

* Set prior error value to same than old default value
@matlabbe
Copy link
Member

matlabbe commented Nov 5, 2023

I updated the code with a change for landmark constraints in localization mode that could help with your issue (see introlab/rtabmap#1156). You may give a try. If the issue is still there, you may increase this new parameter called RGBD/LocalizationPriorError to 0.01 or 0.1.

@ApoloM
Copy link
Author

ApoloM commented Nov 6, 2023

Thanks for the update.

Apparently the landmark constraints are the problem.
By using the Reg/Force3DoF=false and GTSAM/Optimizer=0 I could supress the warnings and crashes.

I'm not using an external node to detect the apriltags.

I will try the suggested version and parameters.

Thanks again.

EDIT: I'm using apriltag_ros built from source. Is there a specific version which I should use?

@ApoloM
Copy link
Author

ApoloM commented Nov 14, 2023

Hello there, I have some updates to share.

As mentioned by me in the previous comment, when I set Reg/Force3DoF=false and GTSAM/Optimizer=0, the warnings and crashes stopped. However, the optimization was converging to weird localization poses. Sometimes the optimization converges for a pitch angle of 90 degrees. However, it does not make any sense, since the robot is a differential robot and its odometry is 2D (constant z=0, pitch=0 and roll=0). Then I tried the master branch as suggested and I got the same behavior.

A few time ago I was using the 0.20.16-noetic version, and it worked just fine. Then, I decided to try it again. As a result the crashes and GTSAM warnings disappeared. Then I decided to look for which commit between versions 0.20.16-noetic and 0.20.18-noetic introduced this behavior in my stack. Then, I found out that the commit fcec981 introduced this issue to my stack.
Then, I reverted the changes one by one and tried each change separately. Then I found out that the following change in Rtabmap.cpp introduced the issue:
image
The function which is called in this part of the code was also changed. It was introduced an "if" statement to skip the transformation of input poses:
image

I'm trying to understand what was the purpose of that change, and why it is necessary for the SLAM 2D.
Could you please explain that part of the code?

The version 0.20.16-noetic is working just fine. Is there any parameter that I should re-parametize to also get it working with the versions 0.20.18-noetic and later?

Thanks for your time.

@matlabbe
Copy link
Member

matlabbe commented Nov 14, 2023

I will try to reproduce the issue that this commit was fixing, I don't remember on top of my head what was the issue addressed there. In your case, if I understand, if you change this line by:

_graphOptimizer->getConnectedGraph(signature->id(), poses, constraints, posesOut, edgeConstraintsOut);

Would it fix your issue?

@ApoloM
Copy link
Author

ApoloM commented Nov 15, 2023

Yes, it would fix my issue.

@matlabbe
Copy link
Member

I removed the last argument of that function and simplified the code. I think I reproduced the issue of switching from a map created with Reg/Force3DoF=true, then try to localize with Reg/Force3DoF=false, this make the robot flips upside down. It is because the tags were transformed for 3DoF optimization, which is not the real pose for 6DoF. Here are two examples of the same tag when mapping with Reg/Force3DoF=true and Reg/Force3DoF=false respectively (the real orientation is the second one, with z axis perpendicular to the tag):
Screenshot from 2023-11-15 22-35-49
Screenshot from 2023-11-15 22-37-04

To make 2D optimization working, as it optimizes only x,y, and yaw, the orientation of the tag should have z-axis up. The bug is that after optimization, we don't flip back the correct orientation of the tag, so if we switch to 6DoF in localization, the robot will think the tag is flipped 90 deg. I need to dig more into this in the next days.

@matlabbe matlabbe added the bug label Nov 16, 2023
@matlabbe
Copy link
Member

matlabbe commented Nov 16, 2023

There is the code flipping the tag pose with Reg/Force3DoF=true:

if(_registrationPipeline->force3DoF())
{
	// For 2D slam, make sure the landmark z axis is up
	rtabmap::Transform tx = landmarkPose.rotation() * rtabmap::Transform(1,0,0,0,0,0);
	rtabmap::Transform ty = landmarkPose.rotation() * rtabmap::Transform(0,1,0,0,0,0);
	if(fabs(tx.z()) > 0.9)
	{
		landmarkPose*=rtabmap::Transform(0,0,0,0,(tx.z()>0?1:-1)*M_PI/2,0);
	}
	else if(fabs(ty.z()) > 0.9)
	{
		landmarkPose*=rtabmap::Transform(0,0,0,(ty.z()>0?-1:1)*M_PI/2,0,0);
	}
}

Well, this works well if we don't change Reg/Force3DoF over time (from 3DoF to 6doF or vice-versa). I'll check to make it work when changing the parameter.

Another note, could you also show the covariance set in your odometry topic (assuming it is called /odom)?

rostopic echo /odom

I saw some GTSAM indeterminant errors while testing on the simulator and it was caused by bad covariance.

@ApoloM
Copy link
Author

ApoloM commented Nov 16, 2023

There is the code flipping the tag pose with Reg/Force3DoF=true:

if(_registrationPipeline->force3DoF())
{
	// For 2D slam, make sure the landmark z axis is up
	rtabmap::Transform tx = landmarkPose.rotation() * rtabmap::Transform(1,0,0,0,0,0);
	rtabmap::Transform ty = landmarkPose.rotation() * rtabmap::Transform(0,1,0,0,0,0);
	if(fabs(tx.z()) > 0.9)
	{
		landmarkPose*=rtabmap::Transform(0,0,0,0,(tx.z()>0?1:-1)*M_PI/2,0);
	}
	else if(fabs(ty.z()) > 0.9)
	{
		landmarkPose*=rtabmap::Transform(0,0,0,(ty.z()>0?-1:1)*M_PI/2,0,0);
	}
}

Well, this works well if we don't change Reg/Force3DoF over time (from 3DoF to 6doF or vice-versa). I'll check to make it work when changing the parameter.

Another note, could you also show the covairance set in your odometry topic (assuming it is called /odom)?

rostopic echo /odom

I saw some GTSAM indeterminant error while testing on the simulator and it was caused by bad covariance.

Here is the odometry data at the instant in which the fatal error crashes the system:

---
header: 
  seq: 70391
  stamp: 
    secs: 1698255109
    nsecs: 269333124
  frame_id: "odom"
child_frame_id: "base_link"
pose: 
  pose: 
    position: 
      x: -197.40680530411618
      y: 29.594699332749887
      z: 0.0
    orientation: 
      x: 0.0
      y: 0.0
      z: -0.9998580829759275
      w: 0.01684677736848209
  covariance: [0.2457867145028996, 0.007579704226761971, 0.0, 0.0, 0.0, 0.0, 0.007579704226762059, 0.024866542874265603, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 5.7500875447710605e-09, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 5.750072786080242e-09, 6.512993045552662e-47, 0.0, 0.0, 0.0, 0.0, -6.512993045552662e-47, 5.750072786080242e-09, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 8.166713637045317e-05]
twist: 
  twist: 
    linear: 
      x: 0.0
      y: 0.0
      z: 0.0
    angular: 
      x: 0.0
      y: 0.0
      z: 3.259415738863996e-27
  covariance: [0.006402685183589715, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0006488214186150694, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.2140061976883634e-06, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.2140058844170501e-06, -7.278402106173485e-51, 0.0, 0.0, 0.0, 0.0, 7.278402177381913e-51, 1.2140058844170501e-06, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.004041672752928651]

EDIT:
Theoretically, I'm not using the topic of odometry in rtabmap_ros, because I'm setting the odom_frame_id parameter.
Will it use the covariance of the topic even in that case?

matlabbe added a commit to introlab/rtabmap that referenced this issue Nov 17, 2023
@matlabbe
Copy link
Member

matlabbe commented Nov 17, 2023

The odometry topic covariance looks fine, but if you are using odom_frame_id, those parameters are used:

~odom_tf_linear_variance (double, default: 0.001)
When odom_frame_id is used, the first 3 values of the diagonal of the 6x6 covariance matrix are set to this value.

~odom_tf_angular_variance (double, default: 0.001)
When odom_frame_id is used, the last 3 values of the diagonal of the 6x6 covariance matrix are set to this value.

which should not cause that issue with those default values.

In the commit above, I updated the code to remove the last argument of getConnectedGraph() so this would fix the current issue. I won't "fix" the problem of switching Reg/Force3DoF from true to false or vice-versa when the map contains landmarks, as the changes would be quite deep to do and I think the current code is more efficient in its current state.

@ApoloM
Copy link
Author

ApoloM commented Nov 17, 2023

Hi, I have some updates again.

I made the tests with the commit above and finally the issue was solved.
The system is working perfectly and everything is great.

Thanks for the help and explanations about this awesome package/library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants