Looping "IMU is not or recently initialized. Reseting active map... "

[qliao001]

Hi many thanks for the great contribution! I have been working with monocular and multi-session version and all went well. Now I try to introduce the corresponding IMU sequence, but the system kept looping "IMU is not or recently initialized. Reseting active map... ". Could you please give some advice how can I tackle the problem? or is there any specific trick of collecting data for better IMU initialization?

Here is a screenshot, you can see that the system basically kept trying to initialize the IMU:

IMU in Map 0 is initialized
PIOLKF: NOT ENOUGH EDGES
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
SYSTEM-> Reseting active map in monocular case

Thanks a lot!

[ShilangChen1011]

I have the same problem. Who could explain it?

[ShilangChen1011]

The reason may be that ORB-SLAM3 only supports Global Shutter cameras.

[ShilangChen1011]

What is the camera type？

[qliao001]

What is the camera type？

It is an Android phone embedded camera and IMU. The streaming and IMU measurements are sent by client (cell phone) and processed in the server (laptop). But I don't think it is the problem of camera, since I could run the Monocular version, but not the Monocular-Inertial version.

[npcdna]

yes,i meet same question. I use camera-imu parameters which calibrated by kalibr, these parameters can work well in vins, but always reset in orbslam3. when i change imu noise parameters, orbslam3 always arise segment error.

[mccoychang]

I also encountered this problem(these parameters can work well in vins), is there a solution?

[ShreyasManjunath]

I find similar error like -

New Map created with 285 points
Point distribution in KeyFrame: left-> 285 --- right-> 0
build optimization graph
start optimization
end optimization
update Keyframes velocities and biases
IMU in Map 2 is initialized
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
SYSTEM-> Reseting active map in monocular case
LM: Active map reset recieved
LM: Active map reset, waiting...
FAIL LOCAL-INERTIAL BA!!!!
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!

This loops continously. Any one found solution?

[Ehsanmb]

I have the same issue! looking for solution

[MiaoMiaoMeng]

What is the camera type？

It is an Android phone embedded camera and IMU. The streaming and IMU measurements are sent by client (cell phone) and processed in the server (laptop). But I don't think it is the problem of camera, since I could run the Monocular version, but not the Monocular-Inertial version.

Hi, @qliao001,
I met the same question, did you figure it out?
I did a lot of work on camera-IMU calibrate, but it doesn't work.
So I try to find the local-Inertial BA to change the threshold, but I didn't finish it yet.
Do you have some idea?

[MiaoMiaoMeng]

Hi guys, @Ehsanmb @ShreyasManjunath @mccoychang

Today I find out how to fix this problem.
If you use those APP in Android (CameraImu & MarsLogger) or other application pls make sure the coordinate is same.
For me I use MarsLogger to catch info form Cam and IMU and creat bag for calibrate, and then use CameraImu to do the real time orbslam.
The reason why my local-Inertial BA function was failed is that those IMU‘s coordinate is totally different.

So in this situation, you can rebuild your application to let them have a unified coordinate system.

[pinkfloydfan]

I've been encountering this, and believe it or not - I've 'mitigated' it by changing the noise/random walk values in the yaml file to ridiculously small numbers, like 10e-8 small. I have no idea why this works.

[iamchoking]

I have tried the suggestion of both @MiaoMiaoMeng and @pinkfloydfan (along with everything else on the web). But neither seem to work. I am currently using:

ZED2 (@15fps publish rate, 1280x720 resolution)
ROS melodic, real-time
opencv-3-2-0

topics: (published with ZED ros wrapper [zed_wrapper])
/zed2/zed_node/imu/data_raw
/zed2/zed_node/left/image_rect_color
/zed2/zed_node/left/image_rect_color

with imu settings calibrated from kalibr. I hope a solution to this is found soon!

*This is my first post ever on any Issue ever (i am this determined for an answer!). So if you need more info, please tell me

[pinkfloydfan]

I have tried the suggestion of both @MiaoMiaoMeng and @pinkfloydfan (along with everything else on the web). But neither seem to work. I am currently using:

ZED2 (@15fps publish rate, 1280x720 resolution)
ROS melodic, real-time
opencv-3-2-0

topics: (published with ZED ros wrapper [zed_wrapper])
/zed2/zed_node/imu/data_raw
/zed2/zed_node/left/image_rect_color
/zed2/zed_node/left/image_rect_color

with imu settings calibrated from kalibr. I hope a solution to this is found soon!

*This is my first post ever on any Issue ever (i am this determined for an answer!). So if you need more info, please tell me

@iamchoking sanity check: are the image frames from the camera timestamped? I was able to get ORB SLAM 3 working on an ESP32-CAM at about 12FPS, and an IMU from a drone flight controller. My workaround was setting the variances to extremely low values. DM me your configuration yaml file and I can have a look.

[iamchoking]

@pinkfloydfan
Sure! I will post it here:

%YAML:1.0

#--------------------------------------------------------------------------------------------
# Camera Parameters ZED2 camera: HD720 mode, grab rate 15 frames per second
#--------------------------------------------------------------------------------------------
Camera.type: "PinHole"

# Camera calibration and distortion parameters (OpenCV) 
Camera.fx: 527.44
Camera.fy: 527.405
Camera.cx: 630.2
Camera.cy: 362.877

Camera.k1: -0.0416274
Camera.k2: 0.0112748
Camera.p1: -0.000148709
Camera.p2: 0.000268278

Camera.bFishEye: 0

Camera.width: 1280
Camera.height: 720

# Camera frames per second 
Camera.fps: 15

# stereo baseline times fx
Camera.bf: 63.16937904

# Color order of the images (0: BGR, 1: RGB. It is ignored if images are grayscale)
Camera.RGB: 1

# Close/Far threshold. Baseline times.
ThDepth: 35.0

# Transformation from camera 0 to body-frame (imu)

#found in zed_wrapper launch
#[ INFO] [1625636907.865509411]: Camera-IMU Transform: 
# 55C0C3837CD0
#0.999998 0.001782 -0.000476 -0.002000
#-0.001782 0.999998 0.000218 -0.023000
#0.000476 -0.000217 1.000000 -0.002000
#0.000000 0.000000 0.000000 1.000000

# Initial Matrix:

#   data: [ 0.999998,  0.001782, -0.000476, -0.002000,
#          -0.001782,  0.999998,  0.000218, -0.023000,
#           0.000476, -0.000217,  1.000000, -0.002000,
#           0.000000,  0.000000,  0.000000,  1.000000 ]

# BUT imu data is wrong!!!, so it is corrected by post multiplication with

#  0  0  1  0
#  -1 0  0  0
#  0 -1  0  0
#  0  0  0  1

Tbc: !!opencv-matrix
   rows: 4
   cols: 4
   dt: f
   data: [0.00725839, -0.99997021,  0.00262767,  0.02413368,
          0.00243735, -0.00261004, -0.99999362,  0.00210845,
          0.99997069,  0.00726474,  0.00241834, -0.02232050,
          0.00000000,  0.00000000,  0.00000000,  1.00000000 ]


#   data: [-1.000000, -0.001782,  0.000476, -0.002000,
#           0.001782, -1.000000, -0.000218, -0.023000,
#          -0.000476,  0.000217, -1.000000, -0.002000,
#           0.000000,  0.000000,  0.000000,  1.000000 ]
#   data: [ 1.000000,  0.001782, -0.000476, -0.002000,
#          -0.001782,  1.000000,  0.000218, -0.023000,
#           0.000476, -0.000217,  1.000000, -0.002000,
#           0.000000,  0.000000,  0.000000,  1.000000 ]



# IMU noise
#IMU.NoiseGyro: 8.6e-05 # 1.6968e-04 
#IMU.NoiseAcc: 1.4e-03 # 2.0000e-3
#IMU.GyroWalk: 2.2e-06 
#IMU.AccWalk: 8.0e-05 # 3.0000e-3
# welp I tried everything at this point
IMU.NoiseGyro: 10e-8  # 1.6968e-04 
IMU.NoiseAcc:  10e-8  # 2.0000e-3
IMU.GyroWalk:  10e-8 
IMU.AccWalk:   10e-8  # 3.0000e-3


IMU.Frequency: 400

#--------------------------------------------------------------------------------------------
# Stereo Rectification. Only if you need to pre-rectify the images.
# Camera.fx, .fy, etc must be the same as in LEFT.P
#--------------------------------------------------------------------------------------------
LEFT.height: 720
LEFT.width: 1280
LEFT.D: !!opencv-matrix
   rows: 1
   cols: 5
   dt: d
   data:[0.0, 0.0, 0.0, 0.0, 0.0]
LEFT.K: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [524.4577026367188, 0.0, 635.6156005859375, 0.0, 524.4577026367188, 365.53564453125, 0.0, 0.0, 1.0]
LEFT.R:  !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
LEFT.P:  !!opencv-matrix
   rows: 3
   cols: 4
   dt: d
   data: [524.4577026367188, 0.0, 635.6156005859375, -62.812259674072266, 0.0, 524.4577026367188, 365.53564453125, 0.0, 0.0, 0.0, 1.0, 0.0]

RIGHT.height: 720
RIGHT.width: 1280
RIGHT.D: !!opencv-matrix
   rows: 1
   cols: 5
   dt: d
   data:[0.0, 0.0, 0.0, 0.0, 0.0]
RIGHT.K: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [524.4577026367188, 0.0, 635.6156005859375, 0.0, 524.4577026367188, 365.53564453125, 0.0, 0.0, 1.0]
RIGHT.R:  !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
RIGHT.P:  !!opencv-matrix
   rows: 3
   cols: 4
   dt: d
   data: [524.4577026367188, 0.0, 635.6156005859375, -62.812259674072266, 0.0, 524.4577026367188, 365.53564453125, 0.0, 0.0, 0.0, 1.0, 0.0]

#--------------------------------------------------------------------------------------------
# ORB Parameters
#--------------------------------------------------------------------------------------------

# ORB Extractor: Number of features per image
ORBextractor.nFeatures: 1200

# ORB Extractor: Scale factor between levels in the scale pyramid 	
ORBextractor.scaleFactor: 1.2

# ORB Extractor: Number of levels in the scale pyramid	
ORBextractor.nLevels: 8

# ORB Extractor: Fast threshold
# Image is divided in a grid. At each cell FAST are extracted imposing a minimum response.
# Firstly we impose iniThFAST. If no corners are detected we impose a lower value minThFAST
# You can lower these values if your images have low contrast			
ORBextractor.iniThFAST: 20
ORBextractor.minThFAST: 7

#--------------------------------------------------------------------------------------------
# Viewer Parameters
#--------------------------------------------------------------------------------------------
Viewer.KeyFrameSize: 0.05
Viewer.KeyFrameLineWidth: 1
Viewer.GraphLineWidth: 0.9
Viewer.PointSize:2
Viewer.CameraSize: 0.08
Viewer.CameraLineWidth: 3
Viewer.ViewpointX: 0
Viewer.ViewpointY: -0.7
Viewer.ViewpointZ: -1.8
Viewer.ViewpointF: 500

The commented lines are the varaibles I have tried with and failed.
Thanks in advance!

[bboysil]

Hi, I've been struggling with this error for a fewdays now. Did anyone find a solution?
I was providing camera stream and IMU data from an Android, then from an iPhone.
I set the IMU gyro and acc axis to the "correct" direction by playing back rosbags with recordings that work and watched carefully when the device rolled, pitched, yawed, how their IMU responded.
I also tried setting the IMU walk and noise to higher and lower values, same error.
Any thoughts?

I'm guessing I would really need to "calibrate" the IMU together with the camera. Can something like this help: https://github.com/ethz-asl/kalibr/wiki/camera-imu-calibration
One problem is I can only send IMU data at about 60 Hz. (I tried sending it at higher rates but I would duplicate/interpolate the data).

[Abner0907]

Hi guys, @Ehsanmb @ShreyasManjunath @mccoychang

Today I find out how to fix this problem. If you use those APP in Android (CameraImu & MarsLogger) or other application pls make sure the coordinate is same. For me I use MarsLogger to catch info form Cam and IMU and creat bag for calibrate, and then use CameraImu to do the real time orbslam. The reason why my local-Inertial BA function was failed is that those IMU‘s coordinate is totally different.

So in this situation, you can rebuild your application to let them have a unified coordinate system.

Hello @MiaoMiaoMeng can you share your configure file about the camera?

I have the same issue as yours, It can not detected the orb features when use imu. work good just with Mono mode. Many thanks.

[maghauke]

@qliao001 I believe the following can be causing this error:

• You have to calibrate your imu and camera and have an accurate transformation between camera and IMU
• Do you have enough motion in all directions, take a look at using ORBSLAM3 with e.g. EuRoC dataset: MH_01_easy.bag, seems like motion initially is important for the initialization of ORBSLAM3 to work
• also the IMU noise must be tuned for best performance, not sure how to find the best parameters here, but maybe that can be done in the calibration?

[Fisine]

我也遇到了同样的问题，我多次标定，并且给了足够的运动，但还是循环失败，我找不出问题所在，有人解决这个问题吗

[zyfff66]

I'm also having this issue, trying to recalibrate the IMU using allan_variance_ros mentioned in A summary of mono-inertial initialization issues, and suggestions for others and then adjust the IMU noise parameters:

# IMU noise (Use those from VINS-mono)
IMU.NoiseGyro: 1e-3 # rad/s^0.5
IMU.NoiseAcc: 1e-2   # m/s^1.5
IMU.GyroWalk: 1e-7 # rad/s^1.5
IMU.AccWalk: 1e-7 # m/s^2.5
IMU.Frequency: 200

Then things got better.