Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

World frame coordinate changes mid operation when IMU is involved #172

Open
thien94 opened this issue Oct 30, 2020 · 13 comments
Open

World frame coordinate changes mid operation when IMU is involved #172

thien94 opened this issue Oct 30, 2020 · 13 comments
Assignees

Comments

@thien94
Copy link

thien94 commented Oct 30, 2020

Thank you very much for making the work public.

I encounter the following issue when using the package, hope someone can help.

In Mono-Inertial or Stereo-Inertial mode, the world frame changes mid-operation (it seems to be right after InertialOptimization is completed according to the console output) as can be seen in the example below. The world frame visualization is done by adding pangolin::glDrawAxis(5.0); to Viewer.cc.
ORB-SLAM3-Wframe-jump

This problem is evident when I print out the Tcw value returned by the TrackMonocular function:

Translation xyz: -0.0268984828 0.147062019 0.0332303531   <--- Before InertialOptimization 
Translation xyz: -0.0243855752 0.138408974 0.033842843
build optimization graph
start optimization
end optimization
update Keyframes velocities and biases
IMU in Map 0 is initialized
Translation xyz: 0.604404151 -1.86084425 -2.65787292      <--- After InertialOptimization
Translation xyz: 0.59955281 -1.87405574 -2.67359495
...

Translation xyz: 0.418520272 -0.799674332 -3.30177617
Translation xyz: 0.412499785 -0.793865025 -3.30544758     <--- Before InertialOptimization 
start VIBA 1
build optimization graph
start optimization
end optimization
update Keyframes velocities and biases
Translation xyz: 0.407404989 -0.789338589 -3.30969024
end VIBA 1
Translation xyz: -0.0678119957 -0.580297947 -3.23924804   <--- After InertialOptimization
Translation xyz: -0.127183706 -0.560011804 -3.25348377

The issue is also somewhat unpredictable and differs from run to run. However, I don't see such behavior with Mono or Stereo mode.

I would like to pose two questions:

  1. My guess is that the first world coordinate alignment corresponds to the IMU initialization (z and scale alignment), but I can't figure out why are the subsequent alignments happening?
  2. Is the returned Tcw value (from TrackMonocular, TrackStereo, and TrackRGBD) recommended to be consumed by other systems that require odometry from ORB-SLAM3? If it is, the inconsistency in the coordinate frame seems undesirable. If it is not, what would be the parameter that can be used?

Thank you greatly for your time.

@richard-elvira
Copy link
Member

When the map is initialized the world reference is set in the first keyframe. Later, IMU is initializated and the world reference is changed to aligned to it, Z axis, previously aligned with the view of the first keyframe, is aligned with respect to the gravity vector.

@thien94
Copy link
Author

thien94 commented Nov 10, 2020

Hi @richard-elvira, thank you very much for the information.

I can understand that the world reference frame is realigned to IMU's z axis when the initialization completes, but why does the origin jump to a different location that seems to be totally random (not linked to any keyframes obtained up to that point and not consistent between runs)?

@martinakos
Copy link

Maybe related to my issue #164

@thien94
Copy link
Author

thien94 commented Nov 18, 2020

Hi @martinakos , thank you for the info. I can confirm that the error "NOT INERTIAL LINK TO PREVIOUS FRAME!!!!" also appears in my tests, so it's possible that the two issues are one and the same.

@1216621137
Copy link

So, if I want get the real-time estimated pose, what would be the parameter that can be used? Or, it is just impossible to get the right pose in real time.

@dimaxano
Copy link

dimaxano commented May 4, 2021

Hi, @thien94
I am reading the ORBSLAM3 paper and found this detail (on the screenshot) about the initialization scheme in ORBSLAM3.
Look like a possible answer to your question
init_BA

@thien94
Copy link
Author

thien94 commented May 5, 2021

@dimaxano Thank you very much for the info. I didn't catch that when reading the paper.

@18851409110
Copy link

I have downloaded the code and tried to run with a mono camera but without imu information.There are some problems such as running speed is a little slow. So have you ever tried running with a real camera but not read the dataset. Maybe the dataset is the point. And have you ever tried other dataset?

BTW, is it convenient to add your wechat or other social account?

@mateosss
Copy link

mateosss commented Oct 2, 2021

but why does the origin jump to a different location that seems to be totally random (not linked to any keyframes obtained up to that point and not consistent between runs)?

Did you get an answer for this?

@AyaRabea
Copy link

AyaRabea commented Nov 1, 2021

I have downloaded the code and tried to run with a mono camera but without imu information.There are some problems such as running speed is a little slow. So have you ever tried running with a real camera but not read the dataset. Maybe the dataset is the point. And have you ever tried other dataset?

BTW, is it convenient to add your wechat or other social account?

I have the same problem did you find any solution?

@dantenoguera
Copy link

Hi, @thien94 I am reading the ORBSLAM3 paper and found this detail (on the screenshot) about the initialization scheme in ORBSLAM3. Look like a possible answer to your question init_BA

Didn't get what this implies. Was someone able find an explanation for the jump when IMU is involved?

@kaitaotang
Copy link

Maybe, the scale changed when the second and third initialization step. Therefore, it caused the pose of world coordinate changed as showed in your gif.

@Yeager-101
Copy link

Yeager-101 commented Jun 13, 2023

I met the same problem, can anyone help me?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests