Configuration and IMU init

[vswdigitall]

Hi,

I want to try your project with Arducam 1mp stereo OV9281, ROS Melodic, Nvidia Jetson NX JP4.4, mpu9250.

I have IMU noise config and cameras2imu config.
Tell me please about camera.txt format.
Intrinsics ok, but other digits what mean?

And i should use mode1?

Best regards, Viktor.

[lukasvst]

You can find some documentation on the camera.txt here: https://github.com/JakobEngel/dso#geometric-calibration-file

The first line contains first the name of you distortion model (e.g. EquiDistant, RadTan, Pinhole for no distortion, etc.), then 4 numbers for focal length an principal point (fx fy cx cy), and then the distortion parameters of your specific distortion model.
E.g. for Equidistant these will be k1 k2 r1 r2

For the third line you can just use crop and it will find a good output focal length and principal point. Note that for fisheye cameras you might want to manually set a smaller focal length instead, as using the full image can be sub-optimal when lengths in the rectified image become very different from undistorting. Your camera seems to have a smaller FOV, so just using crop should be fine.

I recommend starting with mode=1, this means you don't have photometric calibration (vignette, response, exposures).
For best results you should calibrate vignette and response and provide exposure times and use mode=0.
If your sensor doesn't output exposure times but you have calibrated your vignette you should use mode=3

[vswdigitall]

Thank you. Will try.

[vswdigitall]

It is very first result. All software installed without problem. Only changed topics in cpp.
Looks like amazing approach :)
video

Tell me please, it uses stereo or mono only?
And what you can say about random trajectories at the end of video?

[lukasvst]

The system is mono-inertial only.

It looks like the system failed while still being in visual-only mode (before IMU initialization completed). Can you confirm it can get to visual-inertial mode successfully (e.g. by accelerating it a bit sideways so that the scale becomes observable?

If you have failure cases it's always best to paste the full console output here, then I can have a look. In general, the system will be less robust before IMU initialization is finished, as only the camera can be used for tracking during this period.

[vswdigitall]

Here is some results.
Yes it runs visual mode only. Why?

[lukasvst]

The CoarseIMUInit normalized error should be smaller than 1 usually and the estimated scale is also extremely large, suggesting there's something wrong with the IMU data / its calibration.
This could be e.g.

• IMU-camera calibration: Have you passed the correct camchain.yaml?
• IMU-camera time synchronization. Is the IMU well synchronized with the camera?
• IMU noise values: Maybe your IMU noise values are too small, I suggest trying the provided t265_noise_tumvi.yaml first.

[vswdigitall]

Here is all data. Please see it.
dm_conf.zip

[lukasvst]

Thanks for sending the output! For the future it would be best to also include the full commandline output (especially the beginning), as some things are only printed there.

• One thing I have noticed in your config: You should write RadTan instead of Pinhole in your camera.txt file. Pinhole in this context means pre-rectified images without any distortion, which is not the case here.
• The IMU noise values (accelerometer + gyroscope noise density and random walk) seem to be pretty low. The optimal values for DM-VIO are typically orders of magnitude larger than the ones you pass to Kalibr. As mentioned, I recommend starting with the same noisevalues as in tumvi.yaml, and only reducing them after it works.
• Your Kalibr calibration mentions a timeshift_cam_imu, have you modified the timestamps accordingly before passing them to DM-VIO?

I think especially the last point is the one why it fails, but the other two will also be important to get a good performance.

[vswdigitall]

Hi Lukas, thanks for help. Will fix RadTan.

Here is cmd;

What preset i should use?

I started recalibrating process imu to camera now. It is still in process.

I recorded video 40 Hz, and IMU 40Hz. May be here is some issue too?

[lukasvst]

You should always use preset=1 with the live version.

40Hz IMU is a pretty low rate for IMU measurements indeed (200Hz is more usual). I think IMU initialization should still be able to complete however if everything is setup correctly.