odometry is increasing continuously such as drift is not being corrected.

[NicoNorena]

added support to ready stereo feed, and IMU , I am using the Zed mini from stereoLabs.

I added the corresponding offline calibration parameters and noise parameter of IMU and tried modifying the "_sigma_px" parameters. But it seems not to help much

After it starts the launch file, the inertial frame moves towards infinity specially on the z axis.
Do you have any idea why this is happening?
I was wondering whether if there is something on the KTL that is causing this? also I ran it with the KTL set to false (discrete mode), and it broke with an exception.

any ideas?

[goldbattle]

You probably need to adjust the imu detection threshold init_imu_thresh.
I would set this to a large value, then record a bag of you picking it up, and then run the system.
From there you should be able to determine a good threshold (based on the printout) that determines when the system should initialize after being picked up.

[NicoNorena]

the imu detection threshold init_imu_thresh looks to determine when to start ,and I just ran a test setting to 5 which i will have to move a little harder. But this does not solve the issue.

are the parameters of up_msckf_sigma_px and up_slam_sigma_px in terms of pixels? I used fraction assuming that it was normalized,but it did not help. When I use integers like 12, I can see some MSCKF features being updated.

I would expect to see the print message "MSCKF update (0 feats) and SLAM update (0 feats )" higher than zero.

any other ideas? how about the tracker/extractor properties? fast_threshold or the min_px_dist

[goldbattle]

Your camera distortion looks huge. They shouldn't be over 0.5 normally. If your distortion is fisheye, then make sure you enable that.

…

On Thu, Apr 16, 2020, 6:05 PM NicoNorena ***@***.***> wrote: the imu detection threshold init_imu_thresh looks to determine when to start ,and I just ran a test setting to 5 which i will have to move a little harder. But this does not solve the issue. are the parameters of up_msckf_sigma_px and up_slam_sigma_px in terms of pixels? I used fraction assuming that it was normalized,but it did not help. When I use integers like 12, I can see some MSCKF features being updated. [image: Screenshot from 2020-04-16 17-59-24] <https://user-images.githubusercontent.com/37921997/79510863-2c55ba00-800c-11ea-844b-ee67bf885c8f.png> [image: Screenshot from 2020-04-16 18-00-14] <https://user-images.githubusercontent.com/37921997/79510877-31b30480-800c-11ea-968d-b661f3fbec97.png> I would expect to see the print message "MSCKF update (0 feats) and SLAM update (0 feats )" higher than zero. any other ideas? how about the tracker/extractor properties? fast_threshold or the min_px_dist — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#52 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAQ6TYSYHIV77777SOYZ3B3RM56MPANCNFSM4MKEWGHQ> .

[NicoNorena]

I re-calibrated the cameras, because I used the incorrect distortion model. Now, I used the model "pinhole-radtan".
Though, it is doing better after using the correct calibration and playing with the sigma parameters and chi2_multiplier.
starts ..

The parameters I used.

I am wondering if to continue to mess with the parameter , the IMU noise on green are the calculated one. What do you think?

thank you for your help.

[NicoNorena]

I increased the IMU parameters by x80 from the calculated I got from Kalibr_alla. Also, I've set the sigma_px parameters back to one. This helped alot.

[goldbattle]

The imu noise in that image seems to be pretty large. I would try using the eth noise parameters which we have found normally works good on most sensors. The pixel error normally shouldn't go over one if you have good calibration (i.e. 1 pixel reprojection error on the raw image). You can always test and try to use the values that the system calibrates to if you have poor initial.

Also not sure how "fisheye" the ZEDs images are, but if it has some pretty bad distortion I would recommend using equidistant / fisheye. I assume you are following this calibration guide?

edit: The commented noises look on a good order based on my experience. Your random walks shouldn't normally be large at all. If you want to inflate something inflate the noise density terms. The gyroscope random walk seems high in both the commended and very very high in what you are using.

[NicoNorena]

I tested and the fisheye option did not help much. I looked at the documentation and the ZED stereo cameras does not use a fisheye len.

I did noticed that the initial calibration i used was incorrect. It is working much better now.
I inflated the noise density terms only and keep the random walk values small as used for calibration. Also added some more features for tacking and increased the knn ratio. how does the grid side as 3 helps would impact ? I am assuming the size of the input image would dictate this value.
At last, I added the FEJ at initialization as well.

as you see in the picture movement side to side, but in reality I was moving the camera up and down. Do you have any idea of why would the estimation be coming out on a different axis?

[goldbattle]

If you are able to upload a bag, launch file, and your kalibr calibration result.txt I can take a look at it. From what you posted, your transformation between imu and camera might be wrong.
The imu noises still are way too high to be reasonable. Additionally, this might come down to the camera being a rolling shutter and us not supporting that. You can try to move slowly to reduce the rolling shutter effect or increasing the pixel noise.

[NicoNorena]

thanks, I didn't think to look whether the zed used a rolling shutter, effectively they do.
https://cdn.stereolabs.com/assets/datasheets/zed-mini-camera-datasheet.pdf
I will be moving to using a different stereo camera.

Before moving away from using this stereo camera, I ran a quick experiment.
I changed parameters back to ones I calculated with kalibr: 1) increasing the px noise and move slowly, and 2) changed the imu noise values.
I found an article with the calibration info for the zed camera, my values are similar to the ones here except for the IMU being smaller. The IMU to camera transform's values are some what close to the ones on the article.
https://support.stereolabs.com/hc/en-us/articles/360012749113-How-can-I-use-Kalibr-with-the-ZED-Mini-camera-in-ROS-

as you mentioned, it behaved much stable as long as I increased the px noise to at least 10. I was seeing the behavior of the estimation moving up when moving side to side. The motion forward and back was consistent with a lot of resistant when moving back, I moved really slowed. would this behavior occur from not modeling the rolling shutter?

I am convince that a combination of the rolling shutter behavior and increasing the px noise causes the weird behavior.

[goldbattle]

Did you calibrate using Kalibr or tried to get the extrinsics using the ZED driver?
From looking online, the way they handle their coordinate frames in TF might not be standard.
If you have your Kalibr result.txt if you could post that, I would appreciate.

[NicoNorena]

yes I used Kalibr, I didn't used the extrinsic that the ZED wrapper publishes.
imuCam-calib-ZedM.txt

I thought the same thing whether the TF was different, I looked in the code to see whether the follow a different order of the x, y, and z axis . I assumed that the vector was ordered as x,y,z .

[goldbattle]

The transformations you are using are incorrect (if the screenshot is what you are using).
You should use the cam to imu transformations:

T_ic:  (cam0 to imu0): 
[[-0.01080233  0.00183858  0.99993996  0.01220425]
 [-0.99993288 -0.00420947 -0.01079452  0.0146056 ]
 [ 0.00418937 -0.99998945  0.00188393 -0.00113692]
 [ 0.          0.          0.          1.        ]]
T_ic:  (cam1 to imu0): 
[[-0.01043535 -0.00191061  0.99994372  0.01190459]
 [-0.99993668 -0.00419281 -0.01044329 -0.04732387]
 [ 0.00421252 -0.99998938 -0.00186674 -0.00098799]
 [ 0.          0.          0.          1.        ]]

[NicoNorena]

yea, you are correct. I don't know how did missed to used the incorrect transform. I was grabbing it directly from the terminal. Thanks again for all your help.
I ran the test again with correct transform and lower the pix noise to 5 pix, accounting for the rolling shutter behavior. It still looks very good by simply observing it on RVIZ.

[Hyjale]

@NicoNorena Do you mind posting your launch file? I have the same ZED Mini camera and I am trying to run the same tests.

[NicoNorena]

I cant post the whole launch file but you want use all the data above to set the parameters. the calibration on every camera is different, so you will need to calibrate your zed camera offline. if you read slowly above the use can use the calibration file I attached to set your parameters. Since the Zed uses a rolling shutter you have to increase the pix noise as workout, not the best solution.