Question about Evaluation

[AreteQin]

Hello there,

I met a question while I was evaluating my result.

I ran the test datasets from this website "https://www.eth3d.net/slam_datasets" and got my results.

Then I used "evaluate_ate.py" to evaluate my result. I added the addresses of my result file and imu file in the command. However, the ATE RMSE is about 0.98 m which is obviously incorrect.

I have two questions:

1. There are 4 files named imu.txt in each dataset, what is the difference?

2. I found an option named offset in "evaluate_ate.py", but I do not understand how to use it, I think probably this is what I should use to correct my evaluation.

Sorry for bothering you if there is some relative information I missed.

Thanks for your time, I am looking forward to your response.

Best regards,
Arete

[puzzlepaint]

1. See https://www.eth3d.net/slam_documentation#format-of-imu-data . There are different cameras on the camera rig, and each camera contains an IMU. Each file contains the data from one IMU.
2. Which evaluation script do you use, the one from the TUM-RGBD dataset? The 'official' evaluation program would be this one: https://github.com/ETH3D/slam-evaluation . Also, I don't think that the imu.txt file is supposed to be used here, since this (probably) should be the files containing the trajectories, not the IMU raw data.

[AreteQin]

Thanks a lot for your response.

I can find the ground truth files for training datasets and synthetic datasets. But I cannot find any ground_truth.txt in the test datasets. "https://www.eth3d.net/slam_datasets"

Sorry for bothering you if there is some relative information I missed.

Thanks for your time, I am looking forward to your response.

[puzzlepaint]

That is actually the whole point of test datasets: Their ground truth is private and is only used for evaluation of results that are submitted to eth3d.net. This way, a finished SLAM system can be tested on these datasets while ensuring that no specific parameter tuning etc. has been done on those datasets that would artificially improve its results.

[AreteQin]

Thank you so much for your response, I understood.

I will close this issue.