Keypoints doesn't align with its original point cloud

[bin70]

Hello, jiaxin:

I run the code successfully and visualize the detected keypoints together with its original point cloud, but they are not aligned with each other, as shown below:

(seq02-002121, train on oxford & test on kitti)

(seq02-002121, train on kitti & test on kitti)

I completely use the default parameter settings in original codes and the model weights that you provide in README (trained on oxford) and then evalute on kitti dataset. After getting the unaligned keypoints, I also use the model weights I personally train on kitti dataset.

In order to run the code successfully, I only change some default filepaths in several files:

1. save_key_points.py

# ============= dataset ===============
dataset_type = 'kitti'
test_txt_folder = '/home/elvin/kitti-reg-test'
numpy_folder = '/home/elvin/kitti/data_odometry_velodyne/numpy'
output_folder = '/home/elvin/keypoints_knn_ball/oxford_kitti'
# output_folder = '/home/elvin/keypoints_knn_ball/kitti_kitti'
# =============== method ================
method = 'tsf'
detector_model_path = '/home/elvin/models/oxford/16384-512-k1k16-2d/best.pth'
# detector_model_path = '/home/elvin/models/kitti/gpu1_185_-1.559669_net_detector.pth' # train on kitti

2. build_filepath.m

elseif strcmp(dataset, 'kitti')
    keypoint_folder = '/home/elvin/keypoints_knn_ball/oxford_kitti/tsf_GHZU34IR';
    % keypoint_folder = '/home/elvin/keypoints_knn_ball/kitti_kitti/tsf_RBO0SDQY';
    pc_folder = '/home/elvin/kitti/data_odometry_velodyne/numpy';
    % pc_folder = '/home/elvin/kitti-reg-test';

3. load_keypoint.m

elseif strcmp(dataset, 'kitti')
    seq_str = filepath(end-12:end-11);
    calib = read_kitti_calib(['/home/elvin/kitti/calib/', seq_str, '/calib.txt']);

Then I run save_key_points.py followed by visualize_features.m, the output is

kitti - i916 - n39 - /home/elvin/kitti/data_odometry_velodyne/numpy/02/np_0.20_20480_r90_sn/002121.npy, /home/elvin/keypoints_knn_ball/oxford_kitti/tsf_GHZU34IR/02/002121.bin
...

From the file name, the two seem to match each other, but the results of the visualization are not correct. Is there any error in the above settings, which leads me to get the wrong result? Could you give me any help or tips?

[lijx10]

Maybe you may want to check whether the points and the keypoints are at the same coordinate system, e.g. NWU or camera coordinate?

[bin70]

Thanks for your promptly reply. Now I can get the correct visualization results.

[XuyangBai]

@bin70 Hi, Could you please share your solution? I also find that my keypoints doesn't align with original point cloud.

[bin70]

@bin70 Hi, Could you please share your solution? I also find that my keypoints doesn't align with original point cloud.

Sorry for getting back to you late. You just need to read per 3 columns from the keypoints files instead of per 4 columns in the original code.

[XuyangBai]

@bin70 Sorry I am still confused. I think the 4th column of keypoints file is sigma, so why it may affect the location of keypoints ?

[bin70]

@XuyangBai The sigma has not been saved to keypoints files.