Request for complete pipeline for ground truth data generation

[patelajaychh]

Hi,
Thankyou for your responses to #1 and #2 .
Based on the information provided, I tried to create SFM model and consecutively poses for reference as well as query images. The superpoint and superglue is used for feature extractor and feature matching with exhaustive feature matching.

When I evaluated PixLoc - Localization network, on produced data , the performance is worst (result below).

Cambridge ShopFacade Scene -

[12/17/2021 20:35:54 pixloc INFO] Evaluate scene ShopFacade_ours: /home/ajay/pixloc/outputs/results/pixloc_Cambridge_ShopFacade_ours.txt
[12/17/2021 20:35:54 pixloc.utils.eval INFO]
Median errors: 5.241m, 65.716deg
Percentage of test images localized within:
        1cm, 1deg : 0.00%
        2cm, 2deg : 0.00%
        3cm, 3deg : 0.00%
        5cm, 5deg : 0.00%
        25cm, 2deg : 0.00%
        50cm, 5deg : 0.99%
        500cm, 10deg : 2.97%

Can you please help by sharing full pipeline used for producing ground truth pose data?

[tsattler]

Can you tell me what you tried? It is hard to give feedback without details.

[patelajaychh]

My pipeline is like -

command 1 -

colmap mapper --database_path /data/outputs/hloc/ShopFacade/sfm_superpoint+superglue_ref+query/database.db \
--image_path /data/datasets/Cambridge/ShopFacade/ref+query/ \
--image_list_path /data/datasets/Cambridge/ShopFacade/reference_images_list.txt \
--output_path /data/outputs/hloc/ShopFacade/sfm_superpoint+superglue_ref_only  \
--Mapper.num_threads 16

Command 2 -

colmap mapper --database_path /data/outputs/hloc/ShopFacade/sfm_superpoint+superglue_ref+query/database.db \
--image_path /data/datasets/Cambridge/ShopFacade/ref+query/ \
--input_path /data/outputs/hloc/ShopFacade/sfm_superpoint+superglue_ref_only/ \
--output_path /data/outputs/hloc/ShopFacade/sfm_superpoint+superglue_with_query_gt_pose \
--Mapper.fix_existing_images 1

Pipeline -
reference+query images ( single dir) ---> Superpoint local feature detection ---> Superglue feature matching with Exhaustive Pairs --> running colmap mapper with command 1 -->output 1 - Sparse model --> running mapper with command2 --> Output 2 - with Query image poses

• output 2 model is used to extract poses for query images.
• For estimating query poses using PixLoc, I have used sfm_superpoint+superglue_ref_only model as input to Pixloc.
• I have evaluated PixLoc query pose estimates with query poses from output 2 as ground truth poses.

Also, I checked Mapper.fix_existing_images 1 has only fixed qvec associated with each reference image, but there is slight change in tvec values.
I was expecting tvec and qvec gets fixed of existing images while we register more images with colmap mapper with Mapper.fix_existing_images 1 set. Any explanation will be helpful.

[tsattler]

Structure-from-Motion systems such as Colmap are only able to reconstruct the scene up to an arbitrary rotation, translation, and scale. In other words, the units that you get from the reconstruction process do not correspond to physical units. If you want to measure distances in meters, you will need to scale the model accordingly. In the case of the Cambridge Landmarks dataset, this can be done by aligning the model with the provided ground truth camera positions, e.g., via Colmap's model_aligner.

If you want to be able to compare against other methods on a dataset, you should use the ground truth provided by the dataset and not recompute it from scratch as you will end up with (slightly) different poses. The way to go is to extract features and compute matches, import them into a colmap database, and then use the point_triangulator to obtain the 3D structure from the ground truth poses. Please see the Colmap documentation for details or look at how software packages such as hloc obtain 3D models given ground truth poses.

Regarding fix_existing_images: Looking at the code (https://github.com/colmap/colmap/blob/dev/src/sfm/incremental_mapper.cc#L691), setting fix_existing_images to true causes calling SetConstantPose (https://github.com/colmap/colmap/blob/dev/src/optim/bundle_adjustment.cc#L164). When the bundle adjustment problem is set up (https://github.com/colmap/colmap/blob/dev/src/optim/bundle_adjustment.cc#L265), images are added to the problem (https://github.com/colmap/colmap/blob/dev/src/optim/bundle_adjustment.cc#L331). If an image is marked as having a constant pose (https://github.com/colmap/colmap/blob/dev/src/optim/bundle_adjustment.cc#L361), a cost function is added to the bundle adjustment problem that fixes both the quaternion and the translation vector.