How to generate poses for query images using SFM model built on reference images.

[patelajaychh]

Hi,
I want to generate poses for query (new) images of a scene using the SFM model built on reference images only.
One way I found is to use image_registrator in COLMAP (not sure if it is the correct way).

But I'm getting below error while running command -

colmap image_registrator     \
--database_path  /data/outputs/hloc/King_Seq1_ref_only/sfm_superpoint+superglue_with_query/database_with_query.db     \
--input_path  /data/outputs/hloc/King_Seq1_ref_only/sfm_superpoint+superglue   \
--output_path  /data/outputs/hloc/King_Seq1_ref_only/sfm_superpoint+superglue_with_query

Std Output-

==============================================================================
Loading database
==============================================================================

Loading cameras... 261 in 0.000s
Loading matches... 33929 in 0.125s
Loading images... 261 in 0.019s (connected 261)
Building correspondence graph... in 2.984s (ignored 682)

Elapsed time: 0.052 [minutes]

F1213 16:50:19.615032 97173 reconstruction.cc:81] Check failed: existing_image.Name() == image.second.Name() (seq1_frame00261.png vs. seq1_frame00211.png)
*** Check failure stack trace: ***
    @     0x7fd54172d0cd  google::LogMessage::Fail()
    @     0x7fd54172ef33  google::LogMessage::SendToLog()
    @     0x7fd54172cc28  google::LogMessage::Flush()
    @     0x7fd54172f999  google::LogMessageFatal::~LogMessageFatal()
    @     0x55b891c61a6d  (unknown)
    @     0x55b891dc8d4e  (unknown)
    @     0x55b891badaf5  (unknown)
    @     0x55b891b9ca0e  (unknown)
    @     0x7fd53d3afb97  __libc_start_main
    @     0x55b891ba66aa  (unknown)
Aborted (core dumped)

The paper On the Limits of Pseudo Ground Truth in Visual Camera Re-localisation has presented that they have used similar technique (without mentioning the COLMAP function) to find ground truth poses for query images. This is the reason I'm posting this issue here.

Am I going the correct way? If not what is the correct method to do so?

[tsattler]

For the paper, we used the mapper functionality: we first use the --image_list_path option to specify that only the training images should be used for reconstruction. Once the process finishes, we run the mapper again with the output of the first reconstruction process as input and using the --fix_existing_images option set to True to fix the poses of the training images while the test images are added to the reconstruction.

Note that this is different to the image_registrator as the image_registrator adds individual images without any bundle adjustment over all images and without adding new 3D points or new observations.

Looking at the error you show, I would assume that the database you use for the image_registrator is different to the one you used to build the model as the error states that the ID associated with one of the images in the reconstruction points to another image in the new database. You can avoid the problem by either extending the original database you used to include the query image or by running the point_triangulator with the --clear_points option set to True with the new database and the 3D model that you have.