Guide on exporting scene from COLMAP

[DenShlk]

Hi! I have some troubles exporting point cloud from COLMAP to compatible with ADOP format.
Here is what I did:

1. Autoreconstruction in COLMAP (both sparse and dense)
2. File > Export As > as .ply
3. preprocessed it with colmap2adop tool
4. started training
5. got an error that my point cloud file does not contain normal data
6. I checked that your .ply files have normal data in headers, but I struggle to make COLMAP include it in mine

[darglein]

So the .ply point cloud from COLMAP does not have normals?

[DenShlk]

It doesn't

[darglein]

This is weird. The dense point cloud from COLMAP does usually have normals. Do you run an old version maybe? Or did you export the sparse point cloud by accident?

[DenShlk]

I installed it literally yesterday from the last release in their github (So don't be surprized about stupid questions)
I did Undistortion, Stereo, Fusion steps in Dense reconstruction window, and chose latest model, so it should be dense model, right?

[DenShlk]

I opened fused.ply and it contains nx, ny, nz values it looks like normals, but previous time I tried to set point_cloud_file in comap2adop to fusion result it went to endless loop

[darglein]

Yes fused.ply is the correct file you should use. What do you mean by endless loop? At which point does it get stuck? For large point clouds colmap2adop converter can take a few minutes to complete.

[DenShlk]

At removing duplicate points

ADOP/src/lib/data/SceneData.cpp

Line 80 in 51fbf9e

point_cloud.RemoveDoubles(0.0001);

I waited for more than 30 mins... I tried to run it on some of example scenes and it really took minute or so. So I thought I use wrong file.
I'll try with my new cloud, then.

[DenShlk]

UPD: fixed it by passing path to original images directory to --image_dir.

I found this in output of colmap2adop:

Num Point3D 68762
Num         = [116]
Min,Max     = [-nan,-nan]
Mean,Median,Rms = [-nan,-nan,-nan]
sdev,var    = [-nan,-nan]
dynamic range: -nan

Is it ok?

[DenShlk]

So, now the problem is that for some reason RemoveDoubles function takes a lot more time than on example scenes.
It also worth noting that my scene is 5 times smaller than yours.

[darglein]

I would like to take a look at your fused.ply point cloud and see why preprocessing it takes so long. Can you upload it somewhere and send me the link?

[darglein]

UPD: fixed it by passing path to original images directory to --image_dir.

I found this in output of colmap2adop:

Num Point3D 68762
Num         = [116]
Min,Max     = [-nan,-nan]
Mean,Median,Rms = [-nan,-nan,-nan]
sdev,var    = [-nan,-nan]
dynamic range: -nan

Is it ok?

Thanks for pointing that out. This was a bug and is now fixed in the latest commit.

[DenShlk]

Link to my .ply

In meantime, in UnifiedMesh.cpp, in RemoveDoubles function I printed the size of vector, returned from RadiusSearch.
It is equal to 272339 sometimes, so I guess it causes O(n^2) complexity.
https://github.com/darglein/saiga/blob/0d6d033bf88a01ca674c15fb5907d6f5346fa12e/src/saiga/core/model/UnifiedMesh.cpp#L439-L484

[DenShlk]

I made a work-around for this issue. I'd ask you to check that I haven't broken anything 👀

    std::vector<int> to_merge(NumVertices());
    std::vector<bool> processed(NumVertices(), false);

    std::vector<int> to_erase;
    std::vector<int> valid(NumVertices(), 0);
    
    KDTree<3, vec3> tree(position);
    for (int i = 0; i < position.size(); ++i)
    {
        if (processed[i]) {
            continue;
        }

        auto ps     = tree.RadiusSearch(position[i], distance);
        bool found  = false;
        to_merge[i] = i;
        for (auto pi : ps)
        {
            if (valid[pi])
            {
                to_erase.push_back(i);
                to_merge[i] = pi;
                found       = true;
                break;
            }
        }
        if (!found)
        {
            valid[i] = true;
            // all points in radius will find that point i is valid and do the same
            for (auto pi : ps)
            {
                if (!processed[pi] && pi != i) {
                    to_erase.push_back(pi);
                    to_merge[pi] = i;
                    processed[pi] = true;
                }
            }
        }
        processed[i] = true;
    }

It is quite strange that from 1955964 points I get 23659, but there really might be a lot of duplicates

[darglein]

It looks like remove doubles is not broken.

The problem is that Assimp is not able to load your point cloud correctly. A simple workaround for now is to open the pointcloud with meshlab and select File -> export mesh -> Select Color, Normal, Binary Encoding -> Ok

After that Assimp is able to read it correctly.

I have added some debug checks in this commit f360201 to ensure the input point cloud is read correctly.

[DenShlk]

Great thanks! It finally works, I managed to start training 😎

[DenShlk]

So, for those who have similar questions, footage processing pipeline:

1. Create new project in COLMAP

2. Start autoreconstruction and go watch film or something

3. run colmap2adop tool:
   ./build/bin/colmap2adop --sparse_dir <sparse/0 in your project folder> --image_dir <your original footage dir> --point_cloud_file <find fused.ply in your project folder> --scale_intrinsics 1 --render_scale 1 --output_path <ADOP dir>/scenes/<your scene name>

4. Note, that you want to set render_scale to <1 to fit images to gpu memory, but value depends on initial images scale and your gpu

5. @darglein could you explain what scale_intrinsics does, please

6. you can train ADOP on your data now!

[darglein]

@DenShlk Thanks for the instructions. I will add this those to the readme once all bugs are fixed.

I have just updated to use the latest saiga+assimp version a1f0219. @DenShlk could you please check if colmap2adop can process the raw COLMAP point cloud now?

(make sure to do a recursive submodule update after the pull)

scale_intrinsics is another way to increase training speed and reduce memory consumption. Let's say COLMAP was run on 2000x1000 images then the intrinsic parameters from colmap fit to this image size. If you set scale_intrinsics=0.5 then the intrinsic parameters will be scaled to fit images of 1000x500. Therefore you also have to downscale the images by a factor of 2 and then pass the downscaled images to colmap2adop. In some cases this is better than render_scale because the DataLoader has to read less bytes from disk and needs to do less processing.

[DenShlk]

Hi! Sorry for delay.

@DenShlk could you please check if colmap2adop can process the raw COLMAP point cloud now?

I checked it, it works perfectly. Thanks!

I'd also asked you to consider using optimization for RemoveDuplicates function I suggested above, because at the moment computing time of it depends on how close are points relatively to distance parameter. (e.g. if you pass distance=1000 so almost all points should be deleted, it will do number_of_points^2 steps).

[Lu-pin-an]

Hi! I try to build my scenes with COLMAP,but when i run colmap2adop,it said "Assertion 'std::filesystem::exists(img_file)' failed!"
but i check the path,the image files are existence，and the path is also correct，how can i solve this problem?

[darglein]

The converter did not find images.bin in the sparse directory. Please check if this file exists on your provided path.

[Lu-pin-an]

the images.bin is exsist in the sparse directory,but i copy the image directory and sparse directory from another computer,does this matter ?

[Lu-pin-an]

I find it was caused by a Wrong file organization，thank you ！