Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support COLMAP reconstructions with non-flat image dirs #3006

Conversation

zacstewart
Copy link
Contributor

@zacstewart zacstewart commented Mar 18, 2024

The code was previously looking for the file.name, which drops all directory information from the path. Instead, we want to get the file path relative to the data dir, which will only drop parts of the path above the data dir, allowing us to recurse into subdirectories within the images dir

For example, you can use COLMAP reconstructions that were built using multiple captures and have an images directory structured like this:

images
└── DJI_0238
    ├── 000000.jpg
    ├── 000041.jpg
    ├── 000060.jpg
└── DJI_0286
    ├── 000000.jpg
    ├── 000041.jpg
    ├── 000060.jpg

@jb-ye
Copy link
Collaborator

jb-ye commented Apr 1, 2024

@zacstewart Could you describe how you test this PR in step-by-step commands? My understanding is the change should support both single folder and multiple subfolder colmap project.

Also what's the relation of this PR with #2976?

@jb-ye jb-ye force-pushed the zacstewart/support-recursive-image-dirs branch from 7cbc71a to 49a8577 Compare April 1, 2024 17:56
@zacstewart
Copy link
Contributor Author

@jb-ye Given you have a colmap reconstruction that has subdirectories in its images directory, there are no more steps than usual. This PR only effects how ns-process-data images will behave by allowing it to recurse into those subdirectories instead of only taking top-level image files. The tedious part of testing would be getting an applicable colmap reconstruction.

Do you need me to help prepare a small test dataset to demonstrate it? I can do so but I'll need a week before I can get to it.

#2976 is an orthogonal issue. Colmap allows you to build reconstructions from homogeneous camera images, but ns-process-data treats all images as if they are from a single camera and makes each image share intrinsics in the transforms.json it produces. It's not strictly related to this PR (#3006), but typically images from different cameras are in different subdirectories, so in practice they are both necessary if you want to train a nerf from a colmap reconstruction built using heterogeneous images.

In my case, I already have a colmap reconstruction that I built myself, outside of nerfstudio. It required special care to produce because the scene is very symmetrical and images tended to mis-register. Nerfstudio's internal invocation of colmap would not be able to reproduce it.

The code was previously looking for the file.name, which drops all
directory information from the path. Instead, we want to get the file
path relative to the data dir, which will only drop parts of the path
above the data dir, allowing us to recurse into subdirectories within
the images dir
@jb-ye jb-ye force-pushed the zacstewart/support-recursive-image-dirs branch from 49a8577 to 1dff8c9 Compare April 2, 2024 17:58
Copy link
Collaborator

@jb-ye jb-ye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is a low risk change and bring clear benefits.

@jb-ye jb-ye merged commit 1b9dfe8 into nerfstudio-project:main Apr 2, 2024
2 checks passed
@zacstewart zacstewart deleted the zacstewart/support-recursive-image-dirs branch April 8, 2024 04:51
Michael-Spleenlab pushed a commit to Michael-Spleenlab/nerfstudio that referenced this pull request Apr 26, 2024
…roject#3006)

* Support COLMAP reconstructions with nested image dirs

The code was previously looking for the file.name, which drops all
directory information from the path. Instead, we want to get the file
path relative to the data dir, which will only drop parts of the path
above the data dir, allowing us to recurse into subdirectories within
the images dir

* Adhere to style guidelines

* Add testcase to cover image dirs with subdirs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants