-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Current Reality Capture Nerfstudio Import process producing very bad quality for Splatfacto. #3118
Comments
The converter code was created before splatfacto. I believe a possible issue could be related to porting point cloud from reality capture to ply data with proper coordinate transform. Just want to confirm if nerfacto works for your reality capture data? If the quality is also bad, then maybe something has been broken. |
I havent tried with Nerf but my only interest is with Gaussian, I doubt the Nerf would fare much better if the conversion is incorrect to begin with. After looking at the function that handles the conversion, I can see it's definitely overlooking a few things. Reality Capture has documentation regarding the Principal Point and Distortion Models. |
@Maxiviper117 I am not a user of reality capture, if you know how to make it work with splatfacto, please send a PR fixing that. |
I'm not qualified to do that, closing issue hopefully someone else can help in future :( |
Re-opening to leave it to another good samaritan. |
I was able to fix this issue using #3131 , the screenshot is done at about 3k iterations |
Describe the bug
The current nerfstudio instructions on importing from Reality Capture at: https://docs.nerf.studio/quickstart/custom_dataset.html#realitycapture don't produce expected results.
I have a suspicion that the issue may be in using the exported CSV from
Internal/External camera parameters
export option.Does the CSV export take into account the distortion the images undergo as Reality Capture optimizes the alignment?Or is that do in Nerfstudio?
This problem does not happen if using the colmap
.txt
andcolmap.db
format generated from the bundler.out
and imagelist.lst
exports with undistorted images, using the process shown here https://gist.github.com/jo-chemla/258e6e40d3d6c2220b29518ff3c17c40Then taking that colmap format and using the instant-ngp
colmap2nerf.py
script to convert to atransforms.json
file with this section of that script disabled in thecolmap2nerf.py
script:Using this newely generate transforms.json file with the undistorted images, results is far greater quality (as shown in the screenshot) below.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Expecting quality to not be this bad.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: