-
Notifications
You must be signed in to change notification settings - Fork 165
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Broken Offline NeRFCapture #59
Comments
Hi, Thanks for trying out our code! As suggested by this comment, the offline mode in NeRFCapture is broken and tends to give erroneous depth maps: #7 (comment). I believe that this is the problem you are currently facing. We don't have an alternative for the NeRFCapture setup yet. We hope to release an updated variant of the demo soon. Another possibility is to use apps such as Record3D (however, the input format might have to be looked into). |
I try to fix the offlineMode of NerfCapture. (https://github.com/Zhangyangrui916/NeRFCapture)
|
For who are still struggling for offline data, pls use https://spectacularai.github.io/docs/sdk/tools/nerf.html instead of Nerfcapture. |
Thanks for sharing this, looks very cool! We will update our README to reflect the same. |
Thanks for sharing your paper and codes first, they look really cool.
My approach
I tried to run the codes on my own computer, but I don't know anything about cyclonedds at all. So I can only try to run splatam by manually putting the rgb, depth and transforms.json file captured by NeRFCapture on my computer. Here is what I tried to do:
<SplaTAM>/experiments/iPhone_Captures/offline_demo
.<num>.depth.png
to<num>.png
offline_demo
folder and run this to change the images name in json.Question
The code runs without any problems, but unfortunately the results very poorly (even though I have GT Poses for Tracking turned on).
![Screenshot 2024-01-17 163024](https://private-user-images.githubusercontent.com/16590434/297313202-8ab55cea-0123-4b62-90cf-d976cdd17c22.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU0MjE4NTQsIm5iZiI6MTcyNTQyMTU1NCwicGF0aCI6Ii8xNjU5MDQzNC8yOTczMTMyMDItOGFiNTVjZWEtMDEyMy00YjYyLTkwY2YtZDk3NmNkZDE3YzIyLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MDQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTA0VDAzNDU1NFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTZjNDIwYzYzYWE1ZmMyMzRkNzdjYWFhNmEzMGM3ODRmMzA1NTRmNTMxZDQ0MzgxZDljMWNiYjQ4YTlmNWQyZTgmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.Yn23QxgDt9_29JRteuKhLZlBcwi4_uPYeA8knCASax0)
![0001](https://private-user-images.githubusercontent.com/16590434/297313352-ed23cd0e-d453-4ead-8bf2-a81af416b1bb.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU0MjE4NTQsIm5iZiI6MTcyNTQyMTU1NCwicGF0aCI6Ii8xNjU5MDQzNC8yOTczMTMzNTItZWQyM2NkMGUtZDQ1My00ZWFkLThiZjItYTgxYWY0MTZiMWJiLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MDQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTA0VDAzNDU1NFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWIzZmMwYzI4MTk5ZjI3NjM0MTc3NDA5ZTAxNTAyMzQxNTIzMWU0YWIxYzI1MTg0Mzg0NjI4Njk2ZGRlNzBlZTMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.NImIGym9Y4DBxqDORZCIYseZnd-GI7dOf-paMoxTHy0)
![Screenshot 2024-01-17 164003](https://private-user-images.githubusercontent.com/16590434/297315906-6b1a94fc-4c4c-4ca8-b019-5008506101ef.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU0MjE4NTQsIm5iZiI6MTcyNTQyMTU1NCwicGF0aCI6Ii8xNjU5MDQzNC8yOTczMTU5MDYtNmIxYTk0ZmMtNGM0Yy00Y2E4LWIwMTktNTAwODUwNjEwMWVmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MDQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTA0VDAzNDU1NFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTMxMWQxNGZkMzVjNTAwOWE1ZDJjNTk3MTYwOGM3ZWQ3ZmQ1NmRjZTIzYmM2YjExMjk2OWU3YTc3YWQ4NjE3MTAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.zlGZKnpKiaSEl9YoIoKmWgecuLUODkpyeyUNA_wrsww)
Without turning on GT, the generated Gaussian sphere clusters are all squished together, and both Ground Truth Depth and Rasterized Depth within the eval show 0. I'm wondering what I'm doing wrong? One possibility I can think of is that I'm getting the wrong depth map because the same code behaves fine when running replica data.
Any suggestions on adjusting the NeRFCapture depth map please? Or any way to get depth from images/videos.
Appreciate it.
The text was updated successfully, but these errors were encountered: