Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Training time and TQDM #4

Open
antoine-crettenand opened this issue Oct 24, 2022 · 1 comment
Open

Training time and TQDM #4

antoine-crettenand opened this issue Oct 24, 2022 · 1 comment

Comments

@antoine-crettenand
Copy link

antoine-crettenand commented Oct 24, 2022

Hi,

Before all, thank you for your inspiring work !
I'm trying to process some custom data (that I preprocessed with COLMAP as indicated in the original NeuS repository) and compare the result with plain NeuS but I feel like the training process takes much more time than it should be (actually ~5*training time of plain NeuS).
I have strange outputs from TQDM in the logs. It seems that it cannot estimate the true duration of an iteration. I get 7 digits it/s which is obviously wrong and consequently the remaining time metric is wrong as well (0 s. remaining).
Obviously training an additional MLP for the second SDF should make it slower to train but I didn't expect such drawback...

I was wondering if it was 'normal' as in you're aware of the situation or should I consider this as unusual and investigate my data ?

Thank you in advance!

@yiqun-wang
Copy link
Owner

yiqun-wang commented Oct 25, 2022

I just provided pre-trained models for better inference. For custom data, not sure if it is caused by the hardware environment, moreover it is hard to judge the impact of the size of the custom data on time efficiency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants