Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How long do you need to train MVP? #19

Closed
buaacyw opened this issue May 5, 2022 · 2 comments
Closed

How long do you need to train MVP? #19

buaacyw opened this issue May 5, 2022 · 2 comments

Comments

@buaacyw
Copy link

buaacyw commented May 5, 2022

Thanks for your work! But it takes me about 3 days to train MVP (with 4 V100).
I changed your env to spconv 2 and CUDA11.1. If using CUDA10.2 and spconv 1 mentioned in your install guide, it takes longer.
So could you please tell me how long do you take?
I have enough GPU memories and is it possible to accelerate the training? Thanks!

@tianweiy
Copy link
Owner

tianweiy commented May 6, 2022

that sounds reasonable (2-3 days). Pytorch version may also matter for the speed.

I have enough GPU memories and is it possible to accelerate the training? Thanks!
you can try larger batch size (and linearly scale lr). Not sure how much faster it will be with this

@buaacyw
Copy link
Author

buaacyw commented May 12, 2022

Thanks!

@buaacyw buaacyw closed this as completed May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants