-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Runtimeerror during training #2
Comments
Hi, @YangJae96 , it seems that Apex is not properly installed on your machine, you can try to re-install it. There won't be errors during memory initialization because apex amp is not employed in that process. |
I see. I am runnning the training but it is taking a while. |
Hi, @YangJae96 , it takes about 30hours to train the model with PRW as the target domain on a single A100 GPUI. The reverse setting takes much less time because it achieves the best performance at the 7th epoch, you can get more training details from our provided train_log in README. |
@caposerenity Okay! Thank you for your detail explanation |
Hi, I am facing some error while running your codes.
There was some error preparing the setting
such as faiss-gpu, apex, ex..
but I made the code running and like below.
THe memory seems to be initalized but the error comes up right at the
begin of training.
Could you give some help?
The text was updated successfully, but these errors were encountered: