New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
training stopped at epoch 1 #34
Comments
I just run the training on an RTX-4000:
It's about 10 minutes after the data were loaded. |
I m using linux without gpu though i hv a gpu but havnt setup cuda in it and dont know how to setup too, to train it or colab notebook. Could you post up a pretrained model |
Linux gets hanged while in prepro.py but incolab it toom more than 2.5 hours still stuck at epoch 1 |
On CPU it's supposed to be slow. I haven't tested on CPUs before. And I do not have any experience with Colab. Did you complete your prepro.py? |
On colab i did completed but not on cpu it was always getting stuck at dev part |
If you interrupt the kernel, what's the stack trace? It may help by indicating where it was stuck. |
Actually the system got hanged but would you mind if we get up on a short online meeting if you are free |
@hitvoice My Google Colab notebook is successfully working now. Shall I put up a pull request for those who want to implement it? |
Sorry for the late reply. Pull requests are definitely welcomed! |
can you tell me how long does it take for the training process to complete?
i am using a google colab notebook. and it has been stuck at epoch 1 since last 20 mins
The text was updated successfully, but these errors were encountered: