New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A problem when running sh cmds/20/motif/predcls/sup/train.sh #2
Comments
Sorry, I did not meet this problem before. But I guess this might be caused by one of the processes in the distributed training crashes or meets some other problems? So, the program will wait for the predictions of the crashed process and just get stuck here. |
Thank you very much for your reply. After training and testing, what do the three files eval_results.pytorch, result_dict.pytorch, and visual_info.json in the inference folder mean? How to run them to see the results?
…------------------ 原始邮件 ------------------
发件人: "thunlp/VisualDS" ***@***.***>;
发送时间: 2021年10月19日(星期二) 中午11:53
***@***.***>;
***@***.******@***.***>;
主题: Re: [thunlp/VisualDS] A problem when running sh cmds/20/motif/predcls/sup/train.sh (Issue #2)
Sorry, I did not meet this problem before. But I guess this might be caused by one of the processes in the distributed training crashes or meets some other problems? So, the program will wait for the predictions of the crashed process and just get stuck here.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
|
The generated data is the same with the original repo. You can take a try to find details from the original repo. |
The program gets stuck while running. The following problems occur, and the GPU has been stuck at 100%
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
The text was updated successfully, but these errors were encountered: