New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can you please release trained model? #10
Comments
By the way, I tried to retrain bert-base-cased model and the F1 and Ign_f1 score on the dev set are 60.934 and 59.023, respectively. But when I change below line: Line 220 in 9c6f758
to
to evaluate saved model on the test set. The result I got on test is realy low: Am I wrong when changing code like that? And do you know why this error is happening? Thank you so much. |
I've released the trained models. 60.934 is a normal result. I report the result of bert-base as 61.09 +- 0.16, and 60.934 is around 1 std below average. The only way to get test results is to upload result.json to Colab. The downloaded DocRED test data does not provide the ground truth label so evaluating on it leads to undefined behavior. |
Thank you so so much for releasing the pre-trained model! I am truly grateful for that. And I was able to successfully get correct model's result on test set from Codalab (I think you may have a small typo here and in the readme.md, too). I will close this issue. Once again, thank you so much! |
Hi. Thank you for releasing the codes of your model, it is really helpful.
However I tried to retrain ATLOP based on the bert-base-cased model on the DocRED dataset but I can't get high result as your result on the paper. And I can't retrain roberta-large model because I don't have strong enough GPU (strongest GPU on Google Colab is V100). So can you please release your trained model. I would be very very happy if you can release your model, and I believe that it can help many other people, too.
Thank you so much.
The text was updated successfully, but these errors were encountered: