-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Number of gold arguments for ChFinAnn #64
Comments
The number of gold arguments in PTPCG is the same as other baselines that use ChFinAnn. |
Hi there, does my response answer your questions? I'd like to close this issue if there's no further discussion. |
Hi @Spico197 after training the model on ChFinAnn, the test data arguments TP+FN = 28,545 but when I count the arguments from the original test data, it is 29,345. I traced the missing arguments and found that they are dropped during the truncation of sentences and documents. Can you confirm? Thanks. |
Yes. The default setting of the number of sentences in a document is 64, while the max sequence length is 128, so some documents are trucated. Doc2EDAG, GIT, PTPCG use the same setting. It may be potentially unfair if you use other settings. |
I didn't check the exact numbers yet, but do you mean arguments instead of mentions or entities? |
yes, I mean the arguments in event tables |
I understand. I'll try to get the statistics soon. |
@Spico197 Hi! Would you be able to share the model predictions for ChFinAnn and DuEE-fin dev? I really appreciate your valuable time. |
Hi there, sorry for the late response. Things been busy these days. The attachment below contains:
|
In case of any inconvenience for your analysis, I updated the PTPCG task dump trained on DuEE-Fin. |
Thanks a lot! |
** Problems **
How many gold arguments are used to calculate the P/R/F1 for ChFinAnn reported in the paper?
The text was updated successfully, but these errors were encountered: