Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Which table result is the real result #13

Open
JinFish opened this issue Sep 20, 2022 · 4 comments
Open

Which table result is the real result #13

JinFish opened this issue Sep 20, 2022 · 4 comments

Comments

@JinFish
Copy link

JinFish commented Sep 20, 2022

According to the description of the paper, ITA-ALL+CVA in Table 2 should be the performance of the model, but in Table 4 the performance of ITA-ALL+CVA is different from that in Table 2. May I ask which table's result is the true result and why the two tables' results are different?

@wangxinyu0922
Copy link
Member

Hi, the result in Table 1 is the averaged result over 5 times. In table 2, since most of these previous approaches report the best model accuracy instead of the averaged model accuracy, we use the best model accuracy of ITA-All+CVA over 5 runs. You may read section 3.2 for more details.

@JinFish
Copy link
Author

JinFish commented Sep 20, 2022

Thank you for your reply, but I would like to ask why the results of table 2 and table 4 are not consistent and the results of table 4 and table 1 are not consistent. What you replied to me is the reason why the results of table 1 and table 2 do not agree.

@wangxinyu0922
Copy link
Member

wangxinyu0922 commented Sep 20, 2022

Oh, in table 4, I put our initial results of Twitter 15/17. The models are trained with LA+GA+CVA. I will fix this issue in our newest arxiv version. You may take the results in Table 1 and Table 2.

@JinFish
Copy link
Author

JinFish commented Sep 20, 2022

Thank you very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants