Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TREC-50 SOTA #93

Closed

Conversation

hossein-amirkhani
Copy link
Contributor

@hossein-amirkhani hossein-amirkhani commented Sep 6, 2018

The TREC-50 SOTA is changed by the "Universal Sentence Encoder" paper.

@sebastianruder
Copy link
Owner

I think the results from USE are for TREC-6 instead of TREC-50 as they use the dataset prepared by Conneau et al. (2017) who use TREC-6.

@hossein-amirkhani
Copy link
Contributor Author

But they mention "fine grained question classification" to introduce the TREC task. By "Fine-grained" I infer TREC-50.

@sebastianruder
Copy link
Owner

As I said, they use the data from Conneau et al. (2017) who do TREC-6 with 6 classes. This is already more "fine-grained" than binary classification. If they do TREC-50, then there's a mistake in the paper.

@hossein-amirkhani
Copy link
Contributor Author

So, if you are sure, I close this PR and creatre another PR for TREC-6.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants