Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BERT] Reproducing GLUE score on 9 tasks #479

Closed
8 tasks
haven-jeon opened this issue Dec 24, 2018 · 4 comments
Closed
8 tasks

[BERT] Reproducing GLUE score on 9 tasks #479

haven-jeon opened this issue Dec 24, 2018 · 4 comments

Comments

@haven-jeon
Copy link
Member

@haven-jeon haven-jeon commented Dec 24, 2018

Currently only provide MRPC task on https://github.com/dmlc/gluon-nlp/blob/master/scripts/bert/dataset.py

Needs to add other tasks.

  • CoLA
  • SST-2
  • STS-B
  • QQP
  • MNLI
  • QNLI
  • RTE
  • WNLI
@kenjewu
Copy link
Contributor

@kenjewu kenjewu commented Dec 24, 2018

I want to reproduce the STS task.

@szha
Copy link
Member

@szha szha commented Dec 24, 2018

Thanks for bringing it up, @haven-jeon, and thanks to @kenjewu for offering to help.

cc @eric-haibin-lin

@eric-haibin-lin eric-haibin-lin changed the title Reproducing BERT GLUE score on 9 tasks [BERT] Reproducing GLUE score on 9 tasks Dec 24, 2018
@haven-jeon
Copy link
Member Author

@haven-jeon haven-jeon commented Dec 25, 2018

I requested #481.

@eric-haibin-lin eric-haibin-lin added this to In progress in GluonNLP 0.6.0 Dec 25, 2018
@eric-haibin-lin eric-haibin-lin added this to In progress in BERT Jan 3, 2019
@eric-haibin-lin eric-haibin-lin added this to the BERT on 9 GLUE Tasks milestone Jan 4, 2019
@eric-haibin-lin eric-haibin-lin removed this from the BERT on 9 GLUE Tasks milestone Jan 4, 2019
@eric-haibin-lin
Copy link
Member

@eric-haibin-lin eric-haibin-lin commented Jan 4, 2019

@haven-jeon I created separate issues for each of these tasks and we can now track these separately as a milestone in https://github.com/dmlc/gluon-nlp/milestone/2
I'll close this for now

GluonNLP 0.6.0 automation moved this from In progress to Done Jan 4, 2019
BERT automation moved this from In progress to Done Jan 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
BERT
  
Done
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants