Skip to content
This repository has been archived by the owner on Jan 15, 2024. It is now read-only.

[Script] Add BERT SST-2 finetuning result #600

Merged
merged 2 commits into from
Feb 23, 2019
Merged

[Script] Add BERT SST-2 finetuning result #600

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c86b14a
add sst
eric-haibin-lin Feb 18, 2019
48cf935
Merge branch 'master' into sst
eric-haibin-lin Feb 19, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 7 additions & 1 deletion scripts/bert/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,4 +45,10 @@ It gets validation accuracy of `88.7% <https://raw.githubusercontent.com/dmlc/we
It gets RTE validation accuracy of `70.8% <https://raw.githubusercontent.com/dmlc/web-data/master/gluonnlp/logs/bert/finetuned_rte.log>`_
, whereas the the original Tensorflow implementation give evaluation results 66.4%.

Some other tasks can be modeled with `--task_name` parameter.
.. code-block:: console

$GLUE_DIR=glue_data python3 finetune_classifier.py --task_name SST --epochs 4 --batch_size 16 --accumulate 1 --optimizer bertadam --gpu --lr 2e-5 --log_interval 500

It gets SST validation accuracy of `93.0% <https://raw.githubusercontent.com/dmlc/web-data/master/gluonnlp/logs/bert/finetuned_sst.log>`_.

Some other tasks can be modeled with `--task_name` parameter.