The repository contains resources of our paper published at AAAI 2020 ``A Dataset for Low-Resource Stylized Sequence-to-Sequence Generation "
I strongly recommend to use the MT data set to evaluate your Stylized Sequence-to-Sequence Generation system.
The data is uploaded to Google drive, https://drive.google.com/file/d/1TX5-7T4qLrM0PdZVvnDb-ePqo4evJjb0/view?usp=sharing. We also uploaded our predictions to the URL, so you can use your tool to evaluate different methods. According to our evaluation scripts, the results are as follows
| Formality | Fluency | Overall (BLEU Score) | |
|---|---|---|---|
| Base | 0.55 | 3.61 | 33.47 |
| Pivot_Rule | 0.68 | 3.58 | 37.83 |
| Pivot_Model | 0.679 | 3.64 | 38.75 |
| Teacher Student | 0.768 | 3.60 | 40.07 |
| backtranslation | 0.707 | 3.59 | 40.68 |
For more details, please read our paper.
Conversation data can be found at https://drive.google.com/file/d/1KirCpHgCqtBsD_5TY1OOaKdz-6A48Nt6/view?usp=sharing. Regarding training corpus, we only provide twitterID and userID due to legal issue. validation and test files are merged into one file, where 0-1000 are validation files and 1000-2000 are test files. Please ignore those rewriting results are none.
Because conversation outputs are extremely hard to evaluate, we do not recommand to evaluate your predictions with automatic metrics. However, we still provide our evaluation results.
| Formality | Fluency | Overall (Human Evalution) | |
|---|---|---|---|
| Base | 0.506 | 3.25 | 2.21 |
| Pivot_Rule | 0.583 | 3.69 | 2.41 |
| Pivot_Model | 0.807 | 3.72 | 2.51 |
| Teacher Student | 0.862 | 3.73 | 2.87 |
| backtranslation | 0.844 | 3.72 | 3.16 |