Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 

README.md

Introduction

This repository contains additional reference translations for the WMT'14 En-De and En-Fr test sets (newstest2014) as described in the paper: Analyzing Uncertainty in Neural Machine Translation.

In particular we provide 10,000 additional reference translations written by professional translators:

  • 10 additional reference translations for each of 500 sentences of the WMT'14 En-De test set (5,000 total)
  • 10 additional reference translations for each of 500 sentences of the WMT'14 En-Fr test set (5,000 total)

Format

Each line in the data file contains a sentence and an identifier, where identifiers represent:

  • S-123 English source for sentence #123 of newstest2014
  • T-123: Original reference translation (German or French)
  • R1-123 through R10-123: Additional reference translations

Sentence numbers correspond to the index of the sentence in the full test set (see below).

Full test sets

The full test sets from which the sentences were sampled can be reconstructed with sacrebleu:

sacrebleu --test-set wmt14 --language-pair en-de --echo src
sacrebleu --test-set wmt14 --language-pair en-de --echo ref
sacrebleu --test-set wmt14/full --language-pair en-fr --echo src
sacrebleu --test-set wmt14/full --language-pair en-fr --echo ref

Citation

If you use the data in your paper, then please cite it as:

@inproceedings{ott2018analyzing,
  author    = {Ott, Myle and Auli, Michael and Grangier, David and Ranzato, Marc'Aurelio},
  title     = {Analyzing Uncertainty in Neural Machine Translation},
  booktitle = {International Conference on Machine Learning},
  year      = 2018,
}

License

The dataset is licenced under CC-by-NC, see the LICENSE file for details.

About

Analyzing Uncertainty in Neural Machine Translation

Resources

License

Releases

No releases published

Packages

No packages published
You can’t perform that action at this time.