Skip to content
master
Switch branches/tags
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Confidence Through Attention

Use attention alignments from neural machine translations to get a confidence score on how well the text has been translated. In addition, the same score can be used to compare translations (with attentions) from multiple NMT systems and perform hybrid selections of the best outputs.

Usage:

  • Train a neural MT system (Neural Monkey and/or Nematus)
    • Get source + translated texts + NumPy 3d tensor of alignments from Neural Monkey
    • Get translations together with alignments in one file from Nematus
  • Score one or the other, or combine both in a hybrid
    • Nematus
    python score.py -f Nematus -a test_data/nem.out.ali.lv
    • Neural Monkey
    python score.py -f NeuralMonkey -a test_data/nm.alignment.npy -s test_data/nm.bpe.en -t test_data/nm.out.bpe.lv
    • Hybrid
    python hybrid.py -nem test_data/nem.out.ali.lv -nm test_data/nm.alignment.npy -s test_data/nm.bpe.en -t test_data/nm.out.bpe.lv

Parameters for score.py:

Option Description Required Possible Values Default Value
-a Input alignment file Yes Path to file
-s Source sentence subword units For Neural Monkey Path to file
-t Target sentence subword units For Neural Monkey Path to file
-f Where are the alignments from No 'NeuralMonkey', 'Nematus' 'NeuralMonkey'

Parameters for hybrid.py:

Option Description Required Possible Values
-nem Nematus alignment file yes Path to file
-nm Neural Monkey alignment file yes Path to file
-s Neural Monkey source sentence subword units yes Path to file
-t Neural Monkey target sentence subword units yes Path to file

Publications

If you use this tool, please cite the following paper:

Matīss Rikters and Mark Fishel (2017). "Confidence Through Attention." In Proceedings of the 16th Machine Translation Summit (MT Summit 2017) (2017).

@inproceedings{Rikters-Fishel2017MTSummit,
	author = {Rikters, Matīss and Fishel, Mark},
	booktitle={Proceedings of the 16th Machine Translation Summit (MT Summit 2017)},
	title = {{Confidence Through Attention}},
	address={Nagoya, Japan},
	year = {2017}
}