Skip to content
Evaluation of the morphological quality of machine translation outputs
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.


Evaluation of the morphological quality of machine translation outputs. The automatically generated test suite in English should be translated into one of the supported target languages (French, Czech). The output is then analyzed and provides three types of information:

  • Adequacy: has the morphological information been well conveyed from the source?
  • Fluency: do we have local agreement?
  • Consistency: how well is the system confident in its prediction?


How To

Translate the source file morpheval.limsi.v2.en.sents and run the Moses tokenizer on it (with arguments -no-escape and -l {fr|cs|de}). Then:


python3 -i output.tokenized -n -d lefff.pkl


cat output.tokenized | sed 's/$/\n/' | tr ' ' '\n' | morphodita/src/run_morpho_analyze dictionary --input=vertical --output=vertical > output.analysis
python3 -i output.analysis -n


cat output.tokenized | tr ' ' '\n' | sort | uniq | ./smor > output.smored
python3 -i output.tokenized -n -d output.smored


Franck Burlot and François Yvon, Evaluating the morphological competence of machine translation systems. In Proceedings of the Second Conference on Machine Translation (WMT’17). Association for Computational Linguistics, Copenhagen, Denmark, 2017.

You can’t perform that action at this time.