LEPOR is a language independent machine translation evaluation metric. LEPOR is open source, free for research purpose. Experiments on ACL-WMT11 corpora (English to German, French, Spanish, Czech; and the reverse direction) show LEPOR yields higher correlation scores with human judgments at system-level, as compared to BLEU, METEOR, TER, MP4IBM1, and AMBER evaluation metrics. Detailed knowledge of LEPOR is shown in the paper "LEPOR: A Robust Evaluation Metric for Machine Translation with Augmented Factors" by Aaron Li-Feng Han, Derek F. Wong and Lidia S. Chao, in Proceedings of the 24th International Conference on Computational Linguistics (COLING 2012), pages 441–450, Mumbai, December 2012. Source code: https://github.com/aaronlifenghan/aaron-project-lepor & https://code.google.com/p/aaron-project-lepor/ Online paper: http://aclweb.org/anthology-new/C/C12/C12-2044.pdf.
The enhanced version of the metrics hLEPOR and nLEPOR are introduced in the paper "Unsupervised Quality Estimation Model for English to German Translation and Its Application in Extensive Supervised Evaluation" by Aaron Li-Feng Han, Derek F. Wong, Lidia S. Chao, Liangye He and Yi Lu. 2013. The Scientific World Journal, Issue: Recent Advances in Information Technology. Hindawi Publishing Corporation. [http://www.hindawi.com/journals/tswj/aip/760301/]
If you use the LEPOR metric in your researches, please cite the paper.
Citations:
@inproceedings{han2012lepor, title={LEPOR: A Robust Evaluation Metric for Machine Translation with Augmented Factors}, author={Han, Aaron L.-F. and Wong, Derek F. and Chao, Lidia S.}, booktitle={Proceedings of the 24th International Conference on Computational Linguistics (COLING 2012)}, pages={Posters--441}, year={2012}, organization={Association for Computational Linguistics} }
@inproceedings{han2013description, title={A description of tunable machine translation evaluation systems in WMT13 metrics task}, author={Han, Li Feng and Wong, Fai and Chao, Sam and Lu, Yi and He, Liang Ye and Wang, Yi Ming and Zhou, Jia Ji}, booktitle={The eighth workshop on statistical machine translation, Aug. 2013, p. 414-421}, year={2013} }
Contact: hanlifengaaron AT gmail DOT com
Address: ADAPT Centre, DCU, Dublin, Ireland