PROTEST Pronoun Test Suite Annotation Interface
Christian Hardmeier (Uppsala University) and Liane Guillou (University of Edinburgh)
This repository contains the graphical annotation interface for the PROTEST pronoun test suite for machine translation. It allows you to annotate examples of pronouns and their translations for correctness.
The annotation tool is described in the following paper:
Christian Hardmeier and Liane Guillou. A graphical pronoun evaluation tool for the PROTEST pronoun evaluation test suite. Proc.\ 19th Annual Conference of the European Association for Machine Translation (EAMT), Riga LV, May 2016. Baltic Journal of Modern Computing 4 (2), 2016, pp. 318--330. [http://www.bjmc.lu.lv/fileadmin/user_upload/lu_portal/projekti/bjmc/Contents/4_2_21_Hardmeier.pdf]
A description of the PROTEST test suite itself was published here:
Liane Guillou and Christian Hardmeier. PROTEST: A Test Suite for Evaluating Pronouns in Machine Translation. Proc. 10th International Conference on Language Resources and Evaluation (LREC), Portoroz SI, May 2016, pp.\ 636--643. [http://www.lrec-conf.org/proceedings/lrec2016/summaries/327.html]
The current version of the test suite can be found at [https://bitbucket.org/lianeg/protest].
There is some additional documentation in the
documentation directory. For the time being,
however, we recommend that you contact one of the authors for help if you're interested
in using this tool.