Skip to content
Evaluation tool for pronouns in machine translation
Java XSLT Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
examples
java
mmax-import
README
compile.sh
eval.sh

README

This package contains software to compute the MT pronoun precision/recall score
described by Hardmeier and Federico (IWSLT 2010).

To run it, set the environment variable DISCOMT_HOME to the directory
containing this README file and start eval.sh. Run the tool without
arguments to see more detailed usage information.

Note that pronoun precision/recall is NOT the official evaluation score of the
DiscoMT workshop and that it has some well-known weaknesses, most notably that
it just matches pronouns against the reference translation without taking into
consideration the antecedent translations actually occurring in the candidate
translation. As such, the information extracted from the gold standard is not
necessarily correct for a particular candidate translation, so the score
should be taken with a grain of salt.

The scripts in the mmax-import directory may be useful to convert SMT input data
into the annotated MMAX format that can be read by the document-level SMT decoder
Docent. They are not used by the pronoun scorer.

10 April 2015
Christian Hardmeier
Uppsala University
You can’t perform that action at this time.