Switch branches/tags
Nothing to show
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
..
Failed to load latest commit information.
README.md
scorer_semeval18.py

README.md

Official evaluation script

The official evaluation script for this task can be run from the terminal as follows:

python scorer-semeval2018-task2.py [gold_key] [output_key]

Example of usage:

python scorer-semeval2018-task2.py trial/english.trial.gold.txt trial/english.output.txt

Note: Gold and output files should have the same number of lines. If for some occurrence the system is not confident enough to output any answer, just leave that line empty.