SemEval-2018 Task 3 - evaluation script
evaluate.py This is the official scoring script for SemEval-2018 Task 3: Irony detection in English tweets. The scoring script will be used to evaluate both subtask A and B.
The script looks at the input data to define the subtask at hand. It is therefore important that participants give correct names to their submission files.
- Task A: predictionsTaskA.txt
- Task B: predictionsTaskB.txt
The script outputs a file scores.txt with the accuracy, precision, recall and F1-score corresponding to the submission.
Questions or remarks on the scoring programme can be addressed in the Google discussion forum of this task: https://groups.google.com/forum/#!forum/semeval2018-task3.