Switch branches/tags
Nothing to show
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
..
Failed to load latest commit information.
README.md
evaluate.py
sample_output_scorer.txt

README.md

SemEval-2018 Task 3 - evaluation script

evaluate.py This is the official scoring script for SemEval-2018 Task 3: Irony detection in English tweets. The scoring script will be used to evaluate both subtask A and B.

The script looks at the input data to define the subtask at hand. It is therefore important that participants give correct names to their submission files.

  • Task A: predictionsTaskA.txt
  • Task B: predictionsTaskB.txt

The script outputs a file scores.txt with the accuracy, precision, recall and F1-score corresponding to the submission.

Questions or remarks on the scoring programme can be addressed in the Google discussion forum of this task: https://groups.google.com/forum/#!forum/semeval2018-task3.