Medical Relation Extraction and Direction
In this template the goal is to provide the relation as well as the direction of the selected relation.
The workers are given a medical sentence with two highlighted terms (factors) in it. They are asked to select all relations that apply between them. The set of relations contains 23 possible relations: the 11 non-symmetric relations used in RelEx with their inverses (e.g. causes and caused-by), plus NONE and OTHER. The relation definitions available include an explanation of the directionality.
The crowd answers are stored in a vector, which captures each selection of the crowd as following: [R1, R1-inverse, R2, R2-inverse, R3, R3-inverse, R4, R4-inverse, R5, R5-inverse, R6, R6-inverse, R7, R7-inverse, R8, R8-inverse, R9, R9-inverse, R10, R10-inverse, R11, R11-inverse, NONE, OTHER, Answer-Validation-NONE, Answer-Validation-OTHER, Answer-Validation-Directionality]
This template could be executed in four versions, (1) with verification question, (2) without verification questions, (3) with gold sentences and (4) without gold sentences.
