Snippet Question
Each snippet can have a couple of related-questions. They could be used to test whether raters focused on their tasks or just randomly gave ratings.
Snippet questions can be one of four types:
-
SINGLE_CHOICE: Raters can select only one option.
-
MULTIPLE_CHOICE: Raters can select more than one option.
-
RATING: Raters select one option for each given attribute. For example, there are 2 attributes that are "Should they use singleton pattern?" and "Should they use prototype pattern?", and 2 options are "Yes" and "No".
-
INPUT: Raters are asked to fill the answer.
Snippet questions can be added in the rating editor interface. By click on the "Add question" button at the bottom of the page, a pop-up window will appear and show the adding snippet question form. Different question types can have different configurations:
The input snippet questions are the simplest that each question only has the content, constraint, and "Hidden at first" checkbox. Beside the required constraint that enforces raters to answer this questions, the "Input is a number" checkbox marks that this question only accepts a number as its answer.
The "Hidden at first" option ensures that this question won't be displayed at the first time. It will only be displayed after raters already gave ratings and answered all other required questions.
Both single choice and multiple choices questions have the same set of configurations. They have all configurations similarly to input questions except the options configurations.
Rating question is similar to choices questions except that there is a place to add attributes.
Sometimes, the order of questions matters. Administrators can easily reorder questions of a snippet by dragging and dropping.
