Skip to content

Dataset

Jump to bottom
Tien-Duc Nguyen edited this page Sep 9, 2024 · 1 revision

A dataset represents a collection of snippets for rating.

The administrators can create, update, delete, archive, clone a dataset.

To create a dataset, you only need to navigate to the datasets page and hit the Create button. A dialog will be displayed so that you can configure your dataset.

image

There are a lot of configurations that you can use to setup your study:

  • Name: A short piece of text that represents the name of your dataset.

  • Description: The description will be displayed to raters. It is usually used to display a short guideline about how raters can interact with the tool. For example, it could be "Please giving 1-star for the least readable code or 5-star for the most readable code".

  • Hide comment box: Do not show the comment box in the snippet rating page. However, it will still be displayed if Allow no rating is enabled and raters choose No rating for this snippet because comment is required in case of No rating.

  • Hide snippet questions before rating: A snippet question can be set as hidden. If this option is enabled, the hidden questions will not be displayed in the first time raters access this snippet. Those hidden questions can only be displayed after raters hit the next snippet button of this snippet for the first time. This feature gives raters another chance to review their solution.

  • Allow no rating: Give one more option for raters to choose beside the normal rating star. Raters will be required to give a comment if they choose No rating.

  • Language: Select the programming language to highlight all snippets of this dataset correctly. If not provided, the tool will try to determine the programming language based on the extension part of the snippet's file path.

  • Rater management: Currently, CAT support two types of raters management:

    • Local: This is the default strategy in which the raters only need to register themself with the tool and do not need to configure any third-party platform.
    • Prolific: This strategy can be used if the administrators would prefer to use Prolific platform to looking for raters and manage them and the study result. The raters from Prolific will be redirect to Prolific page to complete their surveys. To select Prolific as raters management, the administrators need to configure 2 other options:
      • Complete code: The code that is used when redirecting raters to Prolific to complete their surveys.
      • API key: The API key that is required when interacting with Prolific's APIs. This field will never be displayed to any users even though it was already set.

  • Demographic question groups: Assign a couple of demographic question groups to this dataset so that raters will need to answer these questions before actually taking the study.

  • Complete text: If local raters management was used, this piece of text will be displayed after raters completed this survey.

To update a dataset, tap the pen button on that dataset and a dialog will be displayed similar to when creating a dataset but fields were already filled.

image

To delete a dataset, tap the dustbin button on that dataset. A dialog to display to confirm that you are not deleting a dataset by mistake.

image

To archive a dataset, tap the archive button on that dataset. This feature aims to move completed datasets to another page. The administrators can still access or even bring back archived datasets by navigating to archives page by the menu.

To clone a dataset, tap the duplicate button on that dataset. A dialog to display to allow you to choose if you'd like to clone snippets or not. This feature simply help administrators to create new dataset with the same configurations (and even the snippets).

Clone this wiki locally