Datasets

Add a dataset

Once you have created a project, you can begin adding datasets to the project. A project can have many datasets or only one.

Each dataset should have the same data model. Each dataset has its own form; in order to use a different form you must create an additional dataset. Information is exported from DataScribe by dataset.

To add a dataset, go to the Project view for your project, then select “Add new dataset” from the dropdown in the upper right corner of the browser window.

This will take you to the form to build a new dataset. There are fields for:

• Name (required): a name for the dataset
• Description: A description of the dataset. This will display in the dataset metadata when viewing the project.
• Guidelines: Instructions and useful information for transcribers of this dataset.
• Item Set (required): Select from a dropdown of item sets in the Omeka S install.

Using the eye icon in the upper right corner, you can set the dataset as private or public. Private datasets are only visible to Global Admins of the Omeka S install and those users who have been added to the project. A public dataset can be viewed by anyone who is logged in to the Omeka S installation.

Once you have created the dataset, you will be taken to the dataset browse page. A message should appear in the main work area “No items found. Sync this dataset. If you have recently synced, the sync job likely has not finished.”

To sync the dataset, either click the phrase “sync this dataset” or use the Actions menu in the upper right corner of the window and select the “Sync dataset” option. Syncing updates the dataset with the information from the items in the item set. For more on syncing once a dataset is populated, see below.

Once you have synced the dataset, you may need to reload the page in order to see the items.

Form builder

In order to transcribe items in a dataset, you must build a form for the dataset. You may find it useful to have a reference image or outline of the data you're collecting when you build the form.

Use the action menu in the upper right corner to select “edit dataset,” which will take you to the Edit page. The edit page has two tabs, a general tab where you can edit the information you entered when adding the dataset. The second tab is where you build the form for the dataset.

On the Form Builder tab, you will see a drawer on the right with options for building the form. At the top of the drawer is a “Save Progress” button. Use this at any time to save your progress without reloading the page. If you click the Save button in the upper right corner of the window (not in the drawer) your progress will be saved but you will be directed back to the dataset.

DataScribe has the following options on the form builder: Checkbox, Date, DateTime, Number, Radio, Select, Text, Textarea, and Time. Use the plus button to add a field of the desired kind to the form.

Once you have multiple fields in the form, you can rearrange them using the up and down arrows to shift a property up or down one level. You can also drag and drop fields into a new order using the three-lined handle on the far left of each field. You can have as many fields in a form as you need to capture the information in your dataset.

Form options:

Every field type in the form has the same four initial options:

• Field name (required): will display for transcribers
• Field description: will display for transcribers
• Field is primary: checkbox to set whether this is the primary field, ie the field by which the record is identified. A form can only have one primary field.
• Field is required: checkbox. Note that if a field is marked as required and is left blank by a transcriber, it will be marked as invlaid.

In addition, each field type has its own options, as described below.

Checkbox: create a checkbox option for the form. Options are:

• Checked value: set how this information exports. Defaults to 1.
• Unchecked value: set how this information exports. Defaults to 0.
• Checked by default: checkbox
• Checkbox label: set the label for the checkbox (separate from the field name)

Date: creates a date field using the Gregorian calendar to recreate ISO 8601 formatted dates. Transcribers will select from a dropdown for the year, month, day, and hour, minute, and second. Settings:

• Minimum year: type in the year. If left blank, this will default on transcription to 100 years before the present year.
• Maximum year: a free text entry box, for four digits. If left blank, this will default to the present year.
• Default year: set a default year for entry. Years appear as dropdowns; you can place this at the start or in the middle of your time span. Not required.
• Default month (optional): select from a dropdown of months.
• Default day (optional): select from a dropdown of days.

DateTime: creates a field with both date and time entry options. As with the Date option, all dates are in ISO 8601 format using the Gregorian calendar.

• Minimum year: type in the year. If left blank, this will default on transcription to 100 years before the present year.
• Maximum year: a free text entry box, for four digits. If left blank, this will default to the present year.
• Default year: set a default year for entry. Years appear as dropdowns; you can place this at the start or in the middle of your time span. Not required.
• Default month (optional): select from a dropdown of months.
• Default day (optional): select from a dropdown of days.
• Default hour: select from a dropdown, 1-23
• Default minute: select from a dropdown (00-59)
• Default second: select from a dropdown (00-59)

Number: this field only allows numeric data.

• Minimum value: sets the minimum allowed value for this field.
• Maximum value: sets the maximum allowed value for this field.
• Stepping interval: sets the interval to which any entered data must adhere.
• Placeholder: example value to display when the field is empty.
• Regex pattern: set a regular expression pattern which the field input must match.
• Default value: set a default value for the field
• Number input label: changes the label of the field.
• Datalist: displays recommended options.

Radio: creates a set of radio buttons with options for transcribers to choose from.

• Selection label
• Options: enter the information for the radio buttons. Each option must be on a new line in the text area.
• Default value: must match one of the options entered in the Options field.

Select: creates a dropdown with options for transcribers to choose from, created by the form builder.

• Options: enter the information for the dropdown. Each option must be on a new line in the text area.
• Default value: must match one of the above options.
• Select label: Changes the label for the dropdown (not the information which appears in the dropdown)

Text: a single line of text

• Minimum length: sets the minimum number of characters for this field.
• Maximum length: sets the maximum number of characters for this field.
• Placeholder: example value to display when the field is empty.
• Regex pattern: set a regular expression pattern which the field input must match.
• Default value: set a default value for the field
• Text input label: changes the display label for the field.
• Datalist: displays recommended options

Textarea: a field with space for multiple lines of text.

• Rows: set the number of visible rows for text
• Minimum length: sets the minimum number of characters for this field.
• Maximum length: sets the maximum number of characters for this field.
• Placeholder: example value to display when the field is empty.
• Regex pattern: set a regular expression pattern which the field input must match.
• Default value: set a default value for the field
• Textarea label: changes the display label for the field.

Time: creates a field which allows entry of time data (no date)

• Default hour: select from a dropdown, 1-23
• Default minute: select from a dropdown (00-59)
• Default second: select from a dropdown (00-59)

Below is an example of the form builder with an active form. The Text field at the top, "Account with" is required, as indicated with a *. It is also the primary field, as indicated with a flag icon.

Dataset actions

Dataset actions are available from the dataset page. The Browse and Edit buttons are always visible; other options are available from the "more actions" dropdown.

Browse toggle

Depending on what your current browse view is, this will either display as “Browse records” or “Browse items”. Use this to toggle between the two views.

Edit dataset

Editing the dataset presents you with a page two tabs: General and Form Builder.

The General options are the same as when you created the dataset, allowing you to edit the name, description, guidelines, and select which item set to use. The form builder is discussed above.

Sync dataset

Syncing updates the dataset with the information from the items in the item set. You should synchronize periodically, any time items are added or removed from the item set. Be aware that any items which have been removed from the Omeka S item set will be deleted from the DataScribe dataset, along with any records which have been created for those items.

Syncing does not send any information from DataScribe to Omeka S.

Validate dataset

Running the validation checks input in the records against settings you have in the form builder. For example, checking whether required fields are filled, whether numbers match the required format, and against any regular expressions you may have implemented.

Once you have run the validation, records with invalid values will be marked with a red alert, or exclamation point, on their browse rows. You will be able to filter by items and records “with values marked as invalid” in order to find them more quickly.

You should validate periodically, after fields are added to the form, after field validation rules change, and before exporting the dataset.

Export dataset

Export this dataset to a CSV file. Note that the file will only include records from approved items. Any invalid values will be given as “null” in the CSV. Running an export will overwrite any previous export file.

Exported files are stored in the Omeka install. They are made available in the Dataset metadata drawer, at the bottom of the page, under the heading “Export file.” You will need to download the file in order to open it or save it locally; the link functions as a direct link to the file, and will either download or open when you click on it depending on your browser settings.

An exported csv will have three columns in addition to the data created in the dataset form:

• DataScribe item number: item number in the DataScribe dataset. Does not correspond to Omeka S item number.
• DataScribe Record number: record number for that record in DataScribe. Record numbers are assigned across DataScribe as a whole, so an item may not have sequential record numbers.
• DataScribe Record Position: order of records within an item. If the insert before/after option has been used when editing or creating a record, the position number sequence may not match the record number sequence. For example, the records with positions 1, 2, 3, 4 could have record numbers 3, 4, 7, 5.

View dataset

To view a dataset, go to the project of which it is a part. The project has a table of all datasets, the dataset’s title, creator, and date created. Clicking on the ellipses (three dots) in a dataset’s row will show you the metadata for that dataset. Clicking its title takes you to the items in the dataset.