Creating an Experiment

This section will provide a simple tutorial on making a simple reach task with the framework.

Precursor

The task is described as follows:

1. The user will place their cursor at a starting location.
2. A target will appear some distance away from the starting location.
3. The user will attempt to reach for this target. We will only give the user 1 second from when the user leaves the starting location.
4. Once 1 second has expired, we will freeze their cursor in place and show a path of their travel
5. If the user ends up reaching the target, the target will change from white to green. Red otherwise.

• We will add the ability of perturbations for an added challenge

By the end of this tutorial you will understand:

• How to write a JSON file that is compatible with the framework
• How to write a task object that will be instantiated when the trial begins
• How to handle the logic flow of a task
• How to end the trial and perform data collection

Creating a JSON

This section will go over how to create a JSON file and some reserved keywords used in our framework.

A GUI for creating the required JSON files is available here.

Our framework uses the UXF framework, which uses JSON files to configure experiments. These are stored in StreamingAssets and usually have the following structure:

{
 "experiment_mode" : "experiment_name",
 "optional_params": [ ],
 "per_block_parameter" : ["parameter1", "parameter2", ...]
 "per_block_second_parameter" : [1, 3, ...]
 "per_block_n" : [3, 3, ...]
 "per_block_targetListToUse" : ["property_1", "property_2", ...]
 "property_1" : [1, 2, 3, 4, ...]
 "property_2" : [5, 2]
 ...
}

The framework as well as UXF allows for any key-value pairs to be added to the JSON. However, there are a few reserved keywords that we use in the framework.

Key	Value
experiment_mode	String
per_block_n	List of Integer
per_block_hand	One of "r" or "l"

experiment_mode Is a single string that is used to tell the framework which experiment class to instantiate. We will see later how it works in ExperimentController.cs

per_block_n Is used to denote how many trials per block. In the JSON, we define the number of blocks by defining how many elements are in the array in per_bock_n. For example, an array with 10 elements under the key per_block_n indicates 10 blocks. Each block in the experiment may have properties specific to that block. To define a particular property for a particular block, we use the prefix per_ to denote a property we want to change depending on which block it is.

per_block_hand Is used to denote which hand to enable for VR-based experiments. Even if your experiment does not use this property, it must be set regardless or the application will not work. This may change (04/21/21)

You can also specify any key-value pair with any data type as the value. The JSON supports all of the value types you may possibility need (bool, int, float, string, etc)

Takeaways:

• experiment_mode is reserved and consists of a single string. We use this to determine which task to instantiate in the project.
• per_block_n tells the application how many trials are in each block, where the size of the array is the total number of blocks.
• Any property with per_ as the prefix must be the same length of per_block_n
• You are free to add other pairs to the JSON as needed for the task to be implemented.

Our JSON for our sample task will be as follows:

{
 "experiment_mode" : "sample_reach",
 "per_block_n" : [2, 2, 2],
 "per_block_rotation" : [0, 10, 20],
 "per_block_targetListToUse" : ["tl1", "tl2", "tl3"],
 "tl1" : [5, 10],
 "tl2" : [1, 2],
 "tl3" : [7]
}

Creating your first task class

All of the tasks supported by the framework inherits from an abstract class BaseTask. For this tutorial, we will name our class SampleReachTask.cs

public class SampleReachTask : BaseTask
{
 public void Init(List<float> angles) {
  // Initialization goes here
  Setup();
 }

 public void Update() {
  // Main logic handling of your task goes here
 }

 public override bool IncrementStep() {
  // Any logic you want to execute the moment we move from one step to the next
 }

 protected override void Setup() {
  // Setup prior to the execution of the task. Called from Init()
 }
 
 protected override void OnDestroy() {
  // Any objects/prefabs you may have instantiated for the lifetime of a TRIAL must be DESTROYED here
 }

 public override void LogParameters() {
  // Data logging function that is called from ExperimentController.cs when you call ExperimentController.instance().EndAndPrepare().
 }
}

Once we created our task class, we need to link it's creation to ExperimentController.cs

    public void BeginTrialSteps(Trial trial)
    {
        // psuedo-random generation using values from per_block_targetListToUse

        String per_block_type = trial.settings.GetString("per_block_type");
        
        if (per_block_type == "instruction")
        {
            // instruction initialization code ...
            return;
        }

        switch (Session.settings.GetString("experiment_mode"))
        {
            case "target":
                // initialization code ...
                break;
            case "pinball_vr":
            case "pinball":
                // initialization code ...
                break;
            case "tool":
                break;
            case "sample_reach":
                CurrentTask = gameObject.AddComponent<SampleReachTask>();
                ((SampleReachTask)CurrentTask).Init(trial, angles);
                break;
            default:
                Debug.LogWarning("Experiment Type not implemented: " +
                                    Session.settings.GetString("experiment_mode"));
                trial.End();
                break;
        }
    }

In ExperimentController.cs we have a switch statement in the function BeginTrialSteps which will instantiate our task class. Note that we named our experiment sample_reach in the experiment_mode field of the JSON. This is the value we use here. Once the task is added as a component (instantiated), we run Init which we supply the following:

trial : A Trial object provided by UXF that allows us to access any properties defined in the JSON angles : A list of floats that is psuedo-randomly shuffled supplied by per_block_targetListToUse. Based on the current block in the experiment, it will grab the list of floats from the JSON. For example, if the value of per_block_targetListToUse is currently tl2 it will find the key tl2 in the JSON and grab the associated list of floats, shuffle it, and be placed in the variable angles.

Best Practices

Steps

Each experiment is broken down into steps. For example, in pinball:

Step 0. Checks for user input to launch to ball.

Step 1. Ball in motion. Tracks ball, checking if the ball hits the target or stops.

Step 2. Ball is stopped. Pause the screen, update the score, prepare for the next trial.

By using Switch statements (switch (currentStep)) in conjunction with the IncrementStep() function, all experiments can be more consistent.

Tracking

Certain game objects and values need to be tracked in each experiment. All tracked values are exported to a CSV file at the end of the experiment.

The 'key' of each tracked value becomes the name of a column in the CSV.

AddTrackedPosition/Rotation

ExperimentController.AddTrackedPosition() and AddTrackedRotation() tracks the position/rotation of the provided game object every fixed update.

When an object is being tracked, the time is also collected every fixed update. These functions should be called in Setup() so that the timestamps are consistent with all tracked objects.

These methods don't need to be called each frame- by calling it once, the added object is added to a list in ExperimentController, in which each tracked object's position/rotation is recorded every FixedUpdate.

LogObjectPosition

ExperimentController.LogObjectPosition() adds a Vector3 to the CSV. This method splits the x, y, and z values into separate keys.

'key', 0,1,2 turns into:

key_x, 0, key_y, 1, key_z, 2

This is useful for:

1. Collecting the position of an object that doesn't move, such as the home position.

2. Collecting Vector3s that aren't positions, such as velocity vectors.

Collecting other results

To add individual strings, ints, or floats to the CSV, use ExperimentController.Session.CurrentTrial.result["key"] = value

WIP