Tutorial

Jason Friedman edited this page Mar 16, 2016 · 8 revisions

0. Overview

Repeated Measures (RM) is open-source software for conducting laboratory experiments for measuring human responses, with specialized devices. The software is written in MATLAB, which is a high-level computer language with extensive support for numerical calculations. The heart of the RM software is the Psychophyiscs Toolbox (PTB) ( http://psychtoolbox.org ), a software package for vision research that establishes interfaces between Matlab and the computer hardware such as keyboards, mouse, monitor etc. The software provides core routines for designing customized experiments and storing experimental data for further analysis. Potential applications of the software include research into visual psychophysics and human motor control.

1. System requirements

The software package is developed and tested under Microsoft Windows XP and is compatible with the newer versions, Windows Vista and Windows 7. It also provides routines for recording data from keyboard and mouse under the Mac OS X. To run the software, it requires a recent version of 32-bit MATLAB (2009 onwards) and the Psychophysics Toolbox. Since the PTB package provides the physical hardware interface to RM, the graphic hardware requirements of the software is same as specified by the PTB. The additional requirements of the software are listed in Table 1.

Table 1: List of devices and their respective server and client name used in the RM software package. It also includes the additional software required for running experiments with the software. Some of these programs are publicly available from the link provided, others need to be purchased.

Device Name Server Name Client Name Additional Software
Keyboard keyboardserver keyboardclient -
Mouse mouseserver mouseclient -
Button box (connected to Measurement computing DAQ) DAQserver DAQclient MCC universal library http://www.mccdaq.com/products/software.aspx
Optotrak optotrakserver optotrakclient Matlab Optotrak toolbox http://webapp6.rrz.uni-hamburg.de/allpsy/vf/OptotrakToolbox and Optotrak APIs
Tablet tabletserver tabletclient Wacom bundle software http://www.wacom.com/en-us/support/product-support/drivers
Polhemus Fastrak fastrakserver fastrakclient -
Polhemus Liberty libertyserver libertyclient -
Ascension trakStar trakStarserver trakStarclient trakStar API and matlab demo
CyberGlove gloveserver gloveclient Virtual Hand SDK http://cyberglovesystems.com/support/downloads

2. Supported devices The RM software supports a wide range of motion capturing devices along with the conventional computer input devices. These are:

  1. Keyboard, any standard keyboard
  2. Mouse, any standard mouse
  3. Video capturing device such as webcam
  4. ButtonBox, response buttons connected to a Measurement Computing ( http://www.mccdaq.com ) digital I/O board (tested on PCI-DIO24 Series only, but should work with any Measurent computing device that is supported by the MCC universal library).
  5. Optotrak, an optical motion capture system manufactured by Northern Digital Inc. ( http://www.ndigital.com )
  6. Tablet, a WACOM graphics tablet ( http://www.wacom.com/ ).
  7. Fastrak or Liberty, magnetic motion tracking systems manufactured by Polhemus. ( http://www.polhemus.com )
  8. fastTrak, magnetic motion tracking systems manufactured by Ascension. ( http://http://www.ascension-tech.com/ )
  9. CyberGlove, a glove-based motion capture system manufactured by CyberGlove Systems. ( http://cyberglovesystems.com/ )
3. How to start

Start Matlab, and then go to the directory where you would like to store the experimental data. You can change to the directory by executing the following

cd C:\MyExperiments   % modify as needed 

To be able to run the experiments, you need to add appropriate directories to your search path so that necessary functions/files can be accessed. You can do so with the “addpath” function of MATLAB. For example, addpath ('C:\RepeatedMeasures\matlab'); The above statement will add the directory containing the “Repeated measures” software to the Matlab search path (the directory should be changed to the location where the software has been copied).

4. Architecture of the experiments

Each experiment is conducted on a server/client architecture. The client communicates with the server over a communication channel known as a socket. While running the experiment, the server maintains the physical connection to the device (e.g., through a serial port). It also samples data from the device and stores them into the disk upon receiving request from the client. On the other side, the client runs the actual experiment which includes sending appropriate requests to the server such as ‘initialize device’, ‘close device’, presenting stimulus, running trials under different conditions etc.

5. How to set up the server and the client

This section will describe how to use the RM software package to setup the server and the client to record data from a device.

5.1 Set up the server

To setup the server, you need to start matlab and run the respective server program. For example, if you want to record data with a cyber glove, then you have to run the program “gloverserver”. Each server program has at least three parameters to be set. The first two parameters are mandatory and the third one is optional. These are:

  1. Port number (usually a number greater than 3000).
  2. Sample Rate (specific to a particular device)
  3. Debug (set it to 1 if you want to open the debug mode)
For some devices, such as Fastrak, you need to specify additional parameters such as the number of receivers to be connected to the tracker.

Example: To set up the glove server, you have to execute the statements given in Table 2.

Table 2: List of routine calls that sets up the glove server

gs = gloveserver (3011, 150, 1)	 % set up the Glove server
listen(gs)	                 % keep listening to the socket
close(gs)	                 % close the server at the end of the experiment 

The first two statements start the glove server and keep listening to the socket for receiving request from the gloveclient program. The last statement closes the device at the end of the experiment upon receiving ‘closeDevice’ request from the client.

If your experiment involves multiple devices, you need to open separate matlab windows for running the corresponding server programs. Note: you can use “help” to get some help and see the usage. For example, type in ‘help gloveserver’ for Cyber Glove.

5.2 Set up the client

In this RM package, the ‘Client’ has been considered as a program that sends request for a specific task to the server over the communication channel and the server program usually acts on the received request. The set up process of a client is accomplished in following two steps.

5.2.1 How to create a client

To be able to send request over the socket, the client program needs to know the exact address (such as name, port number etc.) of the server. The client receives these details address of the server as well as the specific descriptions of the experiment during this phase of the experiment.

5.2.2 How to establish a connection

The server starts listening to the socket as soon as it has been set up. The communication channel between the server and the client is established when the client is successfully setup. Next, the client sends a request to the server to make a physical connection to the device. After executing the request, the server sends the connection status back to the client. It also flags appropriate message either success or error on the server side. On the other side, the client receives the connection status from the server and shows an error message in case of failure.

Example: To set up the glove client and get some data from the server, you can run the following statements. The data will be stored in a file (e.g. cg_testfile.csv) under the current folder.

Table 3: Block of statements. The first two statement sets up the client. The rest of them call appropriate routines to record data from the cyber glove and stored them in a file.

   gc = gloveclient('localhost',3011,[],1);     % create the glove client
   setupdevice(gc);                             % make a connection between server and client
   setupRecording(gc,'./testfile',10);          % client sends request for setting up the recording
   startRecording(gc);                          % request for starting the data recording
   getsample(gc);                               % client requests for a sample of data from device
   pause (5);                                                       
   stopRecording(gc);                           % request for stopping the data recording
   saveFile(gc);                                % the client asks the server to save the file
   closedevice(gc);                             % request for closing and deleting the connection 

The initial statement creates a glove client with the specific server details. In the function call, the first two parameters represent the server name and the port number respectively. The third parameter has been left empty in the above example to make it simple. Nevertheless, the experiment details such as type of stimuli, individual trial details etc. are passed through this parameter. The last parameter is set to 1, which opens the client in the debug mode.

The second statement establishes the physical connection between the server program and the cyber glove. You will also see an error message in case of failure.

The third statement sends a request to the server to setup the recording. In the calling, the second parameter holds the name of the file, that stores the data and the last parameter specifies the maximum amount of recording time in seconds. Though, you can stop the recording any time before this maximum. The rest of the statements are self-explanatory. Generally, the client sends the request to the server and the server executes it. For example, the client sends the ‘getsample’ request to the server and the server receives a sample of data (joint angles) from the glove and stores them in a temporary buffer. The server saves the data buffer into a file (permanent storage) on receiving the ‘savefile’ request. Note: Prior setting up the server/client program, you need to make sure that the directories containing the routines have already been added to the matlab search path. You can use the “addpath” function to do so.

6. How to run the experiments

The process of running an experiment starts with preparing a comprehensive specification of it. This includes detailed description of the stimuli used, number of trials in the experiment, type of user feedback to be shown at the end of each trial etc. In this section, we will describe how to setup and run an experiment using the RM package.

6.1 Experiment specification

With the RM package, the specification of each experiment is written in XML format and stored in a file known as protocol file. It contains complete description of the experiment, such as the device(s) to be used, port, types of stimulus, number of trials and specific description of each trial etc. You can either use an existing program to generate the file or manually write it. Table 4 lists sample XML code that describes an experiment to record data from a keyboard. As illustraed in Table 4, the XML protocol file has three major segments: SETUP, COMMON and TRIAL (one/more). We will give a brief description of these segments below.

1. SETUP: This segment describes all the parameters required to setup an individual experiment. It has the following two parts:

  1. General Setup: It includes the description of general experimental setup such as monitorWidth, viewing Distance, images (if there is any), feedback to the user etc.
  2. Server-specific Setup: The detailed address of the server (the recording device) is passed to the client through this segment of the XML code during an experiment. These parameters include server name, port number (same as specified in setting up the server) etc. In cases, where the server has additional parameters, for example, number of markers for an Optotrak, are also included in this segment.
  3. COMMON: This segment includes the description of common parameters of all the trials listed in the protocol file such as recordingTime, WaitTimeBefore, TargetType, textfeedback etc. For example, if you set the recordingTime to 4 seconds in COMMON segment, then all the trials in the experiment will record data for 4 seconds. However, there are experiments, in which each trial records data for a variable amount of time. In those cases, you can put the parameter ‘recordingTime’ in individual TRIAL segment of the protocol file. Therefore, the COMMON segment may include description of zero or more parameters those are common to all the trials in the experiment.
3. TRIAL: Each TRIAL segment corresponds to a single trial in the experiment. It lists the type of stimulus presented during the trial, target of the subject’s response, the filename to store the experimental data etc. This segment may also include any of the parameters listed in the COMMON segment if the parameters are different for each trial. Note: Please refer to the XML schema for detailed information of the tags used in the protocol file.

Table 4: A sample protocol file describing an experiment containing one trial only. It has also includes two more trials to start and finish the experiment. Note that while the XML can be written by "hand", we recommend writing the experiment as a matlab structure and then converting it to xml (see the sample experiments for examples of how to do this)

<?xml version="1.0" ?>
<experimentDescription>
   <setup>
      <monitorWidth> 70 </monitorWidth>
      <viewingDistance>  68 </viewingDistance>
      <xshift> 0  </xshift>
      <images> Female1.png  </images>
      <images> Female2.png  </images>
      <images>  Male1.png  </images>
      <images>  Male2.png  </images>
      <images>  cb.png  </images>
      <boxes>  [0,0.45,0.07,0.12;0.93,0.45,0.07,0.12]  </boxes>
      <labels>
         <location> [0.035,0.51] </location>
         <fontSize> 36 </fontSize>
         <text>  M  </text>
      </labels>
      <labels>
         <location>  [0.965,0.51]  </location>
         <fontSize>  36  </fontSize>
         <text>  F </text>
      </labels>
      <strings>
         <name> SUCCESS </name>
         <value> Correct </value>
         <name> WRONG </name>
         <value> Incorrect </value>
      </strings>
      <keyboard>
         <server> localhost  </server>
         <port>  3002 </port>
      </keyboard>
   </setup>
   <common>
      <recordingTime>  4 </recordingTime>
      <textFeedback> 1  </textFeedback>
      <waitTimeBefore> 2 </waitTimeBefore>
      <targetType> 
          <keyboard>
            <keytopress>
               z
            </keytopress>
            <keytopress>
               m
            </keytopress>
           </keyboard>
       </targetType>
      <starttrial> keyboard </starttrial>
   </common>
   <trial>
      <stimulus>  showText  </stimulus>
      <descriptionText>  Ready?  </descriptionText>
   </trial>
   <trial>
      <stimulus> images </stimulus>
      <stimuli> [1,1,18,0.01;5,19,102,0] </stimuli>
      <target> 2 </target>
      <filename> F1_1 </filename>
   </trial>
   <trial>
      <stimulus>
         showText
      </stimulus>
      <descriptionText>
         Experiment finished!
      </descriptionText>
   </trial>
</experimentDescription>

6.2 Set up the client for an experiment

As described in section 5.2, a client is set up with server details before recording data from the device. While designing the experiment with this RM package, the setup segment of the protocol file contains the description of the server, such as the list of devices used in the experiment, their corresponding name and port number etc. At the beginning an experiment, the client receives these server details through the protocol file and establishes communication channel (socket) with each of the servers.

6.3 Running the actual experiment

The Psychophysics Toolbox plays a major role in conducting the actual experiments. It establishes the interface between the display device and the RM package by calling the ‘SCREEN’ function at the beginning of each experiment. Generally, each experiment is composed of one or multiple trials as specified in the protocol file. The trials can differ from each other in different aspects, such as types of stimulus to be presented, feedback to be shown etc. Nevertheless, each trial in the experiment has a general framework under the RM environment as shown in Figure 3. The framework depicts that each trial goes through four different stages to complete it. The stages are: setting up the trial, start the trial (Wait for the keypress), present stimuli to record data and show feedback on subject’s response. The different components of the framework are discussed in the following sections.

6.3.1 How to set up the Trial

Each individual trial starts with setting up all the required parameters for running the experiment, such as checking the availability of the stimulus, loading the specified stimulus etc. For example, in case of image stimulus, it looks for the specified image files in the ‘Stimulus’ folder and prepare it for display by adding appropriate noise. Likewise, when ‘vr’ is used as the stimulus, it looks for the offset files (if time shift rendering is in use) in the ‘stimulus’ folder and load them into the memory. The other major task of this step is to set up the devices and prepares them for recording data. The client program sends ‘setUp recording’ request to the server and the server calls the appropriate routine to setup the recording.

6.3.1.1 Supported stimulus types

A Stimulus is an event in the environment that influences the behaviour. The RM package supports a wide range of stimuli types. These are:

  1. Image: present images on the screen; e.g. faces
  2. Text: show text on the screen; for instance, if you present either ‘ <>’ as stimulus, the subject has to respond with the direction.
  3. Video: play video on the screen; e.g. human movement
  4. Dots: show some moving dots on the screen;
On top of these stimulus types, the toolbox also supports two advanced stimuli. These are Quest staircase and VR.
  1. Quest staircase: This uses the implementation of the Quest (Watson & Pelli, 1983) staircase method provided with the Psychophysics toolbox, in order to find the psychometric function for certain types of stimuli (i.e., which stimulus levels / noise lead to which accuracy levels). This has been implemented for two cases – RDK (the coherence is altered), and with images (where the amount of noise is altered).
  2. VR: This stimulus type is only supported to record data from the cyber glove and the fastrak. It renders a virtual human hand on the screen either in real time or with a shift in position, orientation and time or any combination of them. In the real time rendering, the virtual hand on the screen is synchronized with the subject’s hand and replicates the movement instantaneously. In case of time shift rendering, the on-screen virtual hand presents a time shifted state of the human hand. For example, if the current time is t and the timeshift is ts, then the virtual hand is rendered with positions, orientations and angles at time (t- ts).
6.3.1.2 How to start the trial

The RM package supports several alternative input devices for receiving instruction from the subject to mark the start of an individual trial. Each of the alternatives has their own code in the RM environment as illustrated in Table 5.

Table 5: Start trial. The table shows the alternative options and their respective code to start a trial.

Code Description
button wait for button press (using a button box connected to a DAQ card)
dontWait auto start of a trial (no wait)
keyboard wait for keyboard space bar (although this can be used when recording from another device)
mouseClick Press the mouse button
mouseInBox Mouse is in a box
pedal wait for right foot pedal press (right only), of the type that come with USB steering wheels
stylusButton wait for button on tablet stylus
targetInSpace Move the markers into a certain area of space
trigger Start when an external trigger is received (e.g. from another device)
Besides this, the package also provides options for quitting a trial and jumps to the next or previous trial. If you press ‘q’, it will quit both the trial and the experiment itself. In order to jump to the next trial, press ‘n’, and press ‘p’ for previous trial.

6.3.1.3 How to record data

This is the most operational phase of the trial. During this phase, the client program presents the stimulus and the server records the data. At the starting of this phase, the client program sets up the triggers and beeps in cases where they are needed. Then it sends a ‘start recording’ request to the server. On the other side, the server executes the appropriate routine to start recording and samples data from the device and stores it in memory. Meanwhile, the client presents stimuli on the screen and optionally can get samples from the server (if realtime feedback is required). This process continues until the trial is finished. The client program concludes the trial by sending ‘stop recording’ request over the socket. The server receives the request and stops recording data. Upon receiving the ‘savefile’ request from the client, it saves the recorded data to a file.

6.3.1.4 Show position

The RM package provides routines for running experiments in which previously recorded data can be used to show earlier positions. For example, while recording data from any standard mouse, it stores the x and y coordinates values of the cursor on the screen. With the use of show position routines, we will be able to set the mouse on any previously recorded position. Another more effective example of this routine is the ‘vr’ stimuli. This stimulus usually renders the human hand on the screen using the previously recorded data from the devices. The joint angles of the human hand are set according to the data recorded from the cyber glove and the position and the orientation of the posture is set with the data recorded from a tracking device such as Fastrak.

6.3.1.5 How to show feedback

The RM package supports two types of feedback to be shown to the subject. The first form of feedback is designated to work with some devices such as keyboard and provides feedback on subject’s reaction time. For instance, while recording data from a keyboard or a button box, the subject may response either too quickly or too slowly. The package will provide feedback with appropriate message to warn the subject regarding his/her respond time. The other form of feedback is associated with some experiments and show feedback to the subjects in response of their answers at the end of the trial. For example, in an experiment where you present a human face as stimulus during the trial and the subject has to classify whether it is a male or female. After recording the subject’s answer from the input device, you may show feedback to the subject whether the answer is ‘Correct’ or ‘Incorrect’. The various types of feedback supported by the RM package and their short description is illustrated in Table 6.

Table 6: List of feedback supported by the RM package while running an experiment

Feedback Text Feedback type Description
SUCCESS in response to subject’s answer The subject responds correctly. You can customize this feedback using the <value></value> tag in the protocol file, e.g. in the sample XML code in Table 4, it specifies to show the text ‘Correct’ as feedback in cases of SUCCESS.
WRONG in response to subject’s answer The subject enters incorrect answer. You can also customize this feedback using the <value></value> tag in the protocol file, e.g. in the sample XML code in Table 4, it specifies to show the text ‘Incorrect’ as feedback in cases of WRONG.
TOO_EARLY in response to subject’s reaction time The subject has responded too early
TOO_LATE in response to subject’s reaction time The subject has taken too much time to respond to the stimuli.

6.4 How to close the devices

On completion of the trials, the client sends ‘closedevice’ request over the socket. The server receives the request and systematically closes all the active devices and the socket. It also shows the “Success” message on the server side. This concludes the experiment and the data files are stored in a folder named ‘Output’. The whole process also generates a log file in the same directory which records the step by step status of the experiment.

7. How to run a first experiment

In the sampleexperiments directory, there are several examples of sample experiments which can be run.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.