-
Notifications
You must be signed in to change notification settings - Fork 12
Tutorial
0. Overview
Repeated Measures (RM) is open-source software for conducting laboratory experiments for measuring human responses, with specialized devices. The software is written in MATLAB, which is a high-level computer language with extensive support for numerical calculations. The heart of the RM software is the Psychophyiscs Toolbox (PTB) ( http://psychtoolbox.org ), a software package for vision research that establishes interfaces between Matlab and the computer hardware such as keyboards, mouse, monitor etc. The software provides core routines for designing customized experiments and storing experimental data for further analysis. Potential applications of the software include research into visual psychophysics and human motor control.
1. System requirements
The software package is developed and tested under Microsoft Windows XP and is compatible with the newer versions, Windows Vista and Windows 7. It also provides routines for recording data from keyboard and mouse under the Mac OS X. To run the software, it requires a recent version of 32-bit MATLAB (2009 onwards) and the Psychophysics Toolbox. Since the PTB package provides the physical hardware interface to RM, the graphic hardware requirements of the software is same as specified by the PTB. The additional requirements of the software are listed in Table 1.
Table 1: List of devices and their respective server and client name used in the RM software package. It also includes the additional software required for running experiments with the software. Some of these programs are publicly available from the link provided, others need to be purchased.
| Device Name | Server Name | Client Name | Additional Software |
| Keyboard | keyboardserver | keyboardclient | - |
| Mouse | mouseserver | mouseclient | - |
| Button box (connected to Measurement computing DAQ) | DAQserver | DAQclient | MCC universal library http://www.mccdaq.com/products/software.aspx |
| Optotrak | optotrakserver | optotrakclient | Matlab Optotrak toolbox http://webapp6.rrz.uni-hamburg.de/allpsy/vf/OptotrakToolbox and Optotrak APIs |
| Tablet | tabletserver | tabletclient | Wacom bundle software http://www.wacom.com/en-us/support/product-support/drivers |
| Polhemus Fastrak | fastrakserver | fastrakclient | - |
| Polhemus Liberty | libertyserver | libertyclient | - |
| Ascension trakStar | trakStarserver | trakStarclient | trakStar API and matlab demo |
| CyberGlove | gloveserver | gloveclient | Virtual Hand SDK http://cyberglovesystems.com/support/downloads |
2. Supported devices The RM software supports a wide range of motion capturing devices along with the conventional computer input devices. These are:
- Keyboard, any standard keyboard
- Mouse, any standard mouse
- Video capturing device such as webcam
- ButtonBox, response buttons connected to a Measurement Computing ( http://www.mccdaq.com ) digital I/O board (tested on PCI-DIO24 Series only, but should work with any Measurent computing device that is supported by the MCC universal library).
- Optotrak, an optical motion capture system manufactured by Northern Digital Inc. ( http://www.ndigital.com )
- Tablet, a WACOM graphics tablet ( http://www.wacom.com/ ).
- Fastrak or Liberty, magnetic motion tracking systems manufactured by Polhemus. ( http://www.polhemus.com )
- fastTrak, magnetic motion tracking systems manufactured by Ascension. ( http://http://www.ascension-tech.com/ )
- CyberGlove, a glove-based motion capture system manufactured by CyberGlove Systems. ( http://cyberglovesystems.com/ )
Start Matlab, and then go to the directory where you would like to store the experimental data. You can change to the directory by executing the following
cd C:\MyExperiments % modify as needed To be able to run the experiments, you need to add appropriate directories to your search path so that necessary functions/files can be accessed. You can do so with the “addpath” function of MATLAB. For example, addpath ('C:\RepeatedMeasures\matlab'); The above statement will add the directory containing the “Repeated measures” software to the Matlab search path (the directory should be changed to the location where the software has been copied).
4. Architecture of the experiments
Each experiment is conducted on a server/client architecture. The client communicates with the server over a communication channel known as a socket. While running the experiment, the server maintains the physical connection to the device (e.g., through a serial port). It also samples data from the device and stores them into the disk upon receiving request from the client. On the other side, the client runs the actual experiment which includes sending appropriate requests to the server such as ‘initialize device’, ‘close device’, presenting stimulus, running trials under different conditions etc.
5. How to set up the server and the client
This section will describe how to use the RM software package to setup the server and the client to record data from a device.
5.1 Set up the server
To setup the server, you need to start matlab and run the respective server program. For example, if you want to record data with a cyber glove, then you have to run the program “gloverserver”. Each server program has at least three parameters to be set. The first two parameters are mandatory and the third one is optional. These are:
- Port number (usually a number greater than 3000).
- Sample Rate (specific to a particular device)
- Debug (set it to 1 if you want to open the debug mode)
Example: To set up the glove server, you have to execute the statements given in Table 2.
Table 2: List of routine calls that sets up the glove server
gs = gloveserver (3011, 150, 1) % set up the Glove server
listen(gs) % keep listening to the socket
close(gs) % close the server at the end of the experiment The first two statements start the glove server and keep listening to the socket for receiving request from the gloveclient program. The last statement closes the device at the end of the experiment upon receiving ‘closeDevice’ request from the client.
If your experiment involves multiple devices, you need to open separate matlab windows for running the corresponding server programs. Note: you can use “help” to get some help and see the usage. For example, type in ‘help gloveserver’ for Cyber Glove.
5.2 Set up the client
In this RM package, the ‘Client’ has been considered as a program that sends request for a specific task to the server over the communication channel and the server program usually acts on the received request. The set up process of a client is accomplished in following two steps.
5.2.1 How to create a client
To be able to send request over the socket, the client program needs to know the exact address (such as name, port number etc.) of the server. The client receives these details address of the server as well as the specific descriptions of the experiment during this phase of the experiment.
5.2.2 How to establish a connection
The server starts listening to the socket as soon as it has been set up. The communication channel between the server and the client is established when the client is successfully setup. Next, the client sends a request to the server to make a physical connection to the device. After executing the request, the server sends the connection status back to the client. It also flags appropriate message either success or error on the server side. On the other side, the client receives the connection status from the server and shows an error message in case of failure.
Example: To set up the glove client and get some data from the server, you can run the following statements. The data will be stored in a file (e.g. cg_testfile.csv) under the current folder.
Table 3: Block of statements. The first two statement sets up the client. The rest of them call appropriate routines to record data from the cyber glove and stored them in a file.
gc = gloveclient('localhost',3011,[],1); % create the glove client
setupdevice(gc); % make a connection between server and client
setupRecording(gc,'./testfile',10); % client sends request for setting up the recording
startRecording(gc); % request for starting the data recording
getsample(gc); % client requests for a sample of data from device
pause (5);
stopRecording(gc); % request for stopping the data recording
saveFile(gc); % the client asks the server to save the file
closedevice(gc); % request for closing and deleting the connection The initial statement creates a glove client with the specific server details. In the function call, the first two parameters represent the server name and the port number respectively. The third parameter has been left empty in the above example to make it simple. Nevertheless, the experiment details such as type of stimuli, individual trial details etc. are passed through this parameter. The last parameter is set to 1, which opens the client in the debug mode.
The second statement establishes the physical connection between the server program and the cyber glove. You will also see an error message in case of failure.
The third statement sends a request to the server to setup the recording. In the calling, the second parameter holds the name of the file, that stores the data and the last parameter specifies the maximum amount of recording time in seconds. Though, you can stop the recording any time before this maximum. The rest of the statements are self-explanatory. Generally, the client sends the request to the server and the server executes it. For example, the client sends the ‘getsample’ request to the server and the server receives a sample of data (joint angles) from the glove and stores them in a temporary buffer. The server saves the data buffer into a file (permanent storage) on receiving the ‘savefile’ request. Note: Prior setting up the server/client program, you need to make sure that the directories containing the routines have already been added to the matlab search path. You can use the “addpath” function to do so.
6. How to run the experiments
The process of running an experiment starts with preparing a comprehensive specification of it. This includes detailed description of the stimuli used, number of trials in the experiment, type of user feedback to be shown at the end of each trial etc. In this section, we will describe how to setup and run an experiment using the RM package.
6.1 Experiment specification
With the RM package, the specification of each experiment is written in XML format and stored in a file known as protocol file. It contains complete description of the experiment, such as the device(s) to be used, port, types of stimulus, number of trials and specific description of each trial etc. You can either use an existing program to generate the file or manually write it. Table 4 lists sample XML code that describes an experiment to record data from a keyboard. As illustraed in Table 4, the XML protocol file has three major segments: SETUP, COMMON and TRIAL (one/more). We will give a brief description of these segments below.
1. SETUP: This segment describes all the parameters required to setup an individual experiment. It has the following two parts:
- General Setup: It includes the description of general experimental setup such as monitorWidth, viewing Distance, images (if there is any), feedback to the user etc.
- Server-specific Setup: The detailed address of the server (the recording device) is passed to the client through this segment of the XML code during an experiment. These parameters include server name, port number (same as specified in setting up the server) etc. In cases, where the server has additional parameters, for example, number of markers for an Optotrak, are also included in this segment.
- COMMON: This segment includes the description of common parameters of all the trials listed in the protocol file such as recordingTime, WaitTimeBefore, TargetType, textfeedback etc. For example, if you set the recordingTime to 4 seconds in COMMON segment, then all the trials in the experiment will record data for 4 seconds. However, there are experiments, in which each trial records data for a variable amount of time. In those cases, you can put the parameter ‘recordingTime’ in individual TRIAL segment of the protocol file. Therefore, the COMMON segment may include description of zero or more parameters those are common to all the trials in the experiment.
Table 4: A sample protocol file describing an experiment containing one trial only. It has also includes two more trials to start and finish the experiment. Note that while the XML can be written by "hand", we recommend writing the experiment as a matlab structure and then converting it to xml (see the sample experiments for examples of how to do this)
<?xml version="1.0" ?>
<experimentDescription>
<setup>
<monitorWidth> 70 </monitorWidth>
<viewingDistance> 68 </viewingDistance>
<xshift> 0 </xshift>
<images> Female1.png </images>
<images> Female2.png </images>
<images> Male1.png </images>
<images> Male2.png </images>
<images> cb.png </images>
<boxes> [0,0.45,0.07,0.12;0.93,0.45,0.07,0.12] </boxes>
<labels>
<location> [0.035,0.51] </location>
<fontSize> 36 </fontSize>
<text> M </text>
</labels>
<labels>
<location> [0.965,0.51] </location>
<fontSize> 36 </fontSize>
<text> F </text>
</labels>
<strings>
<name> SUCCESS </name>
<value> Correct </value>
<name> WRONG </name>
<value> Incorrect </value>
</strings>
<keyboard>
<server> localhost </server>
<port> 3002 </port>
</keyboard>
</setup>
<common>
<recordingTime> 4 </recordingTime>
<textFeedback> 1 </textFeedback>
<waitTimeBefore> 2 </waitTimeBefore>
<targetType>
<keyboard>
<keytopress>
z
</keytopress>
<keytopress>
m
</keytopress>
</keyboard>
</targetType>
<starttrial> keyboard </starttrial>
</common>
<trial>
<stimulus> showText </stimulus>
<descriptionText> Ready? </descriptionText>
</trial>
<trial>
<stimulus> images </stimulus>
<stimuli> [1,1,18,0.01;5,19,102,0] </stimuli>
<target> 2 </target>
<filename> F1_1 </filename>
</trial>
<trial>
<stimulus>
showText
</stimulus>
<descriptionText>
Experiment finished!
</descriptionText>
</trial>
</experimentDescription>6.2 Set up the client for an experiment
As described in section 5.2, a client is set up with server details before recording data from the device. While designing the experiment with this RM package, the setup segment of the protocol file contains the description of the server, such as the list of devices used in the experiment, their corresponding name and port number etc. At the beginning an experiment, the client receives these server details through the protocol file and establishes communication channel (socket) with each of the servers.
6.3 Running the actual experiment
The Psychophysics Toolbox plays a major role in conducting the actual experiments. It establishes the interface between the display device and the RM package by calling the ‘SCREEN’ function at the beginning of each experiment. Generally, each experiment is composed of one or multiple trials as specified in the protocol file. The trials can differ from each other in different aspects, such as types of stimulus to be presented, feedback to be shown etc. Nevertheless, each trial in the experiment has a general framework under the RM environment as shown in Figure 3. The framework depicts that each trial goes through four different stages to complete it. The stages are: setting up the trial, start the trial (Wait for the keypress), present stimuli to record data and show feedback on subject’s response. The different components of the framework are discussed in the following sections.
6.3.1 How to set up the Trial
Each individual trial starts with setting up all the required parameters for running the experiment, such as checking the availability of the stimulus, loading the specified stimulus etc. For example, in case of image stimulus, it looks for the specified image files in the ‘Stimulus’ folder and prepare it for display by adding appropriate noise. Likewise, when ‘vr’ is used as the stimulus, it looks for the offset files (if time shift rendering is in use) in the ‘stimulus’ folder and load them into the memory. The other major task of this step is to set up the devices and prepares them for recording data. The client program sends ‘setUp recording’ request to the server and the server calls the appropriate routine to setup the recording.
6.3.1.1 Supported stimulus types
A Stimulus is an event in the environment that influences the behaviour. The RM package supports a wide range of stimuli types. These are:
- Image: present images on the screen; e.g. faces
- Text: show text on the screen; for instance, if you present either ‘ <<’ or ‘ >>’ as stimulus, the subject has to respond with the direction.
- Video: play video on the screen; e.g. human movement
- Dots: show some moving dots on the screen;
- Quest staircase: This uses the implementation of the Quest (Watson & Pelli, 1983) staircase method provided with the Psychophysics toolbox, in order to find the psychometric function for certain types of stimuli (i.e., which stimulus levels / noise lead to which accuracy levels). This has been implemented for two cases – RDK (the coherence is altered), and with images (where the amount of noise is altered).
- VR: This stimulus type is only supported to record data from the cyber glove and the fastrak. It renders a virtual human hand on the screen either in real time or with a shift in position, orientation and time or any combination of them. In the real time rendering, the virtual hand on the screen is synchronized with the subject’s hand and replicates the movement instantaneously. In case of time shift rendering, the on-screen virtual hand presents a time shifted state of the human hand. For example, if the current time is t and the timeshift is ts, then the virtual hand is rendered with positions, orientations and angles at time (t- ts).
The RM package supports several alternative input devices for receiving instruction from the subject to mark the start of an individual trial. Each of the alternatives has their own code in the RM environment as illustrated in Table 5.
Table 5: Start trial. The table shows the alternative options and their respective code to start a trial.
| Code | Description |
| button | wait for button press (using a button box connected to a DAQ card) |
| dontWait | auto start of a trial (no wait) |
| keyboard | wait for keyboard space bar (although this can be used when recording from another device) |
| mouseClick | Press the mouse button |
| mouseInBox | Mouse is in a box |
| pedal | wait for right foot pedal press (right only), of the type that come with USB steering wheels |
| stylusButton | wait for button on tablet stylus |
| targetInSpace | Move the markers into a certain area of space |
| trigger | Start when an external trigger is received (e.g. from another device) |
6.3.1.3 How to record data
This is the most operational phase of the trial. During this phase, the client program presents the stimulus and the server records the data. At the starting of this phase, the client program sets up the triggers and beeps in cases where they are needed. Then it sends a ‘start recording’ request to the server. On the other side, the server executes the appropriate routine to start recording and samples data from the device and stores it in memory. Meanwhile, the client presents stimuli on the screen and optionally can get samples from the server (if realtime feedback is required). This process continues until the trial is finished. The client program concludes the trial by sending ‘stop recording’ request over the socket. The server receives the request and stops recording data. Upon receiving the ‘savefile’ request from the client, it saves the recorded data to a file.
6.3.1.4 Show position
The RM package provides routines for running experiments in which previously recorded data can be used to show earlier positions. For example, while recording data from any standard mouse, it stores the x and y coordinates values of the cursor on the screen. With the use of show position routines, we will be able to set the mouse on any previously recorded position. Another more effective example of this routine is the ‘vr’ stimuli. This stimulus usually renders the human hand on the screen using the previously recorded data from the devices. The joint angles of the human hand are set according to the data recorded from the cyber glove and the position and the orientation of the posture is set with the data recorded from a tracking device such as Fastrak.
6.3.1.5 How to show feedback
The RM package supports two types of feedback to be shown to the subject. The first form of feedback is designated to work with some devices such as keyboard and provides feedback on subject’s reaction time. For instance, while recording data from a keyboard or a button box, the subject may response either too quickly or too slowly. The package will provide feedback with appropriate message to warn the subject regarding his/her respond time. The other form of feedback is associated with some experiments and show feedback to the subjects in response of their answers at the end of the trial. For example, in an experiment where you present a human face as stimulus during the trial and the subject has to classify whether it is a male or female. After recording the subject’s answer from the input device, you may show feedback to the subject whether the answer is ‘Correct’ or ‘Incorrect’. The various types of feedback supported by the RM package and their short description is illustrated in Table 6.
Table 6: List of feedback supported by the RM package while running an experiment
| Feedback Text | Feedback type | Description |
| SUCCESS | in response to subject’s answer | The subject responds correctly. You can customize this feedback using the <value></value> tag in the protocol file, e.g. in the sample XML code in Table 4, it specifies to show the text ‘Correct’ as feedback in cases of SUCCESS. |
| WRONG | in response to subject’s answer | The subject enters incorrect answer. You can also customize this feedback using the <value></value> tag in the protocol file, e.g. in the sample XML code in Table 4, it specifies to show the text ‘Incorrect’ as feedback in cases of WRONG. |
| TOO_EARLY | in response to subject’s reaction time | The subject has responded too early |
| TOO_LATE | in response to subject’s reaction time | The subject has taken too much time to respond to the stimuli. |
6.4 How to close the devices
On completion of the trials, the client sends ‘closedevice’ request over the socket. The server receives the request and systematically closes all the active devices and the socket. It also shows the “Success” message on the server side. This concludes the experiment and the data files are stored in a folder named ‘Output’. The whole process also generates a log file in the same directory which records the step by step status of the experiment.
7. How to run a first experiment
In the sampleexperiments directory, there are several examples of sample experiments which can be run.