1) AOI Analysis
MAGiC automatically annotates gaze data of each image-frame in a scene video to identify whether a participant was looking at the interlocutor’s face (viz. in), or he/she was looking away the interlocutor’s face (viz. out).MAGiC offers interfaces and options to monitor and/or to measure the success ratio of outcomes(see Figure-1). If the success-ratio is not sufficient enough to meet the experiment-dependent considerations, user can either re-track with trained detector or label AOIs manually.
Figure-1: MAGiC automatically annotates each image-frame to identify whether a participant was looking at the interlocutor’s face (viz. in), or he/she was looking away the interlocutor’s face (viz. out). Moreover, the relative positions of gaze data to the face location is stored. If the gaze data was inside the detected face, ‘e’ character is assigned as an AOI-label. The face region is divided into 3 areas: namely mouth, nose and eyes. Then, which of these three areas corresponds to the gaze data is evaluated. When the gaze location is outside the face boundary, 1 of 8 character values, namely, ‘a’ ‘b’ ‘c’ ‘d’ ‘f’ ‘g ‘h’ ‘i’, is assigned.
(Please see the Walkthrough pages in an application to provide brief description of a function and step-by-step guidance on how to conduct analysis.) Each function produces the output files that might be needed for the next step. In this way, the user does not have to perform all the functions in a single session. Also, user will have the opportunity to examine the outputs generated in each step and accordingly to make further analysis. Such approach also increases the usage diversity. Accordion panels are arranged in accordance with the procedure sequence as follows:
-
Face Tracking
- Face Tracking with default detector <br>to track face with the default face detector that is employed from the dlib toolkit
-
Face tracking with trained detector
- Export Images <br>to export frames of the given video file
- Training <br>to train face detector by specifying face area in a pre-selected group of frames
- Tracking <br>to track face in the selected video with the trained face detector<br>
-
AOI Analysis
- Pre-process Gaze Data <br>to fill missing data in the eye tracker generated raw-data file
- Face as AOI<br> to define the boundary of face and then to test whether the raw gaze-data is within the face(see Figure-1)<br>
-
Review Outcomes
- Visualize Tracking<br> to watch the video file overlaid by facial landmarks, AOI-label and gaze point location
- Find AOIs Detection Ratio<br> to deliver the number and percentage of labelled frames
- Label AOIs Manually<br> to label AOIs manually by using numpad (see Figure-2)
- Re-analyze AOIs<br> to merge automatically extracted and manually labelled AOIs
Figure-2: AOIs of frames can be labelled by using numeric keypad. For instance, pressing the button 5 will assign “e” and pressing enter button will skip the current frame without annotating. Moreover, UI allows to navigate between previous frames by clicking left-arrow key.
Since each function produces the output files that would be needed for the next step, outputs of the previous function would be an input for the next function. Those files will be given under the "What are the outputs" section.
Video file
Raw gaze-data file:
In social communication research, when eye movements of single participant or two participants are recorded, a frame-by-frame raw gaze data could be exported with the data export tools provided by eye-trackers.
Each function produces the output files that might be needed for the next step. Please click following items to access the sample files.
2d landmarks<br> 3d landmarks <br> Estimated Pose <br> Action Units <br> Image Resolution <br> Exported Frames <br> Training xml <br> Test xml <br> Face Detector <br> Filled Raw Gaze Data <br> AOIs File <br> Manually Labelled AOI File <br> Re-analyzed AOIs File <br>