Finding Pupils

A general guide on how to find pupils

The main SharpEyes window provides functionality for finding pupils in a video.

This is how I generally do things.

I refer to the parts of the window as follows:

• "Video player": the left-hand half of the window that displays the video
• "Controls panel": the right-hand half of the window that has all the knobs

1. Open a eyetracking video (Open Video button/File->Open Video)
2. Read in timestamps. For calibration purposes, we need to know the timestamps on each frame. In the controls panel, under the "Pupil Finding" tab, click "Read timestamps." Wait for the computer to do its thing. There is a progressbar at the bottom.
3. Draw a window around where you expect the pupil to be. Click on the "Draw window" button in the toolbar, and then click-and-drag over the video. You'll need to click on the button again to update the window. This restricts the search for the pupil over the window instead of over the whole frame and makes things to faster.
4. Start the pupil finding by clicking on the "Find pupils" button in the controls panel. You can stop the pupil finding process by pressing the button again (which should read "Cancel" when pupil finding is happening). By default, SharpEyes will go through 120 frames, and then pause. This number can be adjust by the file picker under the "Find Pupils" button. You can also check the "All Frames" check box to tell SharpEyes to go all the way to the end. This is not recommended. You will see the blue circle overlaid on each frame with where the SharpEyes thinks the pupil is.
5. Adjust pupils on-the-fly. If the "Move Pupil" button is not highlighted when the video is pause, click on it. When it's highlighted, you are in "edit pupils" mode. In this mode, you can click and drag the pupil to move it, and the scroll wheel adjusts pupil size. The left/right keys (and the two arrows next to the playu/pause button under the video player) allow you to manually step through the video by frames. Any edits made on one frame will be propagated to the following frames, because we assume the error in the found pupil location to be a stationary error.
6. Tune the pupil finding process on-the-fly. By default, the template pupil finder uses idealized templates (black circle on white background) to find the pupils. This is not ideal. Instead, when you have manually adjusted the pupil, you know that it's correct. You can tell SharpEyes to used that part of the frame as a template instead. To do so, click the "Add current as template" button in the bottom half of the "Pupil Finding" tab in the controls panel. If you add a template from a frame, the idealized templates will be cleared. Through the course of a run, I will normally end up with 70-100 templates. You can check the "auto add" checkbox to automatically add each frame that you have manually edited as a template. Sometimes, templates from earlier in the video won't work well for the latter half, so you can use the drop-down combobox to tell SharpEyes to use only the N most recent templates.
7. Process the entire video. Go through the video, pause and adjust where necessary. There is a colorbar on the top of the video player's slider. Pink parks have not been processed. Green parts have.
8. Save out the data. In the File menu, save the timestamps, pupil positions, and the templates. Both timestamps and pupil positions are stored as standard numpy arrays. The templates are stored as a *dat file, which is just a renamed zip file with all the templates. If you have "Auto save" checked in the controls panel, these are automatically saved whenever you pause the pupil finding process. These files can the loaded back in later to edit the found positions, so you don't need to re-do the whole process.

Now you are reading to build the mapping between the pupil position and gaze position.

For all controls, see the pupil finding reference.