Skip to content

Pupil Capture, Player, and Service release
Compare
Choose a tag to compare
@papr papr released this 07 Jan 09:23
· 733 commits to master since this release
v3.0
a18e63f
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

We are pleased to announce the release of Pupil Core software v3.0!

Download the latest bundle (scroll down to the end of the release notes and see Assets).

Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄

Motivation

We are excited to introduce pye3d - a complete overhaul of the 3D pupil detector employed by Pupil Core software. In building pye3d, we have left no stone unturned, reimagining many of the algorithmic building blocks constituting the final gaze-estimation pipeline. pye3d is the result of three years of hard work and we are excited to get it into your hands!

Improvements

Improved built-in 3D pupil detector - #2011, #2065

pye3d implements a novel eye-model fitting strategy that incorporates a corneal refraction-correction step, accounting for nonlinear effects in ocular optics. Gaze estimates and pupil diameter estimates are now more accurate than ever before [2,3]. Thanks to a new sphere center estimation methodology [3], it is also considerably faster than our previous 3D detector [1] in regards to fitting robust and accurate eye models. Read more about how it works in our documentation.

Academic references

[1] L. Świrski and N. A. Dodgson. A Fully-Automatic, Temporal Approach to Single Camera, Glint-Free 3D Eye Model Fitting. In Proceedings of ECEM 2013. "L. Świrski and N. A. Dodgson. A Fully-Automatic, Temporal Approach to Single Camera, Glint-Free 3D Eye Model Fitting. In Proceedings of ECEM 2013."
[2] K. Dierkes, M. Kassner, A. Bullling, A novel approach to single camera, glint-free 3D eye model fitting including corneal refraction. In ETRA ’18: Symposium on Eye Tracking Research and Applications, 2018.
[3] K. Dierkes, M. Kassner, A. Bulling. A fast approach to refraction-aware eye-model fitting and gaze prediction. In ETRA ’19: Symposium on Eye Tracking Research and Applications, 2019.

Changes

Removal of old Pupil 3D detector

With this release, we also stop the maintainance of the previous 3D detector. Starting with the new 2.0 release, our pupil-detectors package no longer includes the previous 3D detector. This allows us to remove Ceres as a dependency, making the project much easier to maintain and build from source.

Should you rely on this detector, you can still download previous Pupil Core software releases or install previous pupil-detectors via PyPI.

Bug Fixes

  • Fixed issue that would leave an unresponsive eye window due to plugin crash - #2059
  • Fixed crash on canceling single marker calibration window mode - #2062
  • Fixed crash on post-hoc pupil detection when the frame count is zero - #2063

Developer notes

Network API Changes

Reworked pupil detector plugin and network API

In this release we reworked the pupil detector plugin base class. It is now easier than ever to implement a custom pupil detector and accompanying pupil detector plugin.

We have also introduced breaking changes to our network API for pupil detector plugins. Here is a list of possible notification payload descriptions that are handled by detector plugins:

# Notification for enabling/disabling any or all pupil detector plugins
{
    'topic': 'notify.pupil_detector.set_enabled',
    'subject': 'pupil_detector.set_enabled',
    'value': <is_enabled: bool>,
    'eye_id': <eye_id: int>,  # optional; possible values: 0 or 1
    'detector_plugin_class_name': <detector_plugin_class_name: str>,  # optional
}

# Notification for setting the Region-Of-Interest (ROI) for any or all pupil detector plugins
{
    'topic': 'notify.pupil_detector.set_roi',
    'subject': 'pupil_detector.set_roi',
    'value': <roi: (min_x: int, min_y: int, max_x: int, max_y: int)>,
    'eye_id': <eye_id: int>,  # optional; possible values: 0 or 1
    'detector_plugin_class_name': <detector_plugin_class_name: str>,  # optional
}

# Notification for updating a partial set of pupil detector properties for a specific pupil detector plugin
{
    'topic': 'notify.pupil_detector.set_properties',
    'subject': 'pupil_detector.set_properties',
    'values': <detector_properties: dict>,
    'eye_id': <eye_id: int>,  # required; possible values: 0 or 1
    'detector_plugin_class_name': <detector_plugin_class_name: str>,  # required
}

# Notification for requesting pupil detector properties broadcast for any or all pupil detector plugins
{
    'topic': 'notify.pupil_detector.broadcast_properties',
    'subject': 'pupil_detector.broadcast_properties',
    'eye_id': <eye_id: int>,  # optional; possible values: 0 or 1
    'detector_plugin_class_name': <detector_plugin_class_name: str>,  # optional
}

For more information about custom pupil detector plugins, please read the documentation. You can also see examples of custom pupil detection plugins here.

For an example script that showcases the network API, please consult this helper script.

Changed 3D pupil datum format

pye3d uses a slightly different pupil format than our previous 3D detector. Because pye3d continously updates a single model instead of fitting multiple parallel ones, the new 3D pupil datum no longer contains the following keys: model_id and model_birth_timestamp.

Our latest Hmd-eyes release handles this by falling back to "0" and 0.0, respectively.

Binocular 3D gaze data with string-only keys - #2068

In order to be compatible with the msgpack-python v1.0.0 (specifially the new strict_map_key=True default), this release uses strings only as dictionary keys for all msgpack-encoded data that is published via the Network API.

This change affects primarily binocular 3D gaze data which used to contain integer dictionary keys.

Raw Data Exporter - Optional model_id key - #2061

Some of the fields in the pupil datum related to the 3D model no longer exist in pye3d (see above). This change ensures the raw data export handles missing fields and outputs a compatible csv file.

Updated msgpack dependency to version 1.0 - #2068

We continue to streamline and simplify the installation of Pupil Core dependencies. In this release we updated the msgpack to the latest 1.0. To ensure you have the latest required versions of all dependencies, please install them via requirements.txt.

python -m pip install --upgrade pip wheel
pip install -r requirements.txt

Added support for custom receiver high-water mark for HMD Streaming source - #2058

This change adds the ability to set a custom ZMQ high watermark value for HMD Streaming source. This gives users more control over the streaming behaviour when using Pupil Core with an HMD headset.

Added launcher_process.should_stop notification handling #2070

This change adds the ability to request stopping Capture, Player or Service with the same notification.

import zmq
import msgpack


# create and connect PUB socket to IPC
pub_socket = zmq.Socket(zmq.Context(), zmq.PUB)
pub_socket.connect(ipc_pub_url)


subject = "launcher_process.should_stop"
topic = "notify." + subject
payload = {
    'topic': topic,
    'subject': subject,
}

pub_socket.send_string(topic, flags=zmq.SNDMORE)
pub_socket.send(msgpack.dumps(payload, use_bin_type=True))

Simplified running from source on Windows - #2073

We have removed the necessity to run Pupil from source using the run_<app>.bat files on Windows. Instead of extending the PATH environment variable via the .bat file, we extend it via the application launcher. This also makes it easier to attach a debugger, e.g. via Visual Studio Code.

Downloads

To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).

macOS (11) Big Sur compatibility - #2079

Note: Pupil Core is not currently supported on Mac OS Big Sur. We aim to release a compatability update within the next few weeks. 👷

Assets 5