Assets 5

We are pleased to announce the release of Pupil v1.9!

Download the latest bundle and let us know what you think via the #pupil channel on Discord ๐Ÿ˜„

Features

Capture

Remote Recorder - #1194

The Remote Recorder plugin no longer requires Pupil Mobile streams to be active in Capture. Instead, it lists all available devices and allows you to start and stop recordings with the click of a button. It is also possible to change the recording session names for all available devices at the same time.

This feature requires Pupil Mobile version 0.25.1 or higher.

Annotation Plugin

Annotations are no longer special types of notifications. Annotations are now sent via the IPC under the topic annotation. Recorded annotations are now stored in annotation.pldata.

Pupil Detectors - #1358

Reduced the default "maximum pupil size" setting in order to decrease false-positive detections.

Time Sync Group Members - #1277

All Time Sync actors (e.g. Capture or Pupil Mobile) join a network group for discovery and time sync announcements. We added the functionality of listing all present group members in the Time Sync plugin menu.

Be aware that all Pupil Mobile instances will be listed as pupil-mobile-follower.

Player

Buffered Playback - #1279

We added buffering to Pupil Player. Instead of decoding each frame on demand, Player decodes multiple frames ahead of time. This improves smooth playback, even at high playback speeds of h264 encoded videos.

Be aware, that this will require a bit more of memory as a trade-off.

Video Exporters

We improved the different video export plugins.

  1. There is a new Eye Video Exporter plugin (#1301). This plugin will export eye videos for your recording.
  2. Video Export Launcher was renamed to World Video Exporter to make its task clearer. By default, the exported video is no longer named world_viz.mp4 but world.mp4 (#1295).
  3. We implemented a new export management UI that is now used in World Video Exporter, Eye Video Exporter, and iMotitions Exporter. You can add as many export tasks as needed and they will all be processed in order (#1322).
Raw Data Exporter

@fneitzel added the possibility to individually turn off pupil and gaze exports in the Raw Data Exporter (#1239).

The exported timestamp, index, and id csv columns have been renamed to more explicit titles (#1352).

iMotions Exporter: Support for world-less recordings - #1308

We added support for world-less recordings to the iMotions Exporter. World-less recordings do not include a scene (world) video and are common for AR/VR setups.

Annotation Plugin

After opening a recording in Player, all annotations (recorded or added in Player) are stored in annotation_player.pldata instead of the offline data directory. This does not override annotation.pldata, so you can reset annotations to the recorded ones by deleting annotation_player.pldata.

Recordings created by former versions of Pupil will be updated to the new format.

Bugfixes

  • Surface edit bug on Macs with Retina displays - #1252
  • Crash when recording during Frame Publisher enabled - #1263
  • Synchronization Drift between Cameras and Recording Computer - #1266
  • Incorrect Frame Publisher world frame topic - #1276
  • Fixation Detector bug where binocular 3d gaze data was ignored - #1286
  • Race condition during Player exports - #1304
  • Incorrect log message handling in background tasks - #1305
  • Timebase changes break fixation and blink detection in Capture - #1324
  • Use of deprecated fields in Blink Detection - #1283

Developers notes

New dependencies

  • pyav 0.4.2
  • cysignal (on macOS and Linux only)

black format

The entire code base has been changed to the black format (#1343, #1344, #1346).

API changes

  • seek_control.trim_indeces_changed notification - #1329
  • set_min_calibration_confidence notification - #1361
  • Fixed field consistency for monocularly mapped gaze - #1291

We are hiring Python developers!

Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.

We are also looking for full stack Python developers that have experience with kubernetes, docker, and async server-side Python.

Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.

@papr papr released this Aug 2, 2018 · 169 commits to master since this release

Assets 5

We are pleased to announce the release of Pupil v1.8!

Download the latest bundle and let us know what you think via the #pupil channel on Discord ๐Ÿ˜„

We have been working hard to significantly reduce the memory usage of Pupil Capture and Player. See incremental data serialization and deferred deserialization for details.

Features

Capture

Auto-exposure mode -- #1210

We have added an auto-exposure mode for the 200Hz Pupil cameras. You can enable it in the UVC Source menu of the eye windows.

Incremental data serialization -- #1141

Prior to this release, data was cached in memory during recordings and written to disk after the recording had finished. This resulted in large memory consumption during recording.

Starting in v1.8, Pupil Capture will store data directory to the disk as it becomes available during the recording. This reduces the memory footprint and improves reliability of Pupil Capture. See the New Recording Format section on our documentation.

Automatic recording stop on low disk space

The recorder will show a warning if less than 5GB of disk space is available to the user. Recordings will be stopped gracefully as soon as less than 1GB of disk space is available.

Fingertip Calibration -- #1218

We introduced a proof-of-concept fingertip calibration with Pupil Capture v1.6. It was based on traditional computer vision approaches and the fingertip detection was not very stable.

Now, we are releasing a revised version that uses convolutional neural networks (CNN) for the hand and fingertip detection. For details checkout our documentation.

Note - The current bundle support CPU inference only. If you install from source and have an NVIDIA GPU with CUDA 9.0 drivers installed, you can install pytorch and our fingertip detector will use your GPU!

Player

Deferred Deserialization

Before v1.8, opening a recording in Pupil Player would read the entire pupil_data file and deserialize the data into Python objects. This enabled fast processing of the data but also used excessive amounts of memory. This led to software instabilities for users who were trying to process recordings with long durations.

Starting with v1.8, Pupil Player only deserializes data if required. This reduces memory consumption dramatically and improves software stability.

Please be aware that the initial upgrade of recordings to the new format can take a bit of time. Please be patient while the recording is converted.

Temporally disabled features

We had to disable the following features due to changes on how we handle data within Pupil Player:

  • Vis Scan Path
  • Manual gaze correction for Gaze From Recording

We are working on a solution and will hopefully by able ro re-enable these with in the next release.

Bugfixes

  • Fixed a bug were Player crashed if info.csv included non-ASCII chracters -- #1224
  • Correctly reset the last known reference location after stopping the manual marker calibration in Capture -- #1206

Developers notes

New dependencies

We have added PyTorch to our dependencies. If you want to make use of GPU acceleration you will have to run Pupil Capture from source and install the GPU version of PyTorch. We will work on bundling GPU supported versions in the future.

New recording format

We had to make changes to our recording format in order to make the incremental serialization and deferred deserialization features possible. Please, see our documentation for more details on the New Recording Format.

API changes

zmq_tools.Msg_Streamer.send() has been reworked. The previous required two arguments: topic, payload. The new version only accepts the payload argument and expects the payload to have a topic field instead.

Real-time fixation format changes -- #1231

Online fixations are published in a high frequency. Each fixation has a base_data field that includes gaze data related to this fixation. In turn, gaze data has a base_data field on its own including pupil data. As a result, recordings grew unreasonably fast in size if the Online Fixation Detector was enabled. E.g. for an eleven minute long recording, the pupil_data file grew to 1.4GB of which 1.1GB were fixations alone.

As a consequence, we are replacing each gaze datum in the base_data field of online fixations with (topic, timestamp) tuples. These uniquely identify their corresponding gaze datum within the gaze data stream.

We are hiring developers!

Hey - you're reading the developer notes, so this is for you!

Pupil Core Contributors - We're hiring developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.

Full Stack Developers - We are also looking for full stack developers that have experience with one ore more of the following tools/platforms: kubernetes, docker, vue.js, and flask.

Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.

@papr papr released this May 4, 2018 · 338 commits to master since this release

Assets 5

v1.7 Release Notes

Overview

We are pleased to announce the release of Pupil v1.7!

Download the latest bundle and let us know what you think via the #pupil channel ๐Ÿ˜„

Features

  • Adjustable audio playback volume - #1131
  • Audio wave visualization - #1131
  • Online Surface Tracker: Map and publish fixations - 4bb1aeb
  • 2d and 3d detector improvements - #1168
  • Capture opens both eye windows by default - e2d9576

Bugfixes

  • Service: Enable Blink Detection plugin - #1137
  • Player: Use correct timing for background video sources
  • Player: Fix jumping back and forth between fixations - #1151
  • Linux dependencies: Fix compatibility with current OpenCV master - 639b041
  • Fixation Detector: Improve false negative rate - bdd2333

@papr papr released this Mar 21, 2018 · 403 commits to master since this release

Assets 5

Overview

We are pleased to announce the release of Pupil v1.6!

Download the latest bundle and let us know what you think via the #pupil channel ๐Ÿ˜„

Features

Audio Playback in Player -- #1111

Starting with v1.6, Pupil Player supports playback of recorded audio. Audio playback only works if the playback speed is set to 1x. Mute functionality and audio wave visualization will be implemented in the next release.

Developer notes

The audio playback implementation is based on portaudio and its pyaudio wrapper. Installing these will be required to run from source. See the developer doc changes for details.

Audio Capture for Windows -- #1110

We are very happy to add support for audio capture on Windows!

iMotions Exporter -- #1118

We added a plugin in Pupil Player that will enable you to export Pupil data to iMotions data format. You can import the exported Pupil data into iMotions software.

Experimental Fingertip Calibration -- #1108

This experimental calibration routine allows you to self-calibrate by using your fingertip as a reference point. This method should be used for prototyping and experimentation only. The detection robustness is not yet production grade, but we will put a lot more effort into this going forward. We wanted to release the idea so that you can try it out, suggest improvements, and provide feedback feedback.

Accuracy Visualizer -- #1126

Accuracy Visualizer is now a default plugin. This plugin will visualize the calibrated area. You can turn the visualization off in the plugin's settings menu.

Bugfixes

  • HoloLens Relay: Fix gaze topic format change regression -- 6534ec
  • UVC Source: Use hardware timestamps for Pupil Labs cameras only -- #1123
  • Automatic detection and reinitialization if 200Hz eye camera exhibits stripes: issue #1116 -- #1024. If you are still experiencing this behavior after updating to v1.6, please open an issue and we will follow up with you to debug.

@mkassner mkassner released this Mar 2, 2018 · 467 commits to master since this release

Assets 5

Overview

We are pleased to announce the release of Pupil v1.5!

Download the latest application bundle and let us know what you think via the #pupil channel ๐Ÿ˜„

Features

Capture

  • Indicate annotation firing with a short log message. This provides realtime feedback for users that fire annotations via Pupil Remote -- #1080
  • Screen-based calibration plugins: Session-persistent monitor selection -- #1091
  • Capture allows creation of recordings without an active world video source, requires pyndsi>=0.4 -- #1101.
  • Enable eye video recordings by default - #1101
  • Enable/disable eye video recordings using the recording.should_start notification -- #1101

Player

Trim section and export improvements -- #1093

  • Remove trim section indicators from video and raw data exporters
  • Add trim section indicators to general settings, showing both:
    • relative recording time
    • frame indices
  • Add export_info.csv file to export folder. This csv file contains the following information:
    • Player version
    • Data format version
    • Export start date and time
    • Frame index trim range
    • Relative time trim range
    • Absolute time trim range (world timestamp range)
  • We no longer use trim range with frame indices as a naming convention
  • We now use incrementally increasing folder names with format %d%d%d, e.g.:
    • <recording>/exports/000/
    • <recording>/exports/001/
    • <recording>/exports/002/

Confidence threshold consistency -- #1100

  • Previously, all binocular mappers dismissed pupil data with confidence lower than 0.6 to prevent bad binocular mapping. Now, low confidence pupil data is mapped monocularly.
  • Introduction of a user settable global minimum calibration confidence threshold. This threshold sets the lower bounds of the pupil data confidence that is used for calibration. This value was previously fixed to 0.8 in Pupil Capture and to 0.0 in Pupil Player -- effectively using low confidence pupil data for calibration. This threshold can be set in all calibration plugin menus.

These changes were introduced based on #1052 and #1047.

Minimum data confidence: Lower bounds gaze data by confidence that is used to generate visualizations and high level data, e.g. fixations. This does not effect the raw data export. Following data points are effected by it:

  • Fixations
  • Offline surface heatmaps
  • Circle, cross, light points and polyline visualizations

Bugfixes

  • Fix Fake_Source compatibility with Frame Publisher -- #1085
  • Service does not crash when starting Frame_Publisher via a notification -- #1099
  • Frame_Publisher only warns once on encountering an incompatible format instead of warning each time -- #1099
  • Fix an issue where Fake_Source would return frames at 2 fps in Capture if it was restored from the session settings. -- #1085
  • Offline Fixation Detector: Correct upper limit of fixation lenghts to 4000ms -- #1084
  • Avoid ZeroDivisionError when calculating hdpi factors on Windows -- #1090
  • Avoid positioning windows by default behind the Windows task bar -- #1103
  • Fix bug where export folders were written into the recording folder itself instead of into the exports subfolder
  • UVC control settings are correctly restored from session settings and on reconnect -- #1099 and #1101
  • Avoid crash on recordings with less than 10 world frames -- #1101
  • Player always checks for recordings without world videos instead of only checking on upgrading the recording format - #1101
  • Fixed a bug that crashed Player when removing a surface - #1106
  • Fixed a bug that crashed Capture on restoring a minimized window on MS Windows 10 when log display was loaded - #1097

@mkassner mkassner released this Feb 16, 2018 · 515 commits to master since this release

Assets 5

v1.4 Overview

We are pleased to announce the release of Pupil v1.4!

Download the latest bundle and let us know what you think via the #pupil channel ๐Ÿ˜„

Features

Pupil Player

  • Support for Pupil Mobile recordings without world video -- #1071
  • Annotation csv exports include user-defined fields -- #1061
  • Trim calibration and mapping ranges using trim marks -- #1057
  • Add Jump to previous fixation thumb button to the Offline Fixation Detector plugin -- #1072
  • Export ranges are based on time instead of frame indices. The export folder will be named appropriately -- #1072
  • Resuming playback at the end of the trim section will seek to the beginning of the trim section instead of continuing playing after the trim section. This allows you to play back the trimmed section repeatedly without having to seek manually -- #1072
  • Pupil Player default plugin list -- d40a3b
    • Remove the Vis Scan Path
    • Add the Raw Data Exporter
  • FakeCapture (aka Test Image) frame content now displays the Pupil Labs gradient.

Bug Fixes

  • Improve Player timeline drawing performance by caching the drawn data -- #1063, #1068
  • Improve performance while playback in Player is paused -- 2df34d
  • Remove bug where the Vis Scan Path plugin would generate increasingly bad results with each playback iteration -- bf1977
  • Catch one of reasons that lead to the GLFW: Cannot create window error on Windows

Feedback

As always we hope you enjoy the new release and please let us know if you discover any bugs. We look forward to your feedback!

@papr papr released this Feb 2, 2018 · 575 commits to master since this release

Assets 5

v1.3 Overview

We are pleased to announce the release of Pupil v1.3!

There are a lot of new features - especially for Pupil Player - that we are excited to share with the Pupil community.

screen shot 2018-02-02 at 13 34 22

_Screenshot of Pupil Player demonstrating the timeline graphs features during a "blink" event._

Download the latest bundle and let us know what you think via the #pupil channel ๐Ÿ˜„

Features

General

  • Text input - Support for cut/copy/paste and double click to select all in text input fields - #1027
  • Collapsed menu - Main menu is now collapsed by default#1019. You can expand the menu by clicking any of the the icons on the right hand side of the screen.
  • Pupil detection - Disable coarse detection for eye videos with width less than 200 pixels #1022
  • Pupil Service UI #1016

Pupil Service UI

Pupil Capture

  • 200hz eye camera - Improved support for the new Pupil Labs 200Hz eye cameras
  • Online pupil detection opt-out - This feature enables one to opt-out of online pupil detection in Pupil Capture #1009. If you use this feature, you should record eye videos so that you can use offline pupil detection and calibration in Pupil Player (Recorder > Record eye).
  • Real world start time - Add record start time as Unix timestamp to info.csv #1017 - this enables users to calculate when the recording started in real-world time.
  • Single marker calibration - Add display modes for Single Marker Calibration #1021

Pupil Player

  • Time based seeking in Player #1041
  • Improved Player seek control
  • Add more Player timelines
  • Offline blink detection #1040

Offline Calibration Improvements

  • Disabled automatic search for calibration markers
  • Added option to cancel calibration marker seach
  • Added option to use detected calibration marker as natural features for manual filtering
  • Improved caching of detected data

Improved Player Seek Control

  • Highlight elements on hover that can be activated when being clicked or dragged
  • Playback modes and speeds #1023
    • Frame-by-frame seeking when paused
    • Decrease and increase playback speed while playing
  • Point & click somewhere on the seek bar to jump to this position

Player Timelines

  • Added pupil confidence graph in the timeline #1040
  • Added 2d pupil diameter (pixels) graph in the timeline diameter, 2d, in pixels #1040
  • Added blink events graph in the timeline #1040
  • Added recorded FPS, for world and eye videos as a graph in the timeline #1051

Bug Fixes

  • Fixed visual bug where offline gaze mapping would be stuck at 99% #1014
  • Fixed crash when changing a text field in the Accuracy Visualizer menu #1007
  • Fix passing a recording to Pupil Player as a command line argument #1006
  • Fix bug where main menu would not expand #1006
  • Improved ROI handling #1031

Feedback

As always we hope you enjoy the new release and please let us know of any bugs. We look forward to your feedback!

200hz video Pupil mobile bundle support

This has been added with v1.3-9! Please update to this version and re-download the recordings from your device.

@papr papr released this Dec 14, 2017 · 703 commits to master since this release

Assets 5

v1.2 Overview

The new v1.2 release improves overall stability and includes support for the new 200Hz eye cameras. We highly recommend upgrading Pupil Player, Service and Capture to v1.2.

Features

  • Support for the new 200Hz eye cameras (#983)
  • Default eye video size to qVGA or smaller for better performance.
  • Bundle support for RealSense 3D on macOS and Windows
  • Add Hololens Relay - UDP relay for Pupil to HoloLens communication
  • Improve calibration marker detection, see below for details (#961, #965)
  • Offline Surface Tracker - Add confidence column to gaze export (#941, #945)
  • Refactor Video Export procedure (#927)
  • Refactor Batch Exporter (#931)

Bug Fixes

  • Improve Retina display support (#935)
  • Improve Camera Intrinsics Estimation stability (2028f1e, #969)
  • Fix Offline Fixation Detector (#950)
  • Improve HMD Calibration (3d) stability (#970)
  • Include pyndsi v0.3.3 which fixes recording Pupil Mobile streams (#910 #976 #981)

Calibration marker detection

The calibration marker detection accuracy and efficiency has been improved. The procedure works best with our new markers:
pupil calibration marker v4

Feedback

As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!

@willpatera willpatera released this Nov 9, 2017 · 792 commits to master since this release

Assets 5

v1.1 Overview

We are pleased to announce the release of Pupil v1.1 - this release happily coincides with the 100th release of Pupil software!

There are a lot of new features in this release and we are especially excited to introduce Pupil's new GUI!

image

We highly recommend upgrading Pupil Player, Service and Capture to v1.1.

A note on versioning

TL;DR Please donโ€™t read too deeply into version numbers. We will be deploying new software on a continuous basis and version numbers will continue to increment accordingly.

The core team had a healthy debate on about version numbering. Should we release v1.0 or v0.10.? This could seem like a trivial debate about naming conventions, but we also realize that there is a lot of โ€œbaggageโ€ associated with versioning. We acknowledge that many people see v1.0 as a big step for a project. It often signifies a level of maturity of the project and community. Pupil is certainly growing into a platform with a healthy group of contributors and researchers around the world who depend on Pupil software and hardware. Pupil is certainly more mature now compared to where it was 5 years ago. But we want emphasize that Pupil is an active project that will always be in flux. We will always need feedback from the community, there will be bugs and together we will fix them! This version is just one small step forward of many to come. ๐Ÿ˜„

Prior to this release we were (ab)using Semantic Versioning. For those not familiar with the Semantic Versioning, the version numbers follow the format major.minor.patch. From a technical perspective, we should have released a major version when we first defined the plugin structure, and another major version whenever we redefined the plugin interface, and yet another major version when we defined the โ€œAPIโ€ with the IPC Backbone in v0.8.

Whereโ€™s v1.0?

With all the changes that we made with the new UI, it looks like we flew right past v1.0 and went straight to v1.1! There are more than 100 commits between the v1.0 and v1.1 tags, so a minor bump was required.

Features

  • New GUI - A lot of work went into redesigning the GUI in Pupil Capture and Pupil Player. There are ton of new features; too many to list! Just dive in and let us know what you think via chat or make an issue if you find a bug!
    • Pupil Capture - Checkout the docs for a high level overview of the new GUI structure here
    • Pupil Player - The most significant GUI changes are in Pupil Player. The new GUI greatly simplifies the interface. No more floating plugin windows cluttering the window! We also introduce an expandable events timeline so that all temporal events can be clearly labeled and correlated with the timeline. For a high level overview check out the docs.
  • Plugin Manager - The central place for loading and unloading other plugins. No more tedious Select to loaddrop down menus! ๐Ÿ˜„
  • Pupil diameter history graph - You can now visualize changes in pupil diameter over time in Pupil.
  • Improvements to libuvc, our usb video backend. Considerably more accurate timestamp generation and more robust failure behaviour on usb disconnect and recovery.

Bug Fixes

  • Pupil Mobile file import - Pupil Player can now properly read and display files recorded by Pupil Mobile that were saved on the Android device (see #873)
  • Fixation detector - Substantially decreased number of false negative detections
  • Realsense backend - Improved failure behaviour on usb disconnect and recovery. Added pointcloud visualizer window.
  • HMD calibration - Improved Network API based on feedback from HMD-eyes developers.
  • Manual marker calibration - Improved detection accuracy of markers.

Backwards incompatible changes

We revised the plugin API with v1.0 in order to ensure that all plugins work consistently with the new GUI. Please be aware that some of these changes are incompatible with the old API. This means that old plugins need updating before they can run correctly in v1.1. Please see the these pull request notes on details about the changes. We also updated the plugin guide accordingly.

Disabled Plugins

These plugins will be disabled due to legacy incompatibilities:

  • Batch Exporter
  • Marker Auto Trim Marks

We will re-add them on demand and when we have time to refactor them.

We will work on refactoring these plugins and will release an up-to-date version with the next release. Follow development progress and express your interest in these features in the following issues:

Windows Driver Installation

Drivers are automatically installed and updated! Please let us know if this does not work for you so that we can improve the installer!

Feedback

As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!


Linux bundle has been updated to support Ubuntu 17.10.

Oct 10, 2017

v1.0

Merge pull request #860 from pupil-labs/ui_refactor
Ui refactor