Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raw data exporter: Add option to not export low confidence values #2210

Merged
merged 6 commits into from Nov 19, 2021

Conversation

papr
Copy link
Contributor

@papr papr commented Nov 18, 2021

Pupil Core software assigns "confidence" values to its pupil detections and gaze estimations. They indicate the quality of the measurement. Until now, the raw data exporter plugin exported all data, independently of its confidence. New users are often not aware of this and wonder why the exported data looks noisier than what they have seen in Pupil Player.

For visualization, Pupil Player hides gaze data below the "Minimum data confidence" threshold (adjustable in the general settings menu). Often, we recommend users to filter their data by a confidence threshold after exporting.

Starting with this PR, users will have the option to only including data above the "Minimum data confidence" threshold in the export. The option can be enabled in the raw data exporter's menu.

(credits to @N-M-T for raising awareness of this issue)

Screenshot 2021-11-18 at 15 48 00

@papr papr changed the title Raw data exporter: Filter low confidence values by default Raw data exporter: Don't export low confidence values by default Nov 18, 2021
Copy link
Contributor

@N-M-T N-M-T left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The export functionality works as expected. However, I think the user should have to 'opt in' to filter out low-confidence data. My reasons are as follows:

  1. A lot of existing users are familiar with the raw export containing low-confidence data
  2. By defaulting to filter out low-confidence data, we might need to revise tutorials that filter blinks, or at least clarify for the user in each tutorial which option they should have toggled
  3. Filtering low-confidence data means there are missing samples in the export. This might lead to confusion for users who are calculating, e.g. eye rotation velocity using the intersample duration, or just when investigating sampling rates in general

@papr
Copy link
Contributor Author

papr commented Nov 18, 2021

  1. Fair enough. But they should also understand what is happening if these are gone. But I get the sentiment, that changing the default "include low conf data = True" is more backwards compatible than the current implementation.
  2. That is a very good point
  3. eye rotation velocity should not be calculated with low confidence values as they introduce high velocity outliers. The intersample duration can be easily calculated even if you have gaps in the data

@papr papr changed the title Raw data exporter: Don't export low confidence values by default Raw data exporter: Add option to now export low confidence values Nov 18, 2021
@papr papr changed the title Raw data exporter: Add option to now export low confidence values Raw data exporter: Add option to not export low confidence values Nov 18, 2021
@papr papr requested a review from N-M-T November 18, 2021 14:48
@N-M-T
Copy link
Contributor

N-M-T commented Nov 18, 2021

3a. Indeed, users should not calculate eye rotation velocity using low-confidence data points
3b. More generally though, I think having the low-confidence data removed by default would have thrown users off when they were calculating, e.g the average FPS of their gaze data – if they weren't aware of the gaps, diff(timestamps) would contain some erroneously large values. In anycase, include low conf data = True will prevent this!

Co-authored-by: Neil M. Thomas <neil87@mail.com>
@papr
Copy link
Contributor Author

papr commented Nov 18, 2021

3b. Agreed!

@papr papr requested a review from N-M-T November 18, 2021 16:30
@peteratBHVI
Copy link

3a. Indeed, users should not calculate eye rotation velocity using low-confidence data points 3b. More generally though, I think having the low-confidence data removed by default would have thrown users off when they were calculating, e.g the average FPS of their gaze data – if they weren't aware of the gaps, diff(timestamps) would contain some erroneously large values. In anycase, include low conf data = True will prevent this!

Good points,
also the amount of low confidence data is a good quality measure. e.g., indicate if the eye cameras need to be in a better position for particular visual tasks. I like the option to be on.

@papr papr merged commit b8fd82c into develop Nov 19, 2021
@papr papr deleted the filter_low_conf_raw_data branch November 19, 2021 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants