Extend plugin architecture to better handle data-generating overlays

[becky-gilbert]

This is for things like eye-tracking, mouse-tracking, and mouse-view features, which all aren't standard 'trials' like other plugins, but instead involve additional code and data collection on top of 'normal' jsPsych trials. Once one of these 'overlay' features is initiated during an experiment, it should be able to run arbitrary code in response to trial events (on_start, on_finish etc.) and save its own data to the trial data. This way, any arbitrary code that is needed in response to trial events can be handled in the overlay feature's code, rather than requiring the researcher to add anything to the trial's event-related callback functions.

Here's an example of what the trial definition might look like:

var trial = {
  type: 'image-button-response',
  ...,
  extensions: ['mouse-view']
}

Note that we probably wouldn't be able to guarantee that plugin overlay extensions don't interfere with one another, so it might be risky to allow multiple extensions on the same trial. If multiple extensions are allowed, then we should probably at least warn the user about this.

[jodeleeuw]

A few quick thoughts about implementation...

We should define a template structure like we do for plugins right now. This structure would include a number of functions that are tied to specific events in the course of a plugin:

var extension = {}

// when the trial is starting, executes before plugin.trial runs
extension.on_start = function() { ... } 

// when the trial is loaded, executes after plugin.trial runs
extension.on_load = function() { ... }

// when the trial finishes, executes after jsPsych.finishTrial(...) but before the on_finish trial event.
// this is where it would be logical to add data into the trial's data object based on what the extension grabbed.
extension.on_finish = function() { ... }

It might make sense to also have an initialize function that runs once when the extension file is loaded. We may also want some method for getting/setting internal state of the extension. This could be important for calibration routines. I'm imagining that a flow for eye tracking might involve having a plugin that runs the calibration routine, and then adding the extension to subsequent trials. Not all extensions might require this (e.g., mouse tracking doesn't need to be calibrated) flow, so we should think about how initialization routines can be run in either kind of setup.

[chrisbrickhouse]

You may want to look into MediaWiki's solution which uses hooks. It's very similar to Josh's solution, but the event handlers would pass an object (probably plugin.trials) back and forth to core which extensions can use to prevent collisions through locks.

I'm looking into designing a web-based eye tracking study using WebGazer. I can't guarantee any code since the project is still in planning, but I can report back with ideas for how to support that workflow.

[becky-gilbert]

@jodeleeuw I know this is a priority so that we can get eyetracking up and running. Let me know if there's anything I can do to keep it moving forward 👍

[jodeleeuw]

Thanks @becky-gilbert ! I've been working a lot on this in the corresponding branch. I think the core tech is in place now. I'm mostly fiddling with webgazer stuff to try and get good calibrations. I don't think there's a good way to split this work at the moment, but I'll keep you updated for sure.

[becky-gilbert]

Got it, thanks for the update @jodeleeuw !