Automated event renaming doesn't always do the right thing

[hoechenberger]

# %%
from pathlib import Path
from mne_bids import BIDSPath, read_raw_bids

root = Path('~/mne_data/eeg_matchingpennies')
bp = BIDSPath(
    subject='05',
    task='matchingpennies',
    suffix='eeg',
    extension='.vhdr',
    datatype='eeg',
    root=root
)
raw = read_raw_bids(bp)

The event "left" refers to multiple event values. Creating hierarchical event names.
    Renaming event: left -> left/n/a
    Renaming event: left -> left/n/a
    Renaming event: left -> left/1
    Renaming event: left -> left/n/a
    Renaming event: left -> left/n/a
    Renaming event: left -> left/1
    Renaming event: left -> left/n/a
    Renaming event: left -> left/n/a
    Renaming event: left -> left/1
...

The n/as clearly shoulnd't be there.

[hoechenberger]

@sappelhoff (in case you have a minute to spare :))

It appears that the eeg_matchingpennies events TSV file for sub-05 starts out like this:

onset              	duration 	trial_type 	response_time      	stim_file      	value 	sample  	stage 	trial 	bci_prediction 	bci_prediction_valid 	n_repeated 	latency
15.038933333332999 	n/a      	right      	116.666666667      	left_hand.png  	n/a   	n/a     	1     	1     	right          	1                    	2          	-4.2
18.038933333333    	n/a      	right      	116.666666667      	left_hand.png  	n/a   	n/a     	1     	1     	right          	1                    	2          	-4.2
18.1556            	0        	right      	116.666666667      	left_hand.png  	2     	90778   	1     	1     	right          	1                    	2          	-4.2

IMHO this is the same event listed 3 times, and only once associated with a value and a sample, right?

The duration of the first two seems to be in violation of BIDS:

https://bids-specification.readthedocs.io/en/stable/04-modality-specific-files/05-task-events.html

Now I'm wondering if we should just explicitly skip rows that don't have a proper duration set , or issue a warning, or just remain silent and potentially produce garbage and blame it on the user, who fed us a dataset that's not fully BIDS-compliant?

cc @agramfort

[sappelhoff]

The n/a durations indicate that they are NOT zero (not instantaneous events), but that they are also unknown --> this is data from back when I was a masters student and recording some things for an internship, so ... there are many things that didn't go perfect 🙂

The basic problem is that back then, I formatted the BIDS dataset in a "by trial" fashion --> each row in events.tsv corresponds to one event in a trial ... and many columns encode additional stuff that happened during the trial ... but not necessarily at the time of the event corresponding to that row

This is bad practice and I should have just recorded all events and organized events.tsv in a "by event" fashion, where each row is an actual event, and each trial has multiple rows (because typically there are a ton of events happening each trial).

---

The current "version" of matchingpennies was created after a long discussion with Kay because we wanted to add HED tags to the dataset. See: bids-standard/bids-examples#191

We then realized the "by trial" versus "by event" issue, and I pushed changes to make the dataset to be more in a "by event" style ... unfortunately I didn't have enough data to reproduce all the details, so that's where the n/as come from.

Fast forward: Meanwhile, HED has moved to version 3, and we've all gotten a little smarter along the way, so I have a PR open to "revert" the dataset back to the original "by trial" style ... because it's tidier, and HED should be able to deal with "by trial" event structures as well (it's just less detailed than "by events"). See: bids-standard/bids-examples#265

Unfortunately, that PR has stalled for quite a while now.

---

One last comment: I don't think n/a as duration should be invalid ... sometimes you just don't know the duration 🤔

[hoechenberger]

Thanks @sappelhoff! I'll try to make things work better with the current version of the dataset. Don't bother touching it.