Re-referencing concerns

[Andesha]

This is just a concern, but I've been working a lot more handson with data coming from the pipeline and am not sure that after we are rejecting a source (channels) that we are rereferencing the data.

I see that there's a function for it for channels here but it does not appear to be used after things like the rank channel are thrown out...

[Andesha]

Poking through a bit more of the code it seems like it's done via the get_epochs function - can someone just confirm this is being done correctly?

[scott-huberty]

I accidentally deleted my response. I'll post it again:

robust average re-referencing happens anytime get_epochs is called:

pylossless/pylossless/pipeline.py

Lines 520 to 521 in b507983

	def get_epochs(self, detrend=None, preload=True, rereference=True,
	picks='eeg'):

Here:

if rereference:
    self.flags["ch"].rereference(epochs)

MOST pipeline steps will call get_epochs before starting, thus average re-referencing the data, excluding any channels that are in pipeline.flags['ch'] or pipeline.raw.info['bads'] from the average.

The exceptions are flag_ch_bridge and flag_ch_rank:

pylossless/pylossless/pipeline.py

Lines 1070 to 1083 in b507983

	# 6. calculate nearest neighbort r values
	data_r_ch = self.flag_ch_low_r(message="Flagging uncorrelated"
	" channels")
	# 7. Identify bridged channels
	self.flag_ch_bridge(data_r_ch,
	message="Flagging Bridged channels")
	# 8. Flag rank channels
	self.flag_ch_rank(data_r_ch,
	message="Flagging the rank channel")
	# 9. Calculate nearest neighbour R values for epochs
	self.flag_epoch_low_r(message="Flagging Uncorrelated epochs")

above, flag_ch_low_r calls get_epochs. At the end of it's funciton it returns the 2D matrix of nearest neighbour values that it operated on, which we call data_r_ch.

flag_ch_bridge and flag_ch_rank operate on data_r_ch directly so they don't themselves call get_epochs and thus the data is NOT robust average re-referenced before.

The next time robust average referencing occurs is flag_epoch_low_r, which calls get_epochs.

@Andesha @jadesjardins if this deviates from the MATLAB code, let us know.

[jadesjardins]

The pipeline was designed to re-reference after every identification of bad channels. flag_chan_bridge is a detection of bad channels so should be followed by re-referencing (excluding the marked channels). The rank channel is not an artifact detection, and the channel flagged by flag_ch_rank should not be removed from the scalp channels. The rank channel remains in the scalp channel data, but this channel is not included in the ICA decomposition. Re-referencing after identifying the rank channel should not do anything because this channel is not marked for rejection (the average reference would be identical to before marking the rank channel).

[scott-huberty]

Thanks @jadesjardins !

From what I can tell, MATLAB lossless didn't re-reference directly after flag_ch_low_r, or afterflag_ch_bridged , either :

Looking at the MATLAB code and my description of pylossless above, it seems like we directly translated the MATLAB code.

Maybe this was a bug in MAT Lossless that we ported over to Python. But when I think about it, we exclude channels from the average so that noisy channels don't skew the ref. Bridged channels don't quite fit that bill I don't think, so I'm not sure how much of an impact adding another re ref would have. I kind of feel the same about low_r chans.

Unless you feel strongly that skipping the re-ref was a bug that pyLossless should correct (or unless I'm reading the matlab code incorrectly) I'm tempted to consider this a feature request and not a bug fix.

In that case I'm definitely +1 to add another re-reference if others think it's important, but personally I won't have time to do it I don't think. but if anyone is really pressing for this step and implements it in their local fork, a PR is welcome

[scott-huberty]

The rank channel remains in the scalp channel data, but this channel is not included in the ICA decomposition.

We are excluding rank_channel from following pipeline steps such as flag_epoch_low_r, so I think I'd consider this part a bug and something we should fix.

[Andesha]

I'm pretty sure it does the rereference in the snippet you linked here:

[scott-huberty]

I'm pretty sure it does the rereference in the snippet you linked here:

Totally, but this is done after finishing flagging low_r, bridged, and rank channels in one go. This is the same thing we do in pylossless.

[Andesha]

@jadesjardins ?

[jadesjardins]

I have not looked at the Matlab lossless scripts in a long time... and I could definitely picture that the re-referencing would only happen at the end of a sequence of channel flagging procedures in a row. That being said, I would recommend implementing the feature of re-referencing after every channel rejection flagging procedure individually. I agree that it is likely to have little impact (so can be low priority), but it would be the correct thing to do.