Adaptive line noise removal differs between PyPREP and Matlab PREP

[a-hurst]

Haven't really dug into it much yet, but I've verified that adaptive line noise removal (a.k.a. CleanLine) differs somehow between PyPREP and Matlab PREP.

This is sort of a weird one, where the MNE notch filter that PyPREP uses seems to be very aggressive and distorts the waveband, whereas the MatPREP version doesn't seem to do much of anything (there's a very subtle difference around the 50Hz and 100Hz bands if you a/b the images). I should probably try different test files though, since average-referencing seems to handle 99% of the line noise distortions in the PSD for this data (is this a feature of CMS/DRL I didn't know about?)

Pre-cleanline data:

Post-CleanLine data (MatPREP):

Post-CleanLine data (PyPREP):

Anyway, I think this the final component I hadn't formally tested yet for #58!

[a-hurst]

Oh, another quick note on this before I head to bed: the MatPREP CleanLine method is much slower than the MNE function PyPREP currently uses. Like, for the tiny test file we use in matprep_artifacts (60 seconds, 64 channels at 160 Hz), it takes about 15 seconds to run cleanline. For the same file in PyPREP, it takes about a second. Maybe MatPREP's doing some more demanding math?

[sappelhoff]

Yes, the adaptive noise filtering in MNE may be a problem (faulty implementation?), we have noticed that before: see e.g., #18 (comment) and #40 (comment)

[sappelhoff]

I mark this for the 0.5.0 milestone, as it requires a fix on MNE-Python, and I am not sure how trivial this will be ...

[a-hurst]

and I am not sure how trivial this will be ...

Yep, I'm dreading this one whenever I end up getting around to it. A lot more layers and files involved than removeTrend or even NoisyChannels, based on my initial investigation.

[sappelhoff]

relevant discussion about this: https://mne.discourse.group/t/notch-filter-to-remove-power-line-noise/3193/4?u=sappelhoff

Also, "zapline" might be a valuable alternative to look into ... not so much for the "MatPREP numerical equivalence" that we'd like, but maybe to go beyond.

[a-hurst]

Just noting this for future reference, but in working on #73 I noticed that the default cleanline settings differ between MatPREP and PyPREP: for the former, the size of the sliding window it uses is 4 seconds. For PyPREP, it's 10 seconds. Will need to re-test with 4-second windows on the off-chance the numbers actually line up!