Spiky artifacts in estimated drift

[lshaheen]

Hi,

I'm trying out the new drift correction in KS3. For some reason I'm getting a lot of spiky artifacts in the drift correction. In the first plot here I'm plotting rez.dshift, and changed the x axis to be time. I set rez.nblocks=1 for rigid registration to make it simpler, but I saw the same artifacts with the default, usually they would happen on all the segments together. There are several recordings concatenated together from a single site, and there are short periods between them where we didn't capture data. The dashed lines show the start of each recording. The electrode was a 64D from UCLA, which has a 3-column checkerboard pattern. But for some reason the center column wasn't giving good data so I excluded all those electrodes from this sort, making the electrode 2 columns spaced 40 um apart, with 50 um between rows. Maybe the problem is that this is just a little too sparse to capture the drift well? Have you seen these kind of artifacts in your data?

In the second plot I added the rez.dshift from two more sorts. The orange is a sort with just the first 6 recordings, the blue has just the 6th. The artifacts are in different places than when you include all the data, which seems odd.

Any insights are much appreciated!

Thanks,
Luke

[marius10p]

This happens when there is not enough data to estimate the drift well. To compensate, you can increase the batch size ops.NT by a factor of 8 or so. I can also look into adding an optional smoothing factor to encourage smooth drifts.

[lshaheen]

Thanks. I increased the batch size by a factor of 4 and added a 3-point median filter which got rid of a few remaining blips. If anyone else reads this and wants to try, it, I added this code to line 97 of datashift2.m:

med_filt = getOr(rez.ops, 'drift_estimate_median_filter_pts', 0);
if med_filt>0
    imin_pre_filt=imin;
    imin = medfilt1(imin,med_filt,'truncate');
end

And set ops.drift_estimate_median_filter_pt=3;