Anti-phase templates

[czuba]

---

NOTE: This issue is [thought to be] related to a separate post #60 documenting punctate temporal stuttering in kilosort2 output clusters

---

I've been having trouble achieving usable sorting results from Kilosort2, and results are actually so far from usable that I suspect that there must be some underlying initialization or parameterization problem.

The data

• Examples are based on a single plx file recorded during part of a relatively brief [~40 min] duration & [subjectively] stable session.

  • 32 channel plexon U-probes with stereotrode configuration: 50µm within, 100µm between stereo pairs; 1.5 mm total
  • Sampled continuously 40kHz via Plexon MAP system
    (...old but stable hardware, bootstrapped into the modern era)
  • Converted to .dat files with common median referencing to remove cross channel artifacts

• Though these examples are from a single file, similar issues are present across many attempts at applying Kilosort2 to files both long (2-3 hr) & concatenated (2-4 consecutively recorded data files), and with varying degrees of drift.

• All kilosort parameters are set to default values or corresponding equivalents for 40kHz
  (...reverted to defaults after many attempts at tuning parameters to no avail.)

  • e.g.

    % make waveform length independent of sampling rate
    ops.nt0    = ceil( 0.0012 * ops.fs); % width of waveform templates (makes consistent N-ms 
    either side of spike, regardless of sampling rate)
    ops.nt0    = ops.nt0 + mod(ops.nt0-1, 2); % ensure oddity (...forum something about aiding 
    spike trough alignment)

    % define batches in seconds of data, not abstract bit chunks
    batchSec  = 10;  % define batch number in seconds
    ops.NT    = ceil(batchSec*ops.fs/32)*32 +ops.ntbuff; % convert to 32 count increments of 
    samples

• Batch reordering figure:
  

I'm experiencing two primary failure modes:

1) Temporal stuttering

• [split into issue temporal stuttering #60]

2) Anti-phase templates

The issue of discrepancies between template and mean waveforms was brought up in the issue #31. The previous instance was thought to be related to low-amplitude waveforms (near the noise floor of the recordings) and/or an unmentioned timing issue, but the examples I'm seeing are clearly above the noise, and I've checked & rechecked timing parameters ad nauseam.

The filtered and predicted timecourses are clearly evident in the gui display

And remnants from the ill fit template subtraction (and altogether missed spikes) are readily apparent (actually amplified) in the residual timecourses:

---

...with the futility of further random walks through parameter space, I'm hoping that these dead ends might reflect issues that other users have encountered & defeated, or that the Kilosort authors might be able to chime in on potential causes/solutions.

It seems more than likely that these issues are related, and that part of the temporal stuttering is a result of runaway templates within batches that simultaneously miss & distort (by fitting new templates to the residuals) otherwise corresponding clusters across batches.
Or so one would hope.

Many thanks to any & all who might be able to shed some light on this.

[marius10p]

The discrepancy between templates and mean waveforms probably has nothing to do with this: the templates are after data preprocessing, i,.e high-pass filtering and channel whitening, and that usually changes a little bit what they look like. That change can be larger for some waveforms depending on their spatiotemporal frequency content.

The mismatch between the filtered and predicted trace probably has a different cause. Do you observe this mismatch throughout the recording, or only on segments? If you run it for a shorter segment (i.e. 5 minutes) do you still get mismatches? I ask because the templates are continuously changing as a function of time, and while they might be good for a section of the recordings, they might somehow become mismatched during another segment.