Data integrity when concatenating artifact corrected sessions

[Idavr]

Hi there!

I usually do analysis on straight-forward single recordings, but recently I ran an inactivation experiment with optogenetics while recording with Neuropixels and am trying to look at cell identities across two sessions (done immediately after each other, just 10 min break and animal was connected the entire time). Due to light artifacts I have artifact corrected both sessions with CatGT and then concatenated them and just now finished running kilosort2 on the entire file. I was wondering if the interpretation of the matrix from the batch sorting should change, as the files (both before and after concatenation) creates a matrix with harder lines than in single non-optogenetic sessions. I am assuming that the "jumps"/"drifts" KS2 visualizes are due to the periods of inhibition as well as some shifts at the point of concatenation (end of first/start of second session) instead of degraded data integrity, but would this be a correct interpretation? Also, how well does KS2 keep track of the unit identities across longer periods of silencing? Shorter periods of silence would be natural as all neurons do not fire all the time, but I am experimenting with 30 minute inactivations, and was wondering whether that would be too long of a silent period for KS2 to identify the same unit from the start of the recording and then the same one again at the end?

Thank you very much for your time on this.

[marius10p]

Hi, it is normal to get a blocky matrix if the distribution of active neurons changes. If a unit is completely silent for 30 minutes, that will probably cause problems. If it goes from 10 Hz to 1 Hz, that should still work fine. The only way to know is to examine the results in Phy.

Since we just released Kilosort 2.5, I would especially recommend it for this scenario.

[Idavr]

Hi. Alright, thank you. There are some... Interesting... Visualizations in phy that I am investigating, but will try Kilosort 2.5 and see if there is a difference! I just now cloned into KS2.5, but when trying to select the probe layout in the GUI I see three Phase3B1 (staggered) choices, and regardless of which I choose I get this error message in MATLAB:

Error using unique (line 117)
Invalid input. Valid flags are 'rows', 'first', 'last', 'stable', 'sorted', 'legacy'.

Error in ksGUI/updateProbeView (line 1195)
ux = unique(cm.xcoords); uy = unique(cm.ycoords);

Error in ksGUI>@(,)obj.updateProbeView('reset') (line 271)
'Callback', @(,)obj.updateProbeView('reset'));

Error while evaluating UIControl Callback.

Also, when running the eMouse_drift (just to check that everything works), the function "clusterSingleBatches" is not found and is not in any folder of Kilosort.

[marius10p]

I fixed the layouts and I am now working on an update the eMouse simulation.

[Idavr]

Thank you, layouts working now! However, more peculiarities have popped up when trying to run it on my data now:

When using main_kilosort I was able to run one recording without any problems, but when trying to another recording I get this error:

Error using gpuArray/nan
An unexpected error occurred trying to launch a kernel. The CUDA error was:
invalid configuration argument

Error in median (line 71)
y = nan(s,'like',x);

Error in learnTemplates (line 248)
toc, ibatch, niter, Nfilt, sum(nsp), median(mu), numel(st0), ndrop)

Error in learnAndSolve8b (line 35)
rez = learnTemplates(rez, rez.iorig);

Error in main_kilosort (line 45)
rez = learnAndSolve8b(rez, iseed);

When trying to run it through the GUI this error occurs: "Error preprocessing! Not enough input arguments."

[marius10p]

I am not sure what your first error is from, restart Matlab and try again.

The second error was a bug, I fixed it now.

[Idavr]

Thanks for helping trouble shooting this quickly. The initial experience so far and the visualization of the drift in KS2.5 is very good.

The same error regarding the gpuArray/nan is still persisting even after restarting Matlab, and happens at the stage of optimizing templates. It is now also in the GUI, in the form of this error: "Error running kilosort! An unexpected error occurred trying to launch a kernel. The CUDA error was: invalid configuration argument" and then it tries to save the data but throws the "Error saving data for phy! Reference to non-existent field "Wphy"." error. Might it be due to some channels without any spikes?

[marius10p]

Let's ignore the GUI for now. Am I right that this error happens on the very first batch during the optimization pass? I think it could only happen if there are no spike at all in the first batch (2 sec by default), and then mu is empty. Could you copy paste the entire output of Kilosort from start all the way up to this error?

[Idavr]

It does appear that way. This is the entire output of Kilosort running this file:

main_kilosort
Looking for data inside C:\Users\idavalik\Data\Neuropixels\Longtail_AM_2_on_2_off_g0_imec0
Time 0s. Computing whitening matrix..
Getting channel whitening matrix...
Channel-whitening matrix computed.
Time 14s. Loading raw data and applying filters...
Time 469s. Finished preprocessing 1374 batches.
pitch is 20 um
0.48 sec, 1 batches, 543 spikes
37.24 sec, 101 batches, 75711 spikes
73.90 sec, 201 batches, 154742 spikes
109.99 sec, 301 batches, 291572 spikes
146.10 sec, 401 batches, 409492 spikes
182.03 sec, 501 batches, 490777 spikes
217.80 sec, 601 batches, 551520 spikes
253.26 sec, 701 batches, 598854 spikes
289.03 sec, 801 batches, 664429 spikes
324.95 sec, 901 batches, 712231 spikes
360.99 sec, 1001 batches, 737831 spikes
396.97 sec, 1101 batches, 762301 spikes
432.98 sec, 1201 batches, 781602 spikes
469.14 sec, 1301 batches, 796781 spikes
495.57 sec, 1374 batches, 809788 spikes
time 1031.20, Shifted up/down 1374 batches.
Time 1044s. Optimizing templates ...
Error using gpuArray/nan
An unexpected error occurred trying to launch a kernel. The CUDA error was:
invalid configuration argument

Error in median (line 71)
y = nan(s,'like',x);

Error in learnTemplates (line 248)
toc, ibatch, niter, Nfilt, sum(nsp), median(mu), numel(st0), ndrop)

Error in learnAndSolve8b (line 35)
rez = learnTemplates(rez, rez.iorig);

Error in main_kilosort (line 45)
rez = learnAndSolve8b(rez, iseed);

[marius10p]

Based on that output, it looks like the first batch does have spikes, so I am not sure what the error is. You can do a few things:

1. put a debug stopping point and see what mu and other variables are when the error happens.
2. try to spike sort a subset of the data using the time limits in the GUI. We want to know if the problem is specific to a particular section of your data, or happens with any subset of your data.

[Idavr]

I am testing the debugging but this is my first time debugging Matlab (or anything, for that matter) so it might take me some doing. However, I did try your second suggestion. There was no trouble running Kilosort in the time range of 0 to 600 sec, but when doing 500 to 2000 sec, I got the same error in the GUI as mentioned above:

"Error running kilosort! An unexpected error occurred trying to launch a kernel. The CUDA error was: invalid configuration argument" and then it tries to save the data but throws the "Error saving data for phy! Reference to non-existent field "Wphy"."

Edit: Another peculiarity: I tried running the same file - only artifact corrected - and it was kilosorted without any issues.