electrical/optical artifact crashes kilosort2

[markagrios]

hi! I'm trying to process a data set with kilosort2 where we optogenetically inactivated neurons we were recording with the neuropixel probe. Each laser pulse creates a large artifact that is not filtered out entirely by the whitening done before running the program. Kilosort2 can run through most of the steps but crashes during the first splits 1/2. From the documentation "The first pass is initialized with the top principal component of the cluster", does kilosort2 fail here because maybe the cluster associated with the artifact is so distance in PCA space that it throws everything else off? I'm wondering if I need to do a preprocessing step before running the data through kilosort2 or if there are setting I can change that will fix this. Thank you!!

[hjalitto]

We have a similar issue for data using electrical stimulation with neuropixels. We have been working on a solution, but because the stimulus artifact saturates the amplifier it is difficult to remove the ringing. If you are interested, contact me and I can put you in touch with two post docs that are working on removing artifacts that are relatively stereotyped (like e-stim and opto). Henry

…

On Fri, Dec 6, 2019 at 8:07 AM Mark Agrios ***@***.***> wrote: hi! I'm trying to process a data set with kilosort2 where we optogenetically inactivated neurons we were recording with the neuropixel probe. Each laser pulse creates a large artifact that is not filtered out entirely by the whitening done before running the program. Kilosort2 can run through most of the steps but crashes during the first splits 1/2. From the documentation "The first pass is initialized with the top principal component of the cluster", does kilosort2 fail here because maybe the cluster associated with the artifact is so distance in PCA space that it throws everything else off? I'm wondering if I need to do a preprocessing step before running the data through kilosort2 or if there are setting I can change that will fix this. Thank you!! [image: badpoint] <https://user-images.githubusercontent.com/6403483/70337035-edb72400-180f-11ea-89db-d693d718f688.PNG> [image: filtered_badpoint] <https://user-images.githubusercontent.com/6403483/70337037-edb72400-180f-11ea-94e8-81a2137806b6.PNG> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#154>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AK5IFKMXSLFRU3AF7QH2L4TQXJ2FNANCNFSM4JW3P33Q> .

-- Henry Alitto, PhD Project Scientist Center for Neuroscience University of California, Davis

[markagrios]

Hello! Yes I would be interested! I think there are a decent amount of people doing cortical stimulation while recording from neuropixels so this is definitely an issue I would be like to have solved. You can reach me at magrios[at]gmail[dot]com

…

On Fri, Dec 6, 2019 at 2:16 PM hjalitto ***@***.***> wrote: We have a similar issue for data using electrical stimulation with neuropixels. We have been working on a solution, but because the stimulus artifact saturates the amplifier it is difficult to remove the ringing. If you are interested, contact me and I can put you in touch with two post docs that are working on removing artifacts that are relatively stereotyped (like e-stim and opto). Henry On Fri, Dec 6, 2019 at 8:07 AM Mark Agrios ***@***.***> wrote: > hi! I'm trying to process a data set with kilosort2 where we > optogenetically inactivated neurons we were recording with the neuropixel > probe. Each laser pulse creates a large artifact that is not filtered out > entirely by the whitening done before running the program. Kilosort2 can > run through most of the steps but crashes during the first splits 1/2. From > the documentation "The first pass is initialized with the top principal > component of the cluster", does kilosort2 fail here because maybe the > cluster associated with the artifact is so distance in PCA space that it > throws everything else off? I'm wondering if I need to do a preprocessing > step before running the data through kilosort2 or if there are setting I > can change that will fix this. Thank you!! > > [image: badpoint] > < https://user-images.githubusercontent.com/6403483/70337035-edb72400-180f-11ea-89db-d693d718f688.PNG > > [image: filtered_badpoint] > < https://user-images.githubusercontent.com/6403483/70337037-edb72400-180f-11ea-94e8-81a2137806b6.PNG > > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > < #154 >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AK5IFKMXSLFRU3AF7QH2L4TQXJ2FNANCNFSM4JW3P33Q > > . > -- Henry Alitto, PhD Project Scientist Center for Neuroscience University of California, Davis — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#154>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABQ3LG5LHH76IPHJA4XOUPLQXKXIJANCNFSM4JW3P33Q> .

[marius10p]

@markagrios Please post the entire command line output for the error. Unless you are doing stimulation every half second or more frequently, those events should be mostly assigned to their own cluster of outliers.

The PCA waveforms are stored in rez.ops.wPCA, so you could plot those and post here for us to check them.

[marius10p]

I see now that this is a duplicate issue. Why do you think the artifact is crashing it? I see no a priori reason for it, nor did we encounter similar problems before.

[Idavr]

Hello @hjalitto!

This is an old thread, but I am combing through the forums looking for people that have experience with KiloSort2, phy2 and optogenetics. Are you guys still working on this or have found a solution? My KS2 does not crash, but there are some funky looking templates in phy2 and some results that don't make sense that I am trying to figure out. Any input/information would be highly appreciated!

[markagrios]

Hi! Yeah we're still stuck, Adam ( Adamforrest2020@u.northwestern.edu) is looking into it more now. I did find some github repo <https://djoshea.github.io/neuropixel-utils/> that has some tools to maybe clean the data before running it through kilosort but I haven't looked at it much in depth yet. Cheers, Mark Agrios

…

On Thu, Oct 8, 2020 at 1:52 PM Idavr ***@***.***> wrote: Hello @hjalitto <https://github.com/hjalitto>! This is an old thread, but I am combing through the forums looking for people that have experience with KiloSort2, phy2 and optogenetics. Are you guys still working on this or have found a solution? My KS2 does not crash, but there are some funky looking templates in phy2 and some results that don't make sense that I am trying to figure out. Any input/information would be highly appreciated! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#154 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABQ3LG7HE3ATCCESXGA27N3SJYDAVANCNFSM4JW3P33Q> .

[Idavr]

Hi Mark, Thank you so much for this, a quick read-through and it seems like this will definitely help with my issues. I’ll start taking a closer look and testing it on the data sets I have. Again, thanks for the tip and the quick reply! Best, Ida 8. okt. 2020 kl. 21:09 skrev Mark Agrios <notifications@github.com>:  Hi! Yeah we're still stuck, Adam ( Adamforrest2020@u.northwestern.edu) is looking into it more now. I did find some github repo <https://djoshea.github.io/neuropixel-utils/> that has some tools to maybe clean the data before running it through kilosort but I haven't looked at it much in depth yet. Cheers, Mark Agrios

On Thu, Oct 8, 2020 at 1:52 PM Idavr ***@***.***> wrote: Hello @hjalitto <https://github.com/hjalitto>! This is an old thread, but I am combing through the forums looking for people that have experience with KiloSort2, phy2 and optogenetics. Are you guys still working on this or have found a solution? My KS2 does not crash, but there are some funky looking templates in phy2 and some results that don't make sense that I am trying to figure out. Any input/information would be highly appreciated! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#154 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABQ3LG7HE3ATCCESXGA27N3SJYDAVANCNFSM4JW3P33Q> .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub<#154 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALMPW3YMT4DSAE3J32NQQ23SJYFABANCNFSM4JW3P33Q>.

[marius10p]

Closing this issue, since it is no longer about artifacts crashing kilosort2. If you want to continue the conversation, please open another issue about how to preprocess and fix artifacts before plugging the data file into Kilosort.