any news on when feature view will be back ?

[jrnvndtslr]

I've noticed that KS3 outperforms older versions, but I'm having trouble doing the manual curation in Phy without the feature view (see issue #317).

Any news on when this functionality will be back? Or possibly a temporary workaround?

Thanks for the great program!

[czuba]

@jrnvndtslr, #317 & Phy issue 1071,

Feature view is functional on KS25, a rewritten version based on Kilosort 2.5

[Idavr]

I'd like to also add the interest in an update on this. Yes, the feature view is functional in KS2.5, but for those of us who like the performance of KS3 and would like to use that for spike-sorting any idea of a potential timeline for the enabling of the remaining phy functionalities would help. When comparing KS2.5 with 3 I get better sorting results on my data, which is why going back to KS2.5 is something I would like to avoid. Your work is simply too good.

[mdiamantaki]

Hi, is there any update on this issue? It would be very helpful to have the feature view for data sorted with kilosort3. Especially for splitting/merging units is essential. Any plans for adding it in the future?

[cowenowner]

Hi, I am liking version 3 but I am having the same problem with FeatureView not opening.
Thanks, Stephen

[cowenowner]

It does seem to boil down to the rez.cProj and rez.cProjPC being empty. I think that the workaround is to generate the waveforms in phy. I did this and the FeatureView appeared. Do this through > phy extract-waveforms params.py

[Idavr]

Thank you so much @cowenowner , this workaround worked perfectly for me!

[FelixAntonSchneider]

Thank you very much @cowenowner.

[benefron]

@cowenowner I ran the command but it is not showing the feature view. Do I need to do it prior to running kilosort or after?

[LaurenzMuessig]

@benefron For me it only worked after resetting the gui state (--clear-state flag). Also, it didn't work if I ran it on a dataset that was sorted in Phy previously unless deleting the .phy folder and recovering the files from kilosort that get overwritten by phy (spike_clusters.npy etc.).
Laurenz

[benefron]

Still not giving me the feature view and the feature template view

[LaurenzMuessig]

You can try and delete the .json file in C:\Users\Laurenz\.phy\TemplateGUI. Also when you run it, does it generate 3 npy files named
_phy_...

[benefron]

Nope it generates 2 files:
spikes.waveformsChannels.npy
spikes.waveforms.npy

[mdiamantaki]

I get the same as @benefron. Same npy files and no feature/feature template view.

If I may ask, what files does it generate for you @LaurenzMuessig? I also tried deleting the .json file and using the --clear-state flag, the result was the same.

Any help is more than welcome!

[cowenowner]

@cowenowner I ran the command but it is not showing the feature view. Do I need to do it prior to running kilosort or after?

Huh. I don't know. I just ran >phy extract-waveforms params.py after running kilosort and before running Phy. Maybe the waveform files are somehow going to a different directory?

[LaurenzMuessig]

It generates 3 files for me:
_phy_spikes_subset.channels.npy
_phy_spikes_subset.spikes.npy
_phy_spikes_subset.waveforms.npy

Otherwise I am doing the same as @coweowner. It only didn't work for me the first time, but either using the --clear-state flag or deleting the .json file solved this for me

[mdiamantaki]

@cowenowner what npy files do you get?

when I run the phy extract-waveforms params.py (after Kilosort and before phy) I get this message in the terminal: model:1113 Extract waveforms to spikes.waveforms.npy and spikes.waveformsChannels.npy...

[LaurenzMuessig]

Enter Path to Data: Z:\current_curation\r1010_20211007T033349
12:29:51.994 [W] model:591 Unreferenced clusters found in templates (generally not a problem)
12:29:53.045 [W] model:613 Unreferenced clusters found in spike_clusters (generally not a problem)
12:29:53.621 [W] model:655 Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed.
Extracting waveforms: 100%|██████████████████████████████████████████████████| 6192.64/6192.64 [01:15<00:00, 81.85it/s]
12:31:18.192 [W] model:591 Unreferenced clusters found in templates (generally not a problem)
12:31:19.227 [W] model:613 Unreferenced clusters found in spike_clusters (generally not a problem)

[LaurenzMuessig]

this is my terminal output

[mdiamantaki]

maybe it is different depending on the operating system? I use Linux.
I will try to run it in a Windows pc and see what I get.
thanks @LaurenzMuessig

[benefron]

I am also using linux

[cowenowner]

FYI - I use Windows10

[Zoe0793]

@benefron, for me the feature view did not automatically show after running the waveform extraction, I had to go to view, add feature view. Apologies if this is obvious but thought it was worth to mention just in case.

[metmic]

Thanks for sharing this great workaround. I was finally able to get the feature view back for KS3. I however noticed that when I split a cluster using the K command after drawing a polygon in the feature view, the coloring in the different view windows doesn't agree (it's somehow inverse). The cluster that I select, which should be blue and have the most datapoints, appears as highlighted in red in all other views. Also, the WaveformView for the splitted cluster with the more datapoints then shows only a few spike waveforms as if some files are not correctly updated.
Any ideas what happens


?

[cowenowner]

Just to confirm, I also have the same problem as described by @metmic above regarding the workaround so be wary of any manually defined cluster boundaries and clusters in Phy if using the 'extract-waveforms' workaround. I will probably avoid cutting in FeatureView as a result. As another work-around, I will likely write a post-kilosort matlab function that will allow users to do a final convex-hull clean-up of the clusters previously identified and merged in Phy.

[GrimmSnark]

Thanks for your advice, I have been trying to get this working for a while!
@cowenowner I would be really interested in using the post-processing script you mentioned when you get it finished if you do not mind sharing it.

[farazmoradi]

Hi, I have reached the same problem! @cowenowner @metmic Did you find a way to fix it?

[cowenowner]

@farazmoradi Sadly, no progress on my end.

[farazmoradi]

@marius10p Could you please let us know when this feature will be added to KS3?

[Datamance]

The workaround only works for FeatureView and not TemplateFeatureView. Is there another recommended workaround for the latter?

[kipkeller]

Agree with all of the above (especially the issues using 'K' in FeatureView) - makes phy2 much less useful. Has anyone made progress - or should I revert back to phy1 ?

[Zhili-Qu]

For those of you who want to show all the data points in FeatureView (the default maximum is 500), you can change the cap in the file called init.py in the "apps" folder of your phy folder. I used to only be able to cut at most 500 noisy spikes from the cluster even if I used "K" to cut all of the points in the window. Now I changed this cap to 20000, I can now see all of the data points in my FeatureView (you can change this to a higher number but it will take more time to load)

[marius10p]

Sorry folks, the feature will never be added to KS3 (no time for me to do that). It will be added to KS4 when it releases in python.

[metmic]

And when ist that KS4 release date? Michael Metzen

…

________________________________ From: Marius Pachitariu ***@***.***> Sent: Thursday, September 7, 2023 2:56:27 AM To: MouseLand/Kilosort ***@***.***> Cc: Michael Metzen ***@***.***>; Mention ***@***.***> Subject: Re: [MouseLand/Kilosort] any news on when feature view will be back ? (#429) Sorry folks, the feature will never be added to KS3 (no time for me to do that). It will be added to KS4 when it releases in python. — Reply to this email directly, view it on GitHub<#429 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AMSZLQ3TTBDUIATY7W6QX6TXZFVZXANCNFSM5A3TEYOA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[MatiMugnaini]

Hi everyone.

I tried the workaround for getting back the FeatureView (> phy extract-waveforms params.py). I'm getting this message:

'09:48:33.425 [W] traces:458 File F:\PLAY BACKUP\2023-10-19_11-20-04\Record Node 101\experiment1\recording1\continuous\Neuropix-PXI-100.0\temp_wh.dat does not exist.
09:48:33.445 [W] model:1266 Spike waveforms could not be extracted as the raw data file is not available.'

However, as you can see in the caption, the temp_wh.dat is right there in the directory. Can anyone help me with this? I tried all the things you commented before.

Mati Mugnaini

[kipkeller]

just wondering if this is on a remote server and it's taking too long to load - so it times out. Much faster if you have the few necessary files on a local SSD. Just an idea... Kip

…

________________________________ From: MatiMugnaini ***@***.***> Sent: Tuesday, November 28, 2023 1:11 AM To: MouseLand/Kilosort ***@***.***> Cc: kipkeller ***@***.***>; Comment ***@***.***> Subject: Re: [MouseLand/Kilosort] any news on when feature view will be back ? (#429) Hi everyone. I tried the workaround for getting back the FeatureView (> phy extract-waveforms params.py). I'm getting this message: '09:48:33.425 [W] traces:458 File F:\PLAY BACKUP\2023-10-19_11-20-04\Record Node 101\experiment1\recording1\continuous\Neuropix-PXI-100.0\temp_wh.dat does not exist. 09:48:33.445 [W] model:1266 Spike waveforms could not be extracted as the raw data file is not available.' However, as you can see in the caption, the temp_wh.dat is right there in the directory. Can anyone help me with this? I tried all the things you commented before. Mati Mugnaini temp_whdat.existence.PNG (view on web)<https://github.com/MouseLand/Kilosort/assets/35610732/58f97be5-8112-4023-852d-9ccd889a5fea> — Reply to this email directly, view it on GitHub<#429 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AHZUXWAZTOLQYMRK2H6EOHTYGWTEJAVCNFSM5A3TEYOKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBSHEZTSMZSGMZA>. You are receiving this because you commented.Message ID: ***@***.***>

[Zhili-Qu]

@MatiMugnaini you can open your params.py in an editor and check if the first line in your params.py file is indeed the correct pathway for temp_wh.dat.

[MatiMugnaini]

Thanks so much @Zhili-Qu . That was exactly the problem.

[Alchemist-Y]

Sorry folks, the feature will never be added to KS3 (no time for me to do that). It will be added to KS4 when it releases in python.

but i still can‘t open the template feature view in KS4

[marius10p]

Kilosort4 works differently from previous versions and the template feature view is not as useful. We will not prioritize adding this, but we should make a note of it @jacobpennington .