IndexError: index 41 is out of bounds for axis 0 with size 36

[asheff794]

I'm trying to curate the results of kilosort4 using phy2, but on some sessions I am getting index out of bound errors when I try to open the gui. The index numbers provided relate to the number of clusters identified. In the example session I'm posting about here, 36 refers to the number of sorted clusters in spike_clusters.npy (haven't identified where the 41 comes from yet).

I've tried sorting both with the kilosort4 api and spikeinterface's wrapper and both experience the same issue. I've also tried installing phy2 from the environment.yml file and this doesn't change anything either compared to the README instructions. So far I haven't identified anything consistent about which sessions fail to load, it feels random so far. Here is the full error, let me know if any other kilosort output files would be helpful in debugging.

(phy2)[kilosort4]$ phy template-gui params.py
08:38:21.107 [E] init:62 An error has occurred (IndexError): index 41 is out of bounds for axis 0 with size 36
Traceback (most recent call last):
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/bin/phy", line 8, in
sys.exit(phycli())
^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/phy/apps/init.py", line 159, in cli_template_gui
template_gui(params_path, **kwargs)
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/phy/apps/template/gui.py", line 209, in template_gui
model = load_model(params_path)
^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/phylib/io/model.py", line 1433, in load_model
return TemplateModel(**get_template_params(params_path))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/phylib/io/model.py", line 339, in init
self._load_data()
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/phylib/io/model.py", line 419, in _load_data
self.sparse_clusters = self.cluster_waveforms()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/phylib/io/model.py", line 1319, in cluster_waveforms
mean_waveform = self.get_cluster_mean_waveforms(clust, unwhiten=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/phylib/io/model.py", line 1215, in get_cluster_mean_waveforms
template = self.get_template(best_template, unwhiten=unwhiten)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/phylib/io/model.py", line 959, in get_template
return self._get_template_dense(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/phylib/io/model.py", line 881, in _get_template_dense
template_w = self.sparse_templates.data[template_id, ...]
~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
File "/gpfs/gibbs/project/jadi/ags72/anaconda/conda_envs/phy2/lib/python3.11/site-packages/numpy/core/memmap.py", line 335, in getitem
res = super().getitem(index)
^^^^^^^^^^^^^^^^^^^^^^^^^^
IndexError: index 41 is out of bounds for axis 0 with size 36

[zm711]

I would recommend bringing this up on Kilosort repo. On the SpikeInterface side we have seen multiple cases of Kilosort return spikes both before and after the recording time which can lead errors like this, but I'm not sure if this is the cause. You could try scur.remove_excessive_spikes(sorting, recording) followed by export_to_phy and see if that fixes it. Otherwise this seems more like a problem with the KS output that they would need to know about.

Just wanted to point out I had a typo in the curation function. You need to give it the recording too for it to know which spikes are excessive :)

[asheff794]

Thanks @zm711, running remove_excessive_spikes did solve the problem! For me it was the spike times occuring after the recording period, as I had other sessions with negative spike times and those didn't seem to cause any issues.

I raised the issue on the Kilosort repo so they're aware of the problem. Thanks again!