Now exclusive support for HS v0.4 (Lightning)

[mhhennig]

Adds support for the herdingspikes version 0.4, aka Lightning. As this version is intended to replace versions <0.4, support for older versions has been removed.

[zm711]

Cool. Any changes in install that need to be added to this doc?

[mhhennig]

Thanks! Changed the doc slightly.

Also added a tiny fix to the confusion matrix: unit number on one axis were shown as float, now cast to int

[chrishalcrow]

Hey Matthias, nice!

It's not running for me:

si.run_sorter(recording=rec, sorter_name="herdingspikes")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/nolanlab/Chris/Developing/fromgit/spikeinterface/src/spikeinterface/sorters/runsorter.py", line 216, in run_sorter
    return run_sorter_local(**common_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nolanlab/Chris/Developing/fromgit/spikeinterface/src/spikeinterface/sorters/runsorter.py", line 274, in run_sorter_local
    SorterClass.set_params_to_folder(recording, folder, sorter_params, verbose)
  File "/home/nolanlab/Chris/Developing/fromgit/spikeinterface/src/spikeinterface/sorters/basesorter.py", line 190, in set_params_to_folder
    if recording.is_filtered() and cls._check_apply_filter_in_params(params) and verbose:
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nolanlab/Chris/Developing/fromgit/spikeinterface/src/spikeinterface/sorters/external/herdingspikes.py", line 112, in _check_apply_filter_in_params
    return params["filter"]
           ~~~~~~^^^^^^^^^^
KeyError: 'filter'

It's checking for the filter param, but maybe you've gotten rid of it in v4?

[mhhennig]

Ok, now it's hopefully good!

[chrishalcrow]

Not quite - the sorting works but saving it doesn't. Something to do with expinds. I'm just generating a recording, maybe you can test it out locally:

import spikeinterface.full as si
rec, _ = si.generate_ground_truth_recording()
si.run_sorter(recording=rec, sorter_name="herdingspikes")

produces the error trace

/home/nolanlab/Chris/Developing/fromgit/HS2/herdingspikes/hs2.py:507: UserWarning: Number of channels too few for common median reference
  det = detectDataLightning(self.recording, self.params)
HSDetection: Analysing segment 0, frames from        0 to   250000  (0.0%)
writing spikes to /home/nolanlab/Chris/Developing/fromgit/spikeinterface/herdingspikes_output/sorter_output/HS2_detected.hdf5
Reading spikes from detection
Fitting dimensionality reduction using all spikes...
...projecting...
...done
Clustering...
Clustering 598 spikes...
requested -1 cpus
using 20 cpus
number of seeds: 21
seeds/job: 2
[Parallel(n_jobs=20)]: Using backend LokyBackend with 20 concurrent workers.
[Parallel(n_jobs=20)]: Done   3 out of  20 | elapsed:    0.0s remaining:    0.0s
[Parallel(n_jobs=20)]: Done  14 out of  20 | elapsed:    0.0s remaining:    0.0s
[Parallel(n_jobs=20)]: Done  20 out of  20 | elapsed:    0.0s finished
Number of estimated units: 5
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/nolanlab/Chris/Developing/fromgit/spikeinterface/src/spikeinterface/sorters/runsorter.py", line 216, in run_sorter
    return run_sorter_local(**common_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nolanlab/Chris/Developing/fromgit/spikeinterface/src/spikeinterface/sorters/runsorter.py", line 276, in run_sorter_local
    SorterClass.run_from_folder(folder, raise_error, verbose)
  File "/home/nolanlab/Chris/Developing/fromgit/spikeinterface/src/spikeinterface/sorters/basesorter.py", line 301, in run_from_folder
    raise SpikeSortingError(
spikeinterface.sorters.utils.misc.SpikeSortingError: Spike sorting error trace:
Traceback (most recent call last):
  File "/home/nolanlab/Chris/Developing/fromgit/spikeinterface/src/spikeinterface/sorters/basesorter.py", line 261, in run_from_folder
    SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose)
  File "/home/nolanlab/Chris/Developing/fromgit/spikeinterface/src/spikeinterface/sorters/external/herdingspikes.py", line 155, in _run_from_folder
    C.SaveHDF5(sorted_file, sampling=recording.get_sampling_frequency())
  File "/home/nolanlab/Chris/Developing/fromgit/HS2/herdingspikes/hs2.py", line 959, in SaveHDF5
    self._savesinglehdf5(filename, None, compression, sampling, transpose)
  File "/home/nolanlab/Chris/Developing/fromgit/HS2/herdingspikes/hs2.py", line 908, in _savesinglehdf5
    g.create_dataset("exp_inds", data=self.expinds)
                                      ^^^^^^^^^^^^
AttributeError: 'HSClustering' object has no attribute 'expinds'

[mhhennig]

Have I not pushed the fix earlier, @chrishalcrow? Does it work now?

[chrishalcrow]

Oh, sorry, my herdingspikes was out of date. All working now, and looking good!

[chrishalcrow]

Looks good - just a tiny default param documentation error.