Support for multiple Sampling rates for Neuralynx Recordings

[manimoh]

Currently the NeuralynxRawIO.py doesn't support sessions where there are continuously sampled files with multiple sampling rates. Is there a plan to add this feature in the next release?

[samuelgarcia]

@PeterNSteinmetz @JuliaSprenger : any knwoledge on how neuralynx handle multi sample rate ?
In my mind the rate was global and not per channel.

[JuliaSprenger]

Hi @manimoh!
Currently only ncs files with a common sampling rate are accepted as we didn't encounter mixed sampling rates yet. If this is a common output of your system we can add that feature. Would it be possible for you to provide a few small ncs files from one recording that do not share a sampling rate to be added to the public set of test files here?

[manimoh]

Hi @JuliaSprenger!
I created an issue here, but I can't seem to upload the ncs files even when I decrease the size to less than 4 MB
https://gin.g-node.org/NeuralEnsemble/ephy_testing_data/issues/59

[JuliaSprenger]

Thanks for your willingness to provide test data files. For a one time contribution you can just send me the files via email, including the attribution of the files to be listed in the readme.

[manimoh]

Of course! And thank you along with all the other creators of this wonderful tool!

I also emailed the files to you - let me know if you need something else!

[PeterNSteinmetz]

Firstly, a temporary workaround for this is to use either the single file mode and create a separate rawio for each .Ncs file or the multi-file mode and list a group of files all with the same sampling rate.

Secondly, remember that in the RawIO model, all AnalogSignals within a RawIO have the same sampling rate (https://neo.readthedocs.io/en/stable/rawio.html).

This latter suggests that the way to attack this issue would be to provide a Neo IO, rather than RawIO, which would handle the construction of multiple RawIO objects, grouping the .Ncs files appropriately for different sampling rates. This would likely be most easily handled by adding a static builder method, not a constructor, to the NeuralynxRawIO class, since all of the code to scan files and determine sampling rates is presently in and used by the RawIO.

[manimoh]

Just to further clarify our particular use-case, we are trying to use neo via Spikeinterface. See attached screenshot:

I am guessing that the change has to be on neo's side based on the stack trace, but if you think there is something we could do from the SpikeInterface's side, I am happy to take it up with them!

[PeterNSteinmetz]

This will have to do with the way that SpikeInterface is calling the Neo NeuralynxRawIO class.

A workaround will require calling and using that class directly rather than through SpikeInterface.

[manimoh]

I see! But it still makes sense to implement this feature in neo, right? If not, then I can close the issue!

[PeterNSteinmetz]

This feature could be implemented at the Neo IO core level, which is likely the most appropriate. It could also be implemented by SpikeInterface making different calls onto the RawIO classes. Really a question for @samuelgarcia or @JuliaSprenger.

I have worked mostly on the NeuralynxRawIO and getting it to work with the different variety of Neuralynx files. If you need something in the shorter term, suggest writing code to manipulate the current versions of the NeuralynxRawIO classes.

[JuliaSprenger]

Thanks for the files. I will have a look into extending the IO.

[TheChymera]

Hi @JuliaSprenger , thank you for looking into this. Do you have a draft you're working on or a branch for this? We'd be happy to contribute since this is an immediate priority for us. But not having worked on neo before, I assume any work we do independently might just end up being a slower and worse version of yours :)

[PeterNSteinmetz]

For immediate use, I would suggest modifying your code to make calls directly using the RawIO.NeuralynxRawIO class. You can construct as many of these objects as you need to accommodate the differing sampling rate groups of files.

[bendichter]

@manimoh thanks for the report. I'm trying to figure out how all of these pieces will fit together. Can you explain your use-case a little further? Why do you have files with different sampling rates? For instance, are some of them voltage series from electrodes in the brain and some of them something else?

[manimoh]

Hi @bendichter, you are right! We use different sampling rates to collect data from other sensors and feed in directly into the Neuralynx acquisition system, for instance: running speed from a running wheel encoder.

[bendichter]

OK, well in that case I think the best solution would be to sub-select only the neural channels. It doesn't make sense for SpikeInterface to ingest non-neural data. @JuliaSprenger is it possible to sub-select these channels using the current raw neo reader? Could we set up a mechanism to pass this list in from the SpikeInterface call?

[manimoh]

I had opened an issue sometime back for this: #889

[bendichter]

@manimoh ah yes, #889 from 1 yr ago is exactly what I was looking for. Thanks. Would you say that we could close this issue in favor of that one?

[manimoh]

@bendichter I would say that both the issues have their merit! #889 serves the Spikeinterfaces pipeline better while this issue is good for Neo being more flexible to handle all kinds of Neuralynx acquisitioned data :)

[manimoh]

Another use case for multiple sampling rates is that there are experiments (including ours) which use real time spectral analysis and sometimes that software that does this doesn't play well with the wideband sampling that is needed for Automatic spike sorting. In this case, we physically copy the required signals on the acquisition box to an unused input channel on the box and this copied signal can then be sampled at a rate which is compatible with the online spectral analysis tool.
Now this software can stream this differently sampled signal from Neuralynx's acquisition software and do its work. It is worthwhile for us to save this signal too for post hoc analysis.

[samuelgarcia]

In neo we support the "multi stream" for signals.
In spikeinterface when there are several stream we need so to specify which one with the stream_id.