Support separate files per recording option in NeuralynxRawIO

[alafuzof]

Hi,

Since version 5.7.0 Neuralynx Cheetah has supported an option for creating "separate files per recording". When stopping and restarting a recording within the same acquisition with this option enabled, Cheetah creates multiple files for each acquisition entity as well as for events. Thus, you get a folder like the following, where CSC.ncs corresponds to the first recording and so on:

CSC.ncs
CSC_0001.ncs
CSC_0002.ncs
Events.nev
Events_0001.nev
Events_0002.nev

Currently NeuralynxRawIO reads directories like this as containing multiple signal channels for the same acquisition entity (e.g. signal_channels: [CSC, CSC, CSC] in the example above). Ideally, I would like to have separate segments for each set of files. Nota bene: as far as I know there's nothing that rules out discontinuities within a given .ncs file (e.g. if the recording system experiences a hiccup) - so proper handling of these directories requires handling both the separate files and any discontinuities within them.

I would be happy to work on adding support for these recordings, but I would like to hear your suggestions with regards to how to integrate the file handling to the existing code base. @samuelgarcia @JuliaSprenger any comments?

[JuliaSprenger]

Hi @alafuzof!
I didn't know about this new Neuralynx feature, until now I always considered all files of a folder to be recorded simultaneously. When stopping and restarting the recording with the same acquisition, does the global time reset? Or is there an option for this and it's up to the user?

[alafuzof]

Hi @JuliaSprenger!
I only became aware of that option recently, when a co-worker enabled it and broke some of our analysis pipeline 😄 The timestamp clock is not reset on restarting the recording, in fact I think it only resets if you power down the DigitalLynx box (or whatever variant you may have), so I guess you could have multiple recording folders that share the same time base, if you restart Cheetah or open a new folder for acquisition.

[JuliaSprenger]

Ok, that's good news since this means we could in principle check if the files within a folder do belong to different subsessions based on their time stamps and then generate one Segment for each subsession.

[sin-mike]

@alafuzof this feature is sooooo nasty. When we upgraded our cheetah, all our pipelines broke too, in both matlab and python. We then made an agreement to uncheck this option and records in one file. As for the records present, we agreed to split them to separate folders.

[sin-mike]

Ok, that's good news since this means we could in principle check if the files within a folder do belong to different subsessions based on their time stamps and then generate one Segment for each subsession.

@JuliaSprenger
The separate files fit "blocks" concepts rather than "segments", as for one file there could be missed blocks which, in turn, are split in segments. But again, this feature in nlx is no nasty, that I would suggest splitting files manually and warning user of multiple files present instead. Btw, I've seen some events of broken numbering, e.g. when there is an _0001 record, but _0002 event. I consider this feature in nlx highly erroneous and misleading.