"NotImplementedError: This meta file version of spikeglxis not implemented" error when running read_spikeglx() and get_neo_streams() on npx data

[NDolensek]

I'm trying to follow the "Analyse Neuropixels datasets" tutorial and am getting stuck at the first step when trying to analyze my own neuropixels data acquired with Neuropixel 1.0-NHP probes and SpikeGLX, with default settings. Data folder contains .imec0.ap.meta, .imec0.ap.bin, .imec0.lf.bin and .imec0.lf.meta files. Running either si.get_neo_streams or si.read_spikeglx() results in a "NotImplementedError: This meta file version of spikeglxis not implemented".

Does data need to be in a specific format for the spikeinterface to work?
FYI, binary files acquired with Open Ephys using the same probes work fine using si.read_openephys().

stream_names, stream_ids = si.get_neo_streams('spikeglx', path) stream_names

NotImplementedError                       Traceback (most recent call last)
Cell In[7], line 1
----> 1 stream_names, stream_ids = si.get_neo_streams('spikeglx', path)
      2 stream_names

File [c:\Users\ndolensek\mambaforge\envs\spikeinterface\lib\site-packages\spikeinterface\extractors\neoextractors\neo_utils.py:26](file:///C:/Users/ndolensek/mambaforge/envs/spikeinterface/lib/site-packages/spikeinterface/extractors/neoextractors/neo_utils.py:26), in get_neo_streams(extractor_name, *args, **kwargs)
      5 """Returns the NEO streams (stream names and stream ids) associated to a dataset.
      6 For multi-stream datasets, the `stream_id` or `stream_name` arguments can be used 
      7 to select which stream to read with the `read_**extractor_name**()` function.
   (...)
     23     List of NEO stream ids
     24 """
     25 neo_extractor = get_neo_extractor(extractor_name)
---> 26 return neo_extractor.get_streams(*args, **kwargs)

File [c:\Users\ndolensek\mambaforge\envs\spikeinterface\lib\site-packages\spikeinterface\extractors\neoextractors\neobaseextractor.py:142](file:///C:/Users/ndolensek/mambaforge/envs/spikeinterface/lib/site-packages/spikeinterface/extractors/neoextractors/neobaseextractor.py:142), in NeoBaseRecordingExtractor.get_streams(cls, *args, **kwargs)
    139 @classmethod
    140 def get_streams(cls, *args, **kwargs):
    141     neo_kwargs = cls.map_to_neo_kwargs(*args, **kwargs)
--> 142     neo_reader = get_reader(cls.NeoRawIOClass, **neo_kwargs)
    144     stream_channels = neo_reader.header['signal_streams']
    145     stream_names = list(stream_channels['name'])

File [c:\Users\ndolensek\mambaforge\envs\spikeinterface\lib\site-packages\spikeinterface\extractors\neoextractors\neobaseextractor.py:14](file:///C:/Users/ndolensek/mambaforge/envs/spikeinterface/lib/site-packages/spikeinterface/extractors/neoextractors/neobaseextractor.py:14), in get_reader(raw_class, **neo_kwargs)
...
    407                                   'is not implemented')
    408 else:
    409     stream_kind = ''

NotImplementedError: This meta file version of spikeglxis not implemented

[alejoe91]

Hi @NDolensek

What version pf SpikeGLX are you using? Would you mind sharing the meta file with us?
Maybe a recent SpikeGLX update will require a patch on our side.

Cheers
Alessio

[NDolensek]

Hi Alessio, thanks for such a quick reply! SpikeGLX version is the most recent one available on the website, release v20230120-phase30. I'm attaching the .ap.meta file here (zipped due to github file type restrictions):
rec1_g0_t0.imec0.ap.zip
Nate

[alejoe91]

The imDatPrb_type in the .meta file is 1030, which we haven't seen before. Currently we support these types:

• 0 (NP1)
• 21 (NP2 - single shank)
• 24 (NP2 - 4 shanks)

See our implementation

Note that these are the official supported versions in SpikeGLX: https://billkarsh.github.io/SpikeGLX/Sgl_help/Metadata_30.html (search for imDatPrb_type)

What probe are you using? Is it a standard Neuropixels? Maybe in recent versions, SpikeGLX is using a different probe ids for NP1.0.

[NDolensek]

The probes we are using are definitely a bit non-standard, they are 45mm long 1.0-NHP probes (https://www.biorxiv.org/content/10.1101/2023.02.01.526664v2)

[alejoe91]

I see. We're in the process of supporting the new probes, which requires changes both on the NEO and the probeinterface side.

Can you confirm that the probe type is the same as 1.0, in terms of channel gains and so on?

We found a discrepancy between the file you shared and the file that @JonathanAMichaels shared with us. In Jonathan's case, the imDatPrb_type is 0, in your case it's 1030. @JonathanAMichaels do you know if the imDatPrb_type changed in recent versions of SpikeGLX?

[NDolensek]

We checked with Bill Karsh, who develops SpikeGLX:

The NP1030 is the same as the NP1000 electrically {refs, gains, filters, bands}.
The site pitch (spacing) is the same vertically (32 microns along the shank), but the NP1030 is 55 microns wider.

Also, the paper I linked above has this to say:

The Neuropixels 1.0-NHP probe uses the same signal-conditioning circuits as the Neuropixels 1.0 probe1, integrating 384 low-noise readout channels with programmable gain and 10-bit resolution

So it sounds like treating gain in the same way as with the standard NP1 probes should work fine.

Regarding the discrepancy in imDatPrb_type between us and @JonathanAMichaels, that could be caused by our probes differing, he might've recorded with an older, passive NHP probe, or a 25mm version, versus our 45mm one. SpikeGLX might've also changed handling of this over time, with NHP probes becoming more available, to distinguish the standard NP1 probes from the 1.0-NHP ones with same electrical specs but different pad spacing.

The NHP probes only became commercially accessible today, so there were a couple of different version floating around while development was ongoing. Probe version in our hands should be the closest, if not identical, to the ones becoming broadly available. (edit: commercially available NHP probes are the same as our 45mm NHP probes)

[alejoe91]

Thanks, this is helpful. Let me try to intergate this info with @JonathanAMichaels one from this issue: SpikeInterface/probeinterface#155

So from this document (https://www.neuropixels.org/_files/ugd/328966_97dfdcf9421a41ae9c57c9047928834c.pdf) there are three types of NP1.0-NHP:

• short
• medium
• long

I don't see a 1030 code, but the closest would be the NP1032 in the "order table", which corresponds to the "long" version. @NDolensek can you confirm?

Jonathan's .meta file has 1015 (presumably the short shank, @JonathanAMichaels can you confirm?).

So we could do something like this to differentiate between the probes:
If imDatPrb_pn is a field in the .meta file, we'll use that:

• NP101*: NHP-short (x-pitch=32, 960 electrodes - like NP2)
• NP102*: NHP-medium (x-pitch=103, 2496 electrodes)
• NP103*: NHP-long (x-pitch=103, 4416 electrodes)

Does that sound correct?

Thank you for helping us out with this! It's not so straightforward to gather all these info for the newly developed models :)

[JonathanAMichaels]

That sounds correct to me. I am using the NHP short. The only thing to add is that none of these new probes use a ‘staggered’ electrode design, which is currently assumed in the code. Thanks Edit: email client unecessary addition removed.

[alejoe91]

@JonathanAMichaels indeed! What we will do is explicitly add the new probes here and using a stagger=0 accordingly.

Thank you for the help!

[NDolensek]

Sounds great, thank you @JonathanAMichaels and @alejoe91!

[samuelgarcia]

Nate : thank you for all the details and Alessio for your insvestigation.