Alternate name than 'neuropil' when saving Caiman denoised calcium traces?

[EricThomson]

First, thanks a lot for making neuroconv and roiextractors. I just started today and it was incredibly easy to use -- I got things working really quickly with it. I have an annoying question about nomenclature, as I'm relatively new to this I'm probably just missing something.

I'm planning to rebuild Caiman's save nwb functionality using neuroconv. My main question is about the nomenclature. If I understand correctly, the three time series being saved are Neuropil, Deconvolved, and DfOverF. These correspond to Caiman's estimates.C (denoised calcium traces), estimates.S (spike counts from deconvolution), and estimates.F_dff.

My main concern/question is whether there might be another name available within the nwb ophys universe for Caiman's denoised calcium traces in C than Neuropil (my concern is that 'neuropil' typically refers to background noise that nobody wants to look at)? E.g., is there some taxonomy of time series in nwb ophys that includes other types of calcium time series?

I apologize for bringing up a terminological rather than substantive concern here -- it is only because everything else is working so well!

[CodyCBakerPhD]

I apologize for bringing up a terminological rather than substantive concern here -- it is only because everything else is working so well!

We love these kinds of discussions. I will let @alessandratrapani explain our current understanding as we've been discussing exactly these types of things for the past ~4 months now

[alessandratrapani]

Hi @EricThomson!
I see the problem here, which is also related to this issue. It is not just a nomenclature error.

• estimates.C (denoised calcium traces) should be stored in Fluorescence as a ROIResponseSeries (_roi_response_raw in the SegmentationExtractor),
• estimates.S (spike counts from deconvolution) should also be stored in Fluorescence as a ROIResponseSeries named Deconvolved (_roi_response_deconvolved in the SegmentationExtractor)
• estimates.F_dff --> in DfOverF as a ROIResponseSeries named DfOverF (_roi_response_dff in the SegmentationExtractor),
• estimates.f (temporal background components) should be stored in Fluorescence as a ROIResponseSeries named either Neuropil or Background (_roi_response_neuropil in the SegmentationExtractor). Here, we should also add the spatial background components estimates.b and create the appropriate plane segmentation.

It is very important to keep this information for data reuse purposes. For instance, to check the quality of the segmentation output, one might want to compute indices that need the background signal to be estimated, e.g., the “isolation distance” of a cell’s signal from the surrounding signals (computed similarly to the spike sorting equivalent).

This is something we need to fix on our end. I will work on this in the next few days.
CCing @pauladkisson to keep him updated

[EricThomson]

@alessandratrapani thanks for this reply. This all seems exactly right!

There are a couple of things I can add that may not be important. 😄

There is one other quantity people sometimes want, which is the residual for each component (estimates.YrA: yes our variable names are awful 😬 ). This is because sometimes they want the "raw" (rather than denoised) traces (you can get raw traces with C+YrA).

Caiman denoises by taking into account the calcium indicator dynamics (the time constant of offset and sometimes onset) and smoothing based on those. Typically if you visually inspect the original (motion corrected) movie, C matches the fluctuations in the movie more closely than C+YrA. Upshot: not sure if it is worth providing residuals in an nwb file, but just throwing it out there as sometimes users want them. 🤷 (Additional wrinkle: dff can be calculated with or without residuals, i.e., either on denoised or raw calcium traces).

On the background model: things get more gnarly for the background model when we run CNMFE in Caiman, because then there is no b/f (they are None). Instead, we have the ring model, so the parameters are W and b0 (as I'm sure you know, in CNMFE there is a much more complicated background model, where each pixel has its own background fluctuations a certain distance in a ring from that pixel that are subtracted out -- this is captured in W that represents the ring model; b0 is a dc offset for each pixel). Currently I'm working on getting CNMF -> NWB to work well. Next step I plan to tackle CNMFE.

[alessandratrapani]

There is one other quantity people sometimes want, which is the residual for each component (estimates.YrA: yes our variable names are awful 😬 ). This is because sometimes they want the "raw" (rather than denoised) traces (you can get raw traces with C+YrA).

I do agree! I was a bit annoyed by the fact of storing denoised traces in _roi_response_raw and wanted to propose including this distinction myself.
If everyone agrees to reform the CaimanSegmentationInterface a little bit more, I could include all of these different components. For consistency, I would store the raw traces C+YrA, not the residuals, the denoised traces, the deconvolved traces, and the background components.

[EricThomson]

This sounds like a really good solution (if people really want residuals they can subtract denoised traces from raw traces). 😄

[alessandratrapani]

On the background model: things get more gnarly for the background model when we run CNMFE in Caiman, because then there is no b/f (they are None). Instead, we have the ring model, so the parameters are W and b0 (as I'm sure you know, in CNMFE there is a much more complicated background model, where each pixel has its own background fluctuations a certain distance in a ring from that pixel that are subtracted out -- this is captured in W that represents the ring model; b0 is a dc offset for each pixel). Currently I'm working on getting CNMF -> NWB to work well. Next step I plan to tackle CNMFE.

For this, we can open another issue and discuss it there. Please tag me in anything I can help with for the CNMFE --> NWB future implementation. During my PhD studies, I worked with 1p imaging data and used CNMFE for the segmentation step. I am not an expert but I can offer some help.

[EricThomson]

For this, we can open another issue and discuss it there. Please tag me in anything I can help with for the CNMFE --> NWB future implementation

That sounds great thanks for your help!

[pauladkisson]

Closed by #291