err: NaN during re-estimation step

[naomixia]

Hello!

I'm having issues with diam <4 cases where during the cell mask re-estimation step the err becomes NaN. The same data set run with diam >= 5 does not have this problem, but this parameter setting causes under-detection of cells. Any help would be appreciated!

skipping registration, but assembling binary file
Set 1, tiff 1 done in time 38.11
517 total ROIs, err 1.7088, thresh 0.2315
886 total ROIs, err 1.1489, thresh 0.2315
1160 total ROIs, err 0.8322, thresh 0.2315
1361 total ROIs, err 0.6894, thresh 0.2315
1495 total ROIs, err 0.6454, thresh 0.2315
1573 total ROIs, err 0.6297, thresh 0.2315
1617 total ROIs, err 0.6231, thresh 0.2315
1617 total ROIs, err NaN, thresh 0.2315
1617 total ROIs, err NaN, thresh 0.2315
1617 total ROIs, err NaN, thresh 0.2315
Frame 6000 done in time 2.90
Frame 12000 done in time 5.83
Frame 15000 done in time 7.33
Error using bsxfun
Non-singleton dimensions of the two input arrays must match each other.

Error in svdecon (line 36)
V = bsxfun(@(x,c)x./c, V, s');

Error in extractSignalsSurroundNeuropil2 (line 97)
[u, s, v] = svdecon(ratCS);

Error in run_pipeline (line 78)
[ops, stat, Fcell, FcellNeu] = extractSignalsSurroundNeuropil2(ops, stat);

[carsen-stringer]

Have you downloaded the latest version of the code? This chunk of code is just for zdrift estimation, there's an option ops.Zdrift that is set to 0 by default. Are you setting it to 1?

[carsen-stringer]

Sorry it's ops.getZdrift that should be set to zero by default

[naomixia]

Hello! Yes I tried disabling the z-drift code and it allows me to finish running the pipeline but the NaN persists. The deconvolution output is 0 when I allow it to run until the end. Something about the data is causing the neuropil subtraction output to become NaN, and playing with neuropil radius doesn’t seem to help either. Just to clarify, my data is cellular gcamp taken at 10X so the cell body ROIs are small. I appreciate any pointers!

…

Sent from my iPhone

On Jun 19, 2018, at 7:45 AM, carsen-stringer ***@***.***> wrote: Sorry it's ops.getZdrift that should be set to zero by default — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[carsen-stringer]

Oh sorry I misunderstood your problem when it runs to the end what are the stat(k).radius? We estimate the cell radius by fitting an ellipse to the cell pixels and I'm thinking that it might be too small and we should add a default minimum value (the neuropil masks scale with the radius of the cell)

[naomixia]

Thanks for getting back to me so quickly! If I run it all the way I get the following output and the stat(k).radius are all equal to 0: 1617 total ROIs, err NaN, thresh 0.2315 1617 total ROIs, err NaN, thresh 0.2315 1617 total ROIs, err NaN, thresh 0.2315 Frame 6000 done in time 2.85 Frame 12000 done in time 5.64 Frame 15000 done in time 7.07 Spike deconvolution, plane 1... Starting parallel pool (parpool) using the 'local' profile ... connected to 2 workers. 1 0 1 0 1 0 1 0 <img width="1032" alt="screen shot 2018-06-19 at 4 31 23 pm" src="https://user-images.githubusercontent.com/40378317/41622800-bc3eadf2-73de-11e8-93b0-e4630411e7a3.png"> However, if I disable SourceryAddOn, I am able to extract the pre-neuropil subtracted stat(k).radius, I get radius between 0 - 3.36 (the signal output from this step looks really awesome, but I want to make sure that I'm subtracting neuropil signal)

…

On Tue, Jun 19, 2018 at 10:55 AM, carsen-stringer ***@***.***> wrote: Oh sorry I misunderstood your problem when it runs to the end what are the stat(k).radius? We estimate the cell radius by fitting an ellipse to the cell pixels and I'm thinking that it might be too small and we should add a default minimum value (the neuropil masks scale with the radius of the cell) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#123 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AmgfzeK7APWOKpAR8j-uAneiX39yHg9uks5t-RDygaJpZM4Us0nv> .

[carsen-stringer]

Alright I set the minimum radius to 3 pixels (it's computed in createCellMasks.m) try pulling the new code running it now and see if the neuropil looks reasonable (and if things don't go NaN).

[naomixia]

Hello! I attempted it but It's still going to NaN. I noticed that my stat.ipix was also empty, and wondering if this caused createCellMasks.m to not enter the loop for the minimum radius code in line 22?

[carsen-stringer]

Even without SourceryAddOn the cells have no pixels?

[naomixia]

I think it always works without SourceryAddOn. I just re-ran it with min=3 but I'm not certain the output is different from before.

[carsen-stringer]

SourceryAddOn reduces the sizes of the ROIs, which may reduce your ROIs to zero pixels if they are very small. You can run it with this turned on, but you will probably lose some of your ROIs (anything with ipix=0 will not have any signals extracted). But does running without SourceryAddOn and with min=3 return non-NaN neuropil?

[naomixia]

Yes, without SourceryAddOn everything's fine.

…

On Tue, Jun 19, 2018 at 5:55 PM, carsen-stringer ***@***.***> wrote: SourceryAddOn reduces the sizes of the ROIs, which may reduce your ROIs to zero pixels if they are very small. You can run it with this turned on, but you will probably lose some of your ROIs (anything with ipix=0 will not have any signals extracted). But does running without SourceryAddOn and with min=3 return non-NaN neuropil? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#123 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AmgfzVHf3dtBXYJR2Zb4HP4lbzpMO_6Dks5t-XNVgaJpZM4Us0nv> .

[carsen-stringer]

I've added in an option (ops.refine, default is 1). If ops.refine=1, then sourceryAddOn.m runs, if ops.refine=0, then it won't refine. Please reopen this issue if you're still having problems.