3D segmentations and priors

[jonas2612]

Hi,

thank you for the great software.

While using Proseg some questions arose. First of all, we are segmenting merscope data, hence have 7 layers of transcripts. In our use case, we want to create a 3D segmentation, which is already implemented in Proseg. How many voxels would you recommend for a 3D segmentation? Also how do the layers in the cell-polygons-layers.geojson correlate to the axis?

Furthermore, how do priors, especially 2D priors, work with Proseg? Would 3D priors, e.g. from a 3D Cellpose segmentation, result in better Proseg segmentations?

Also, is the --detect-layers flag with merscope data relevant?

Best,
Jonas

[dcjones]

Hi Jonas,

The default behavior with proseg is to initialize with a single layer of voxels and gradually increase voxel resolution until there are four layers. That seems to work pretty well, but I've mostly focused on trying to have accurate transcript assignments. It's general hard to evaluate how accurate the 3d morphology is. Generally I think the 3d information these platforms provide is pretty approximate. There are only 7 focal lengths, and I'm skeptical that assignment of transcripts to those layers is precise. That said, you could try using more voxel layers and see how it affects things.

The main parameters to consider are --voxel-layers which gives the initial number of voxel layers, --initial-voxel-size which gives the initial x/y voxel size in micrometers, and --schedule which is a comma separated list of iterations between voxel resolution is doubled. So you could just try --voxel-layers 2 to run with twice as many z-layers and see if that changes things much.

The way 3d morphology is stored in cell-polygons-layers.geojson is that each cell has a multipolygon on each voxel z-layer. Every feature in that file has two properties cell giving the cell number and layer giving the z-layer.

You can ignore the --detect-layers, which will only affect how noise is modeled.

Using a prior 3d segmentation could definitely improve proseg 3d segmentation, but isn't something I've tested carefully.

[lguerard]

Sorry to hijack this issue but I'm trying to run proseg on 3D dataset from seqFISH and am having issues doing the 3D segmentation.

I managed to run Cellpose in 3D on my data with 94 slices, I have all the transcripts allocated to the cells, but when running proseg, the cell-polygons-layers.geojson only contains 4 slices. I tried changing --voxel-layers to 94, but then it returns cells on layers above 400. Is there something I'm missing ?
I'm happy to make a separate issue if needed.

[jonas2612]

Thank you for your responds. Especially the usage of --detect-layers is now clearer.

If I understand --voxel-layers correctly now, --voxel-layers 3 would produce 3x4=12 layers in the output. Hence if @lguerard would like to produce about 94 layers, he could set --voxel-layers 24, resulting in 24x4=92 layers. Do I understand it correctly?

Furthermore, If we'd decide to use a 3D prior, how would you recommend to provide it to Proseg?

[dcjones]

Right, you end up with more voxel layers because (by default, you can change this with --schedule) proseg uses a strategy of starting inference at a lower resolution and increasing the resolution, so you end up with 4x the number of voxel layers you start with.

Right now, prior segmentation is handled just by considering prior transcript assignments, so you don't have to do anything special to use a prior 3d segmentation. (This may change in the next major release when we support voxel level priors, but that's a ways off.)

[jonas2612]

Great, thank you for explaining the concept behind it. It helped a lot understanding your approach better