Repository navigation
Cell segmentation parallelization #36
Replies: 4 comments 13 replies
|
An easy, and maybe a bit naive, way of doing this with dask through the python API would be from dask.distributed import Client
from dask import delayed
import dask
# Initialize a Dask client to manage distributed workers
client = Client(n_workers=16) # Adjust based on the number of patches you have.
# Define a delayed function to process each patch
@delayed
def process_patch_delayed(patch_index):
segmentation.write_patch_cells(cellpose_temp_dir, patch_index)
# Wrap patch processing in delayed objects
tasks = [process_patch_delayed(patch_index) for patch_index in range(len(sdata["sopa_patches"]))]
# Compute in parallel using Dask
dask.compute(*tasks)
client.close()Works well in one example case, I haven't tested this extensively. Also, I could not get this to work with Baysor yet - it always runs in sequential manner. You can use the |
|
I'm working on Example usage: sopa.setting.parallelization_backend = "dask" # sets globally the backend
sopa.segmentation.cellpose(sdata, "DAPI", diameter=35) # runs cellpose on all patches in parallelWhat do you think? |
|
Thanks all! If in the case of running parallelization if we prefer using the [CLI], will the sopa==2.00 add parallelization using CLI? |
|
Hi, pip install sopa [cellpose] even if I make smaller patches , the segmentation takes more than 24 h. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
For efficiency, cell segmentation must be parallelized. This is done automatically using the Snakemake pipeline, but it's possible to use your own parallelization mechanism if you prefer using the CLI or the API.
Note that Snakemake may require setting up a profile. There is already an existing one for Slurm clusters, but some other profiles can be added in the future (don't hesitate to ask for it!).
You can ask questions here, or show your parallelization technique to help other users.
All reactions