Parallel FindMarkers with Future

[weshorton]

Hello,

I am a beginner in terms of parallel computing in R and am trying to run FindMarkers() using the framework described in the vignette. I'm actually trying to use FindAllMarkers(), but my issue appears with both of them. After setting the plan and running my code, I check out my cores using htop and find that only one core is being used.

> availableCores()
system 
    24 

> plan("multisession")
> plan()
multisession:
- args: function (..., workers = availableCores(), lazy = FALSE, rscript_libs = .libPaths(), envir = parent.frame())
- tweaked: FALSE
- call: plan("multisession")

clusters_v <- levels(seurat_object)
test <- FindMarkers(seurat_object, 
                                   ident.1 = clusters_v[1], 
                                   ident.2 = clusters_v[-1],
                                   only.pos = F,
                                   test.use = "MAST",
                                   latent.vars = "batchID"

### OR
# test <- FindAllMarkers(seurat_object, only.pos = F, test.use = "MAST", latent.vars = "batchID")

Looking at htop, I see this:

The function call does seem to be running faster than previously, but I feel like I should be seeing usage on other cores instead of just the one. Any comments, links to resources, etc. are appreciated!

[AustinHartman]

The MAST differential expression method is unfortunately not configured to run in parallel. If you try a different method (like test.use = "wilcox") you should see more cores used

[ppm1337]

I would like to add to this discussion that the current vignette proves that the parallelization is not working any more. The default test.use = "wilcox" with 4 workers in the example has no speedup according to the plot in the vignette:

In an archived vignette, the same example showed a considerable speedup with 4 workers:

[ishwarvh]

Would like to add that parallelization seems to have broken when using the future package within RStudio.
Neither specifying "multisession" nor "multicore" has any effect on the number of cores used. All the previously parallelized functions run on only a single core!