Run FindAllMarkers function in parallel

[LeaLe88]

Good afternoon,

I am facing long run times with below function since I added the adjustment for batch (via latent.vars):
FindAllMarkers(kid.filtered_new,test.use="poisson",latent.vars = "orig.ident")

I would like to run it in parallel in RStudio. I read some old posts about doing it with the doFuture package.
However, more recent posts mentioned that this is not working anymore? #7000 ?

Can anyone advise me if this is the case and if so could you recommend me any other option to run this function in parallel?

Thank you!

[samuel-marsh]

Hi,

Not member of dev team but hopefully can be helpful. Within the Future framework you could always use FindMarkers and future_lapply instead of FindAllMarkers.

library(future)
library(future.apply)

plan(multisession(workers = XX))
idents <- levels(Idents(obj))

markers <- future_lapply(idents, function(x) {
  FindMarkers(object = obj, ident.1 = x)
}, future.seed = TRUE)

Best,
Sam

[LeaLe88]

Dear Sam

Thank you so much for your reply!

And this would work with latent.vars and test.use argument?

library(future)
library(future.apply)

plan(multisession(workers = XX))
idents <- levels(Idents(obj))

markers <- future_lapply(idents, function(x) {
  FindMarkers(object = obj, ident.1 = x, test.use="poisson",latent.vars = "orig.ident")
}, future.seed = TRUE)

Thank you!

[samuel-marsh]

Hi @LeaLe88,

No problem, happy to help! Ya it will work with whatever other arguments you want in the function. The ident.1 = x is key part as that specifies which of the clusters in idents to use.

Best,
Sam

[LeaLe88]

Hi @samuel-marsh ,

Thank you very much!

[HenrikBengtsson]

PS. @samuel-marsh , the official specification is plan(multisession, workers = XX), not plan(multisession(workers = XX)). I've noticed others using the latter, and don't know when that style started to spread, but it's not official supported (and I surprised it actually works)

[samuel-marsh]

Hi @HenrikBengtsson yes thanks for correcting error! Indeed an older code doc where I copied from for this issue.

[LeaLe88]

Thank you, I have changed it. Continued discussion in future github on this HenrikBengtsson/future#710.