Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About threads/cores #88

Closed
matrs opened this issue Jun 13, 2019 · 3 comments
Closed

About threads/cores #88

matrs opened this issue Jun 13, 2019 · 3 comments

Comments

@matrs
Copy link

matrs commented Jun 13, 2019

In the configuration file, config.yaml, says

## The maximal number of cores to use for FastQC, STAR, Salmon and DRIMSeq.                  
## Note that the actual number of cores available to Snakemake is determined by              
## the --cores argument when it is invoked.
ncores: 1

and then in some rules you have:

threads:
	config["ncores"]

I got confused a couple of moths ago when I didn't know snakemake as I do know. As you say, with --cores you specify the maximal number of cores used by snakemake, so when your are running the rule star_SE with snakemake --cores 12 and you specify in the config file ncores: 2, you'll be running 6 instances of STAR in parallel, but each instance will be using 2 cores, you won't be running STAR in one file with 12 cores (one instance). I didn't get that from the config file. I'm saying this because it's important to know how many input files you have (approximately) to set ncores properly and, I think, many times is better to set threads independently in each rule because of this and also, because different software has different performance when running with multiple cores in parallel. My intention isn't that you change the behavior of this pipeline, It is rather to try to avoid confusing (unintentionally) some users (like me). Maybe you can add some information in the wiki or in the config file about this.

@csoneson
Copy link
Owner

Thanks for pointing this out. We are looking into how we can best clarify the behaviour in the documentation!

@csoneson
Copy link
Owner

Hi again - we have added a clarification to the wiki (https://github.com/csoneson/ARMOR/wiki/Running-the-analysis) - thanks for bringing this up.

@matrs
Copy link
Author

matrs commented Jun 13, 2019

Thanks to you guys for providing so many free and high quality resources :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants