Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Q: Is there a way to control threads used bei pigz? #290
I would probably just use
You can also restrict a process to run on a certain set of CPU cores with
This would run cutadapt on the first four cores (cores 0 to 3). Note that if a second user does the same and enforces use of cores 0-3, then they would share these cores, so this is not quite the same as requiring that a process is limited to 400% CPU.
If you want to be thorough, you could also configure your server so that one user cannot take CPU resources from another. I have no idea how to do that, but I know it’s possible because our cluster is configured this way. It probably involves using
I just started to use cutadapt with the cores option and was a bit negatively surprised that the process took far more than the 8 cores which I had set. I think it would be better to pass the number of maximum cores to pigz. And in case of paired end data just divide the number of set cores by two and pass that over to pigz. most often it isn't a problem if you use 1 additional thread, but using 20 instead of 8 might be a problem and in my opinion not very social to other users of the same server. I use cutadapt in combination with snakemake, so using taskset isn't such a nice way to implement in the pipeline. I hope you will take it in consideration.