Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fastq/fasta threading for output file. #908

Merged
merged 2 commits into from
Aug 14, 2018

Conversation

jkbonfield
Copy link
Contributor

Also fixes #906

Previously multi-threading was only applied to the input file
descriptor.
@jkbonfield
Copy link
Contributor Author

Note: following samtools meeting, we proposed changing it so -c enabled compression (rather than being filename based). I've given up on this for now as it's just too invasive and needs a major refactor of the code.

Specifically stdout ends up being gzipped even if we write nothing to it, so the common idioms of samtools fastq end up dumping binary to the terminal (most people don't specify things like -0 /dev/null to discard them).

If we're interested in gzipping 1.fq and 2.fq but not gzipping the things we discard, then it's a misfeature anyway to gzip before redirecting to /dev/null. Frankly the entire program wants a new CLI such that no output is written unless explicitly asked for rather than vice versa. It also needs to cope with specifying the same filename twice (so we can do -1 - -2 - to explicitly request 1 & 2 to stdout or a joint file, but that in turn really needs a dozen structure members renaming in order to keep it sane.

Given the complexity, that isn't this PR, but something new for another time.

@daviesrob daviesrob merged commit a8642fb into samtools:develop Aug 14, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Default compression level for samtools fastq ?
2 participants