New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alignment compression #426

Merged
merged 14 commits into from Sep 28, 2018

Conversation

Projects
None yet
2 participants
@dbolotin
Member

dbolotin commented Sep 27, 2018

This allows to decrease output file size (vdjca and clna) 4-fold.

Implements block-compression of alignments with LZ4 algorithm.

Uses parallel compression to keep up with massive output from aligner in highly multithreaded environments. 1 compression thread per 8 processing threads in align action.

Fast compression implemented, but not connected to any input options. Also requires additional testing.

@dbolotin dbolotin requested a review from PoslavskySV Sep 27, 2018

@PoslavskySV

This comment has been minimized.

Show comment
Hide comment
@PoslavskySV

PoslavskySV Sep 28, 2018

Member

MiLib must be updated as well.

Member

PoslavskySV commented Sep 28, 2018

MiLib must be updated as well.

dbolotin added some commits Jul 30, 2018

WIP
WIP
Rework of VDJCAlignments writer. Serialisation logic separated. On th…
…e way to compressed alignments in CLNA file.

@dbolotin dbolotin merged commit afab7a6 into develop Sep 28, 2018

0 of 2 checks passed

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
continuous-integration/travis-ci/push The Travis CI build is in progress
Details

@dbolotin dbolotin deleted the compression branch Sep 28, 2018

@dbolotin dbolotin referenced this pull request Sep 28, 2018

Closed

Compressed vdjca / clns ? #400

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment