Repository navigation
The first tagged release. RAMTools began in 2017 at CERN openlab as a format
stored in ROOT's TTree; this release is its rewrite on RNTuple. Files written by
it stay readable by every later release.
Added
- The RAM format on ROOT's RNTuple and
samtoramntuple, which converts SAM to
it (#1, #2). bamtoramntuple, which converts BAM through htslib (#20).ramdump, which writes a file or a region of it back out as SAM and takes the
samtools viewoptions-h,-H,-c,-f,-Fand-o(#75).ramntupleview, which counts the records in a region and reports the query
time (#10, #69).-split, one output file per reference, written as a stream and indexed like
any other file (#9, #79).-compression Non both converters, taking a ROOT compression code (#73).-threads Non both converters;samtoramntupleparses, encodes and
compresses 64 MB blocks on every thread and writes them in input order
(#86, #87, #88).- Region queries find the region by binary search over the positions of a
coordinate-sorted file; the converters record whether the input was sorted,
and an unsorted file is read in full (#79, #83). - Benchmarks: a SAM generator and Google Benchmark suites for conversion and
region queries (#4, #6, #7, #8), and scripts that run them on real data
(#54). - Continuous integration: build and tests, clang-tidy review, formatting checks
and Codecov gating (#3, #5, #11, #17, #18, #19, #21, #32, #35,
#47, #57, #77). - Tests for the index, the quality encodings and the record getters (#39,
#41, #43). - The Apache License 2.0 (#60);
CONTRIBUTING.md, issue and pull request
templates, and a README that walks through installation, conversion and
queries (#12, #89).
Changed
- ROOT 6.38 is the minimum version (#76).
- SEQ is stored as uppercase text instead of 4-bit packed bases, which
compresses better: 3.9% smaller files on chromosome 11. Files with packed
sequences still read (#85). - Opening a RAM file no longer starts ROOT's interpreter. On 2 million records
of chromosome 11, counting a 10 Mb region went from 0.59 s and 150 MB to
0.26 s and 19 MB (#92), and dumping a 1 kb region from 0.61 s and 204 MB to
0.30 s and 78 MB (#93).-hand-Hstill start it, to read the header. - Region queries decide overlap by each read's CIGAR span and accept
rname,
rname:pos,rname:start-endand*(#15).
Removed
- The TTree backend (#71).
RAMNTupleConverter, replaced byramdump(#75).- The sparse index of mapped reads (#16), replaced by the binary search
(#83).
Fixed
- Sequences decoded wrongly by
DecodeSequence(#45). - The SAM parser accepted non-numeric and out-of-range integer fields, and a
query for an unknown reference added it to the reference list (#59). - Float tags lost precision (#64).
- A CIGAR of
*came back as an empty field (#66). - Illumina quality binning was indexed by the ASCII character instead of the
Phred score (#68). - Fields lost or shifted on the way in or out: lines over 10 kB, consecutive
tabs, unknown and lowercase bases, unknown CIGAR operators (#70). - The SAM header was written as one key per line and could not be read back
(#72). - A mapped read with a malformed CIGAR such as
10Qwas stored without one; it
is now reported with its line number and skipped (#74). - A region query past the last index anchor of a reference started from the
first row and read the whole file (#82); the index was later replaced by the
binary search (#83). - README examples that did not run (#78).
bamtoramntupledid not link when htslib is not in a default library
directory, as on macOS (#95).- clang-tidy findings and suppressions (#14, #38).
Full list of changes: CHANGELOG.md.