Developer-targeted detailed documentation of Oxli binary formats

[kdm9]

Hi all,

This PR adds some personal notes I've taken from reading the source code on the binary file formats in khmer/oxli.

I'm not sure if it will be useful to the project, but here it is in any case. I'll polish it up a bit further, and maybe add some more formats, if you think it's useful.

Cheers,
K

[mr-c]

@kdmurray91 I am blown away, this is really wonderful

[kdm9]

@mr-c Is there a way of wrapping lines in RST simple tables that doesn't create a new row? Or do I have to suck it up and make a proper table?

[mr-c]

I think you can line wrap if you keep the correct indentation?

On Sun, Jun 14, 2015 at 10:33 PM Kevin Murray notifications@github.com
wrote:

In doc/dev/binary-file-formats.rst
#1101 (comment):

+Field Len Off Value
+================== ===== ===== ==============================================
+Magic string 4 0 OXLI (SAVED_SIGNATURE)
+Version 1 4 0x04 (SAVED_FORMAT_VERSION)
+File Type 1 5 0x02 (SAVED_HASHBITS)
+K-size 4 6 k-mer length, as unsigned int.
+Number of Tables 1 10 Number of Nodegraph tables.
+================== ===== ===== ==============================================
+
+Then follows the Nodegraph's tables. For each table:
+
+================== ======= ===== ==============================================
+Field Len Off Value
+================== ======= ===== ==============================================
+Table size 8 0 Length of table, in bits. (uint64_t)
+Bins N/8+1 8 This table's bytes, length given by previous field, divided by 8, plus 1. (uint8_t)

@mr-c https://github.com/mr-c Is there a way of wrapping lines in RST
simple tables that doesn't create a new row? Or do I have to suck it up and
make a proper table?

—
Reply to this email directly or view it on GitHub
https://github.com/dib-lab/khmer/pull/1101/files#r32392578.

Michael R. Crusoe: Programmer & Bioinformatician crusoe@ucdavis.edu
mcrusoe@msu.edu
The lab for Data Intensive Biology; University of California, Davis
https://impactstory.org/MichaelRCrusoe http://twitter.com/biocrusoe

[kdm9]

So you can. emphasis on correct 😄

[ctb]

+1

On Sun, Jun 14, 2015 at 09:41:43PM -0700, Michael R. Crusoe wrote:

@kdmurray91 I am blown away, this is really wonderful

[kdm9]

Don't think I'll add much to this in the current round, as that's all I have written. So I'd say it ready for review, and I'll add to it later.

• Is it mergeable?
• Did it pass the tests?
• If it introduces new functionality in scripts/ is it tested?
  Check for code coverage with make clean diff-cover
• Is it well formatted? Look at make pep8, make diff_pylint_report,
  make cppcheck, and make doc output. Use make format and manual
  fixing as needed.
• Did it change the command-line interface? Only additions are allowed
  without a major version increment. Changing file formats also requires a
  major version number increment.
• Is it documented in the ChangeLog?
  http://en.wikipedia.org/wiki/Changelog#Format
• Was a spellchecker run on the source code and documentation after
  changes were made?
• Is the Copyright year up to date?

[kdm9]

Oh, do RST files in doc/dev/ need a copyright header? Like getting-started.rst?

[mr-c]

@kdmurray91 yes, but we haven't decided on what to write there since the move. I'm okay without it for now.

[mr-c]

We always spell it khmer to differentiate from the people group

[mr-c]

LGTM, thank you @kdmurray91