An empty BGZF block elsewhere in the file stream causes premature EOF.

[jkbonfield]

Adding ^_\213^H^D^@^@^@^@^@377^F^@bc^B^@^[^@^C^@^@^@^@^@^@^@^@^@ elsewhere in a BAM file (other than the end) causes samtools view to end prematurely. This will either produce a decoding error, or if it happens to coincide with a genuine end of sequence record then it'll silently just truncate the output. This is visible if we add it as the first block after the SAM header.

Picard continues past this and ignores it as an end of file identifier unless it actually occurs at the end. The proposed amendment to the specification states "should by convention finish with a specific empty BGZF block". Note it does not say anything about the behaviour of empty blocks anywhere else, so we should not be interpreting them as a special indicator for EOF.

[jmarshall]

Background: the desired effect of embedded EOF trailer blocks was discussed in December 2013 arising from this issue, resulting in a spec clarification (see samtools/hts-specs@c02ad4c) noting that they should not be interpreted as EOF.

See [Samtools-help] tabix bug on cat'ed vcf.gz for another use case.

[isthisthat]

Thanks for looking into this John. This bug came up when parallelizing variant calling by chr so it might become more ubiquitous in the future.

[Lenbok]

RTG uses the same trick internally when doing mapping - individual sub BAM-files are written without header or termination block as appropriate in order to permit a fast concatenation at the end.