Validation

Validation

check answers one question: can this file be trusted as FASTQ? It reads every record, stops at the first problem, and says exactly where the problem is.

z-fastq check reads.fastq.gz

A valid file prints nothing and exits with status 0, so check works as a plain pass or fail step in a script.

What a record must look like

_{Figure 1. The four lines of a FASTQ record. The quality line holds one byte per base, so it is exactly as long as the sequence.}

• Header. Starts with @, and the identifier follows at once, with no space or tab in between.
• Sequence. One line of letters; wrapped sequences are not joined into one.
• Plus line. Starts with +; anything may follow it.
• Quality. One byte per base, Phred+33, from ASCII 33 (!) to 126 (~).

An empty sequence with an empty quality line is a valid record. Lines may end in LF or CRLF, even mixed in one file, and the last line may lack its final line break. A carriage return counts as part of a line ending only when a line feed follows it, so a last quality line ending in a lone CR keeps the CR and is one byte too long.

Which letters pass

The default, --alphabet iupac, accepts the IUPAC nucleotide letters in either case. --alphabet acgtn accepts only A, C, G, T, and N. The choice only decides what passes: z-fastq never changes letters or their case.

When a record fails

check reports the first failure with a code, the record number and the byte offset in the decompressed stream (both counted from 0), and the line within the record (counted from 1). A line over the --max-line-bytes limit is reported with the file name only.

Code	Problem
S001	the plus line does not start with +
S002	a sequence letter is outside the alphabet
S003	the header does not start with @ and an identifier
S004	the file ends inside a record
S005	the sequence and quality lengths differ
S006	a quality byte is outside Phred+33

Pair checks add two more codes, P001 and P002, described in Paired reads.

_{Figure 2. Time and peak memory of other validators relative to z-fastq, on 1 GiB inputs and one CPU. The check benchmark also shows which malformed files each tool lets through.}

Checking is not repairing

check does not skip bad records, fix qualities, or write a cleaned file. A broken record usually means a broken transfer or a broken tool upstream, and I would rather you find out than have it smoothed over.

Related pages

• Paired reads: checking that two files, or one interleaved file, hold matching pairs.
• Automation: exit statuses, and check --json for scripts.
• Limits and supported formats: line length and the formats z-fastq reads.