Counting and Statistics

Counting and statistics

Two commands answer the first question asked of any FASTQ file: what is in it? count gives the number of records and nothing else, as fast as the file can be read. stats reads every base and quality score and summarizes the whole file.

count

z-fastq count lane1.fastq.gz lane2.fastq.gz

It prints one number per input, in the order given; an input that fails prints no number, so check the exit status before pairing numbers with inputs. count checks that every record is well formed: four lines, the @ and +, and a quality line as long as the sequence. It does not check the letters or the quality bytes; check does.

_{Figure 1. Time and peak memory of other record counters relative to z-fastq, on 1 GiB inputs and one CPU; lower left is better. The count benchmark has the inputs, the tools, and every number.}

stats

z-fastq stats reads.fastq.gz

Each result starts with an input line naming the file, followed by these fields.

Reads and bases. reads counts records and bases counts sequence bytes. min_length, max_length, and mean_length describe read lengths. An empty record still counts as a read, so the two can tell different stories.

Composition. a, c, g, t, and n count each base, upper or lower case, and other_bases counts any other byte. stats checks the record structure and the quality bytes but not the sequence alphabet, so a - or a digit counts here; check rejects them. GC is measured only among the four definite bases, because z-fastq does not guess what an ambiguous base would have been:

$$\text{gc\_fraction} = \frac{g + c}{a + c + g + t}$$

Quality. Scores are read as Phred+33. quality_sum adds them up, mean_quality divides by bases, and q20_bases and q30_bases count bases scoring at least 20 and 30, with q20_fraction and q30_fraction as their shares of bases. A quality byte outside ASCII 33 to 126 is an error.

A value that cannot be computed, such as a mean over no reads, prints as -, or null in JSON.

_{Figure 2. The same comparison for stats. Details in the stats benchmark.}

Several inputs at once

Both commands take any number of inputs and keep each result separate. When one input fails, the rest are still read, and the exit status reports the worst failure (Automation).

stats summarizes a file as a whole. Quality by cycle, adapter content, duplication, and the rest of a QC report come from process (Quality control).

Related pages

• Validation: the full record checks, and what their error codes mean.
• Automation: the stats --json document.
• Quality control: per-cycle and per-read quality, adapters, and duplication.