Indexing

Indexing

Index once, then GET and stats can jump to the sequence bytes they need. I recommend the default .zfi when z-fasta is the only consumer. Validate does not need an index; validate --index checks that an existing one still describes its FASTA.

Build the default .zfi

z-fasta index genome.fa

The command writes genome.fa.zfi and reports the retained sequence count to stderr.

wrote genome.fa.zfi (N sequences)

Use .zfi unless another program explicitly asks for .fai. It embeds identifiers, stores source size and mtime, and adds per-line side tables when fixed byte geometry is unsafe.

Emit a compatible .fai

z-fasta index --emit-fai genome.fa > genome.fa.fai

.fai is written to stdout. Name the redirected file <fasta>.fai so z-fasta can find it when .zfi is absent.

Important

.fai succeeds only when every retained record has fixed line geometry. Use default .zfi indexing for mixed widths, interior blank lines, trailing sequence whitespace, or mixed line endings.

Before writing stdout, z-fasta scans into an exclusive temporary spool. A scan failure does not replay a partial index. Shell redirection may still create an empty destination file.

What counts as a sequence name

Given this header:

>chr1 chromosome one

The indexed identifier is chr1. Parsing stops at the first space, tab, CR, LF, or end of file. Header descriptions are not lookup keys.

Identifiers are byte strings and do not need to be valid UTF-8. The maximum identifier length is 65535 bytes.

Empty records

Records with no sequence symbols are skipped:

>empty
>kept
ACGT

The index contains only kept. Validate still reports empty as a fasta.empty_record warning.

Duplicate identifiers

Indexing keeps every non-empty record in file order, duplicate identifiers included, so stats counts them all. GET resolves a duplicate identifier to its first record; the Zig package reaches later ones by ordinal. validate reports each duplicate as fasta.duplicate_identifier. If every record needs retrieval by name, rename the duplicates in the FASTA.

Uniform and non-uniform records

A record is uniform when byte offsets can use one line formula. These ordinary cases stay uniform:

• equal-width sequence lines;
• a shorter final line;
• a final line without a newline;
• a final CRLF line after an LF body when base width remains valid.

These cases require a .zfi side table:

• an interior line with a different base width;
• a final line wider than the established width;
• trailing spaces or tabs on sequence lines;
• an interior blank line;
• an interior LF and CRLF width change.

Read Index formats for the operational difference.

Publication safety

The default path builds .zfi in an independent temporary file beside the destination. It checks that the source path still has the original size and mtime before replacing <fasta>.zfi with the completed file. A handled scan or output failure preserves an existing index and removes temporary output.

The .fai path also checks source size and mtime before replaying its spool to stdout. If the source changes during either workflow, indexing fails.

Large .zfi inputs may also use temporary files for names and side tables. Spool files use the first writable directory from TMPDIR, TEMP, or TMP, then /tmp on Linux and macOS, then the current directory. Temporary disk use grows with the staged index data.

Duplicate sidecars

You may keep both genome.fa.zfi and genome.fa.fai. GET and stats always choose .zfi when it exists. .fai is considered only when .zfi is absent.

Recommended workflows

z-fasta only

z-fasta validate genome.fa
z-fasta index genome.fa

Shared with FAI-based tools

z-fasta validate --strict genome.fa
z-fasta index --emit-fai genome.fa > genome.fa.fai

If .fai emission rejects the layout, normalize a separate copy:

z-fasta validate --fix -o genome.clean.fa genome.fa
z-fasta validate --strict genome.clean.fa
z-fasta index --emit-fai genome.clean.fa > genome.clean.fa.fai

Related pages

• Index formats
• Validation and repair
• Troubleshooting
• Limits and supported formats