Little sequence file utilities meant to work within Unix pipelines
Python Other


Ngs crumbs is a collection os small ngs file processing utilities.

Seq Crumbs is a collection of small sequence processing utilities.

Seq Crumbs is modeled after the Unix command line text processing utilities, so every utility tries to perform a specific task. Most of them take a sequence file as input and create a new processed sequence file as output. This design encourages chaining the operation of multiple Seq Crumbs utilities using Unix pipes.

You can find more information about seq_crumbs in the seq_crumbs web site.

Available Crumbs

Extracts the reads from a SFF file used by 454 and Ion Torrent.
Splits the mate-pairs separated by an oligo sequence.
Filters the sequences according to its mean quality.
Filter out identical sequences.
Filters sequences according to a length threshold.
Filters the sequences with a list of names given in a file.
Filters the sequences using BLAST.
Filters the sequences using bowtie2.
Filters the sequences according to their complexity.
Trims the sequences according to case.
Removes a fixed number of residues from the sequence edges.
Removes, using a sliding window, regions of low quality in the edges.
Removes oligonucleotides by using the blast-short algorithm.
Converts between different supported sequence formats.
Guesses the format of a file, including Sanger and Illumina fastq formats.
Concatenates one or more input sequence files, possibly in different formats, into one output.
Outputs only the first sequences of the given input.
Outputs a random sampling of the input sequences.
Modifies the letter case of the sequences. The case can be converted to lower or upper case or swapped.
Filters out orphaned read pairs.
Interleaves two ordered paired read files.
Splits an ordered file of paired reads into two files, one for each end.
Generates basic statistics for the given sequence files.
Counts the number of sequences and the total sequence length for the given files.
Reverse complements transcripts according to polyA, ORF or blast hits.

General Usage

All seq crumbs try to share a consistent interface. By default most Seq Crumbs read from standard input and write to standard output, allowing them to to be easily combined using Unix pipes. Alternatively, several input sequence files can be provided as a list of arguments. Output can also be directed to specific files with the -o parameter (or --outfile).

seq_crumbs supports compressed gzip, BGZF and bzip2 files. When used as input it autodetects the compressed files. It can also generate compressed outputs.

The sequence formats accepted by seq_crumbs are those supported by Biopython's SeqIO module. As output only Sanger and Illumina fastq and fasta files are supported.

seq_crumbs can take advantage of multiprocessor computers by splitting the computational load into several processes.

The filtering seq crumbs can be made aware of paired reads and can filter both reads of pairs at once.


seq_crumbs depends on Python 2.7. Biopython is a recommended dependency. The installation manual is located in the doc/install.rst document.

Related software

seq_crumbs relies heavily on Biopython and without this free software project it wouldn't be able to provide some of its functionality.

Biopieces is a project with a scope similar to seq_crumbs. Biopieces is a great software project for working with different kinds of biological data using Unix pipes. seq_crumbs tries to be more limited in its scope, limiting itself only to sequence files and thus providing a somewhat simpler interface.

Another software package very similar in approach to seq_crumbs is the nice fastx collection.

Other related software: PRINSEQ, ea-utils, Pyrocleaner, Sequence Cleaner, lucy, NGS QC Toolkit, scythe, sickle, cutadapt, trimomatic and FastQC.


Seq Crumbs is free software. Licensed mainly under the General Public License (GPL). For more details on the licensing take a look at the LICENSE.txt file.