Skip to content
This repository was archived by the owner on Apr 20, 2026. It is now read-only.

Reference indexes

Jump to bottom
Rolf Schröder edited this page Nov 3, 2015 · 2 revisions

Already indexed references

Before creating a new reference index, check if it isn't already available in BaseSpace. If you need a reference genome that is not available in BaseSpace and if you do not have the resources to index the reference yourself, feel free to contact craczy@illumina.com.

The structure of that project is as follows:

  • Isaac Indexes analysis: top level directory for all indexes
  • FormatVersion6 - iSAAC-02.15.07 Output Files: top level directory for all sorted references with FormatVersion 6 (compatible with iSAAC-02.15)
    • hg19-packed-reference.tar.gz [1.6 GB]
    • mm9-packed-reference.tar.gz [1.7 GB]

These files are slightly larger than the corresponding reference genome compressed FASTA file. They contain all the information needed to generate the complete Isaac Reference Index, including the complete FASTA and all pre-calculated flags that Isaac needs to generate the index.

Download the packed reference for the genome you are interested in and simply use isaac-unpack-reference to get the sorted reference in a format suited for isaac-align:

export GENOME=mm9
export ISAAC=/opt/iSAAC/iSAAC-02.15.07.16
export ISAAC_REFERENCE=/data/iSAAC/FormatVersion6/${GENOME}
export DOWNLOAD=/tmp
mkdir ${ISAAC_REFERENCE}
cd ${ISAAC_REFERENCE}
${ISAAC}/bin/isaac-unpack-reference -j 1 -w 6 -i ${DOWNLOAD}/mm9-packed-reference.tar.gz

At the end the directory ${ISAAC_REFERENCE} should contain:

  • sorted-reference.xml: the description of the structure of the reference index
  • genome.fa: the original FASTA reference
  • 2uniqueness.16bpb.gz: the flags used by Isaac to quantify the "uniqueness" along the reference
  • kmer-positions-32-<chunk>.dat with `00 <= chunk < 63: the expanded binary index assiSociating 32-mers to reference positions, with appropriate flags

Indexing a reference from FASTA

The procedure is documented in the manual, however some areas deserve further explanation. As the indexing requires a considerable amount of resources (CPU time, memory and IOs), it is better suited to be distributed over a cluster. It is still possible to do it on a single server but there are a number of important considerations to take into account:

  • it takes in the order of a day to generate the index for hg19 on a single server with 150GB RAM and very good IOs
  • the IOs can easily become a very significant bottleneck and slowdown the process very significantly. There is a non-obvious workaround that should be systematically used

The bulk of the work to generate the index is to check for the uniqueness of short sequences (16-mer to 80-mer) in the reference. This also checks for similar sequences with up to 2 mismatches. This operation really requires a comparison of each k-mer against all the other k-mers in the reference. Unfortunately, in the current design, when isaac-sort-reference is invoked with the command line option -w set to any value greater than 0, the files are split from the beginning, significantly increasing the level of IOs for that operation, which can result in a major slowdown.

The recommendation when running isaac-sort-reference on a single server is to always use -w 0 and -j 1 and then to pack the reference (probably always a good idea) and finally unpack the reference with -w 6 to enable the parallel access to the index when running the alignment.

mkdir ${ISAAC_REFERENCE}-tmp
cd ${ISAAC_REFERENCE}-tmp
${ISAAC}/bin/isaac-sort-reference -o ./ -j 1 -w 0 -g ${DOWNLOAD}/genome.fa
${ISAAC}/bin/isaac-pack-reference -r sorted-reference.xml -o ${DOWNLOAD}/${GENOME}-packed-reference.tar.gz
mkdir ${ISAAC_REFERENCE}
cd ${ISAAC_REFERENCE}
${ISAAC}/bin/isaac-unpack-reference -w 6 -i ${DOWNLOAD}/${GENOME}-packed-reference.tar.gz
rm -rf Temp ${ISAAC_REFERENCE}-tmp

Clone this wiki locally