UPPMAX

Tutorials

• by UPPMAX
  • Introduction to Bioinformatics
    • ... but "Getting a node of your own" section does not work in a free study period.
  • Linux introduction (screencast)

How to run the pipeline on UPPMAX?

Installation

git clone -b v3dev https://github.com/shka/STRTprep.git STRTprep3.test
cd STRTprep3.test
bin/install_uppmax.sh

Protocol

• Step 1: (Copy pre-build indexes, or) build indexes of reference genome and transcriptome, like as below. See also a help document.

sbatch -A bXXXXXXX -t 12:00:00 -p core -n 1 <<EOF
#!/usr/bin/env bash
. bin/setup_uppmax.sh
bin/index_hg19_ercc92_ynbA_u13369.sh
bin/index_refGene.sh hg19 src/ebwt/hg19_ercc92_ynbA_u13369/ref
EOF

• Step 2: Edit conf.yaml and src/samples.csv
• Step 3: Run preprocessing and quality-check as follows, then check out/byGene/samples.xls
  • N at (i) should be equal to total lanes of sequences but the maximum is 16.
  • Please put job number of (i) into Xi at (ii), and of (ii) into Xii at (iii); then the latter jobs will run if the former jobs succeeded.
  • ... Please contact when you process multiple libraries urgently - there is a method to run the pipeline more faster.

sbatch -A b2014069 -t 24:00:00 -p core -n N <<EOF ## (i)
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=16 rake -m -j \$SLURM_NTASKS qc1 qc2
EOF
sbatch -A b2014069 -t 24:00:00 -p core -n 2 -d afterok:Xi <<EOF ## (ii)
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=16 rake qc3
EOF
sbatch -A b2014069 -t 24:00:00 -p node -n 16 -d afterok:Xii <<EOF ## (iii)
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=16 rake -m -j \$SLURM_NTASKS qc
EOF

• NOTE:
  • qc3 is frequently terminated by a memory overflow. In this case, check tmp/step2c/logs/tophat.log. If the log was terminated at "Reporting output tracks", then you can restart the qc tasks as below.

sbatch -A b2014069 -t 24:00:00 -p node -n 16 <<EOF
#!/usr/bin/env bash
. bin/setup_uppmax.sh
tophat -R tmp/step2c
PROCS=16 rake -m -j \$SLURM_NTASKS qc
EOF

TopHat2 requires bigger memory only in a few minutes at the end of run. And we cannot estimate size of the required memory; it depends on the library. However, we can easily waste the assigned calculation time, if we occupied many cores for all runtime of TopHat2 (task qc3).

• Step 4: Run differential expression tests and the down-stream analysis by plugins (for gene-based quantification) as follows.
  • Please extend time of job allocation (one hour), and change name of queue (devcore to core), when you met timeout
  • Please extend number of CPU allocation (one) when you met memory overflow

sbatch -A b2014069 -t 1:00:00 -p devcore -n 1 <<EOF
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=16 rake plugins_gene
EOF

• [v3dev] Step 5: Add "CLASS.TFE" column for classification of samples into src/samples.csv table, run differential expression tests, based on transcription-start-regions, then check out/byTFE/diffexp.xls and analyze more further!
  • Yi at (ii) is a job ID of (i).

sbatch -A b2014069 -t 24:00:00 -p core -n 8 <<EOF ## (i)
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=16 rake tfe1
EOF
sbatch -A b2014069 -t 24:00:00 -p node -n 16 -d afterok:Yi <<EOF ## (ii)
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=16 rake -m -j \$SLURM_NTASKS tfe
EOF

• Step 6: Run differential expression tests and the down-stream analysis by plugins for TFE-based quantification as follows.
  • Please extend time of job allocation (one hour), and change name of queue (devcore to core), when you met timeout
  • Please extend number of CPU allocation (one) when you met memory overflow

sbatch -A b2014069 -t 1:00:00 -p devcore -n 1 <<EOF
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=16 rake plugins_tfe
EOF