Skip to content
This repository was archived by the owner on Dec 18, 2020. It is now read-only.

Project 1

Arjun Prasad edited this page Nov 23, 2020 · 19 revisions

Step 0: Set up a work directory

You will need to log into your newly created VM by clicking the 'SSH' link.

cd
mkdir project1
cd project1

Step 1: Get the data

We've done this task for you because it is complex, time consuming, and illustrates a problem with the way data is included from many papers today. Little thought is given to how the data can be accessed even when it is "publicly available". To get the dataset to do further analysis, here's what we did Project 1: Get IDs. You will just copy the processed list of accessions into a work directory

cd ~/project1
mkdir step1
cd step1
    gsutil cp gs://pathogen-workshop/project1/step1/singapore_samples_srafull.txt .

Step 2: Assemble isolates using SKESA

We've used SKESA to assemble the isolates because it is a fast and high-accuracy assembler. You could use another, more thorough assembler such as SPAdes to gain some additional sequence, but at the cost of some expense and accuracy.

We're going to set you up to do an assembly in parallel using dsub. In the interests of saving some compute we'll set up for computation of just two assemblies, but the same setup was used to do all 106 assemblies from the paper.

Create a working directory

cd ~/project1
mkdir step2
cd step2

Link in the sample accession file from the previous step

    # Link in the sample accession file (from the previous step)
    ln -s ../step1/singapore_samples_srafull.txt

Create the submit_list.tsv file which is used to tell dsub what parameters to parallelize

    # Small script to create the submit_list.tsv file for dsub
    cat <<END > create_submit_list.sh
cut -f 3 singapore_samples_srafull.txt \
| perl -ne 'BEGIN { print "--env ERR\t--output ASM\n" }
        chomp;
        print join("\t",
                "\$_",
                "gs://asm-ngs-$USER/project1/step2/asm/\$_.asm.fa"), "\n"
        '
END
    bash create_submit_list.sh > submit_list.tsv
 
    # For the workshop we want to just run two assemblies
    head -3 submit_list.tsv > submit_list_two.tsv 

Create the dsub command to perform the assembly

    # The assembly command. I like to create a script for easier editing and record-keeping
        cat > assemble <<'END'
dsub \\
  --provider google-cls-v2 \\
  --project ncbi-pd-amr \\
  --regions us-central1 \\
  --logging gs://asm-ngs-$USER/project1/logs \\
  --image staphb/skesa \\
  --tasks submit_list_two.tsv \\
  --command   'skesa --sra_run ${ERR} --contigs_out ${ASM} --cores 6 ' \\
  --min-ram 24 \\
  --min-cores 6 \\
  --wait
END

Run the assembly command

    # Run the assemblies in parallel on new VMs using dsub
    # You should see new VMs created and running in the VM instances list
    time bash -x assemble 2>&1 > assemble.out

You should have to wait 5-10 minutes for this to finish.

Copy the assemblies you just made from the bucket to your VM disk

    # copy the assemblies from the bucket onto your local disk.
    mkdir asm
    gsutil -m cp gs://asm-ngs-$USER/acinetobacter/asm/* asm/

You should now be able to see the FASTA assemblies you just created.

    ls -l asm/*

Step 3: Run AMRFinderPlus on the assemblies

mkdir ~/project1/step3
cd ~/project1/step3
...

Clone this wiki locally