140715_7001448.A.Project_Diag excap27 2014 07 03

#140715_7001448.A.Project_Diag-excap27-2014-07-03
28.07.2014

Received Arvind Email 27.07.2014

Requirement

1. Mount the following drives:

• smb://192.168.1.100:/runScratch/ DONE
  change smb://192.168.1.101 to smb://192.168.1.100
• smb://192.168.1.31:/data.odin/ DONE
• smb://192.168.1.31:/data1.odin/ DONE
• "condor_sh" should be under root DONE

2. R and Python libraries

Everything is present

3. TaqMan SNP fingerprinting Results

The TaqMan results should be put fingerprinting results should be under: DONE

/Volumes/data1.odin/001_exome/diagnosticSamples/TaqManSNP-ID/ 

---

Running Variant Calling Pipeline

1. Mapping the reads

• Copy fastq files to "condor_sh/projects"
  If the fastq files are under "Project_Diag-excap27-2014-07-03" from the run "140715_7001448_0269_AC4843ACXXX".
  see tranfert log in "140715_7001448_0269_AC4843ACXXX_transfert.md"

cd /Volumes/runScratch/completed/1407/140715_7001448_0269_AC4843ACXX/Unaligned/
rsync -rRuaP 140715_7001448.A.Project_Diag-excap27-2014-07-03 /condor_sh/projects/  
cd /condor_sh/projects/  
mv 140715_7001448.A.Project_Diag-excap27-2014-07-03 Project_Diag-excap27-2014-07-03

• Change ownership and permissions of the project folder (Project_Diag-excap20-2014-02-24)

- log into tor
su
root@tor:/condor_sh/projects #chmod -R 777 Project_Diag-excap27-2014-07-03/
root@tor:/condor_sh/projects #chown -R SBSUser:SBSUser Project_Diag-excap27-2014-07-03/
- log out

• Run the perl script (generateCMDmappingCondor.pl) to prepare the condor job
  see output ehere "generateCMDmappingCondor_Project_Diag-excap27-2014-07-03.md"

perl /Volumes/data.odin/diagnosticBundle/script/amg/variantcalling/pipeline/pipeline_current/generateCMDmappingCondor.pl

• start the condor job

  • log on Tor (192.168.1.32)
  • run for each sample the "commandMapping.bash" located in the data directory. You can submit 4~5 alignment per day
    cd $cd /condor_sh/projects/Project_Diag-excap27-2014-07-03/
bash Sample_Diag-excap27-14000568901-EE-KIT-Av5/data/commandMapping.bash
bash Sample_Diag-excap27-14000551601-EE-KIT-Av5/data/commandMapping.bash
bash Sample_Diag-excap27-14000521001-EE-KIT-Av5/data/commandMapping.bash

• Check results

  • Mapping takes around 10 hours on the current setup, once the mapping finished the mapping directory should be around 80-100 GB. One can also have a look into the /mapping_*/log directory where there should contain three log files, and there is "Done" in *sam.out, and there is no "Exception" in collectAlignmentSummary.out and *bam.out files.

2. Starting the variant calling

• On the working machine, create a directory named after the project name.

• This group contains only two samples, use the runGroupPipeline script to call the variant.

  mkdir Project_Diag-excap27-2014-07-03 **DONE**

• Move into the newly created directory and start the pipeline by calling the runGroupPipeline.pl with the group.conf as an argument.

  cd Project_Diag-excap27-2014-07-03
perl /Volumes/data.odin/diagnosticBundle/script/amg/variantcalling/pipeline/pipeline_current/runGroupPipeline.pl /condor_sh/projects/Project_Diag-excap27-2014-07-03/group_1.conf &> Project_Diag-excap27-2014-07-03_group_1.log

• To start the last sample's analysis, use the variantCallingPipeline script.

  cd /Volumes/Analysis/Exome/Project_Diag-excap27-2014-07-03/
mkdir Sample_Diag-excap27-14000568901-EE-KIT-Av5
cd Sample_Diag-excap27-14000568901-EE-KIT-Av5
perl /Volumes/data.odin/diagnosticBundle/script/amg/variantcalling/pipeline/pipeline_current/variantCallingPipeline.pl /condor_sh/projects/Project_Diag-excap27-2014-07-03/Sample_Diag-excap27-14000568901-EE-KIT-Av5/data/sample.conf &> ../Sample_Diag-excap27-14000568901-EE-KIT-Av5.log

3. Checks

• The fastest way of checking if everything went well is to check the size and/or the content of the files starting with err*. Most of them should be empty or without "Exception" in the files.
  To do so I wrote a script called checkErrors.py which check for these conditions. It has to be run in the sample's directory. The current version takes the number of sec on between two check in order to follow what's going on.

  python ~/Home/workspace/HTSPipeline/checkErrors.py 1

• If the pipeline crashes, one can restart it and it should pick up where it has been stopped.

• For the fingerprinting test, the scripts are incorporated in the variant calling pipeline. If any of the tests fail the result file will have the _NEED_REVIEW extension and one should open the file to check the problem. The file is located in :

  Project_Diag-excap20-2014-02-24/SampleXXX/070_QC/

4. Post processing

• Once the variant calling pipeline has finish and all checks have been done, the last steps is to run the postProcess.pl script from the project folder. The script will check , rename and delete redundant files in the different Sample folder. Once the script finishes, one can safely transfer the Sample folders to their respective location on loki:/data/diag/samples

perl /Volumes/data.odin/diagnosticBundle/script/amg/variantcalling/pipeline/pipeline_current/postProcess.pl

5. Fill in "Bioinformatic" section in "HTS database" in hospital computer under /R/lab/HTS

• DONE