Pipeline

[mictadlo]

Hi,
I was able to follow the first 4 steps of the DASCRUBBER pipeline but I am not sure how to do 5 - 8 steps.

5. DASmap [-p] <path:db> [ <reads:FILE> |<reads:range> ... ]. What are <reads:FILE> |<reads:range>. Is path:db the patch:db?
6. DASrealign [-v] [-l<int:(800)>] <block1> <block2> <source:las> <target:las>. What are -l option, <block1> <block2>, <source:las> and <target:las>?
7. REPqv <subject:db> .... Is subject:db the source or patch db?
8. REPtrim <subject:db> ... Is subject:db the source or patch db?

Below is what I have done so far:

#DASqv_pbs.sh

#!/bin/sh
# Source: https://unix.stackexchange.com/a/421328/34872
# Usage: ./DASqv_pbs.sh plantDB 6973 38
# use the first 3 arguments for the values to pass to DASqv
db="$1"
H="$2"
cov="$3"

# use shift to get rid of them once we have them in variables, ...
shift 3

# ... so we can loop over the remaining filenames (1 or more) on the command line
for filename in $(find . -type f -name "*.*.las"); do
  outfile="$(basename "$filename" .las).DAStrim"
  qsub <<EOF
#!/bin/bash -l

#PBS -N DASqv
#PBS -l walltime=48:00:00
#PBS -j oe
#PBS -l mem=1G
#PBS -l ncpus=1
#PBS -M m.lorenc@qut.edu.au
##PBS -m bea

cd "\$PBS_O_WORKDIR"

source activate thegenemyers

DASqv -v -H"$H" -c"$cov" "$db" "$filename" | 
  sed -n -e "/Recommend/ { s/Recommend //;
                           s/'//g;
                           s:$: $db $filename:;
                           p }" > "$outfile"
EOF

done

./DASqv_pbs.sh plantDB 6973 38               #MEM:0.5GB; CPU time:00:00:19
Catrack -v plantDB qual

#DAStrim

#!/bin/bash
# sh HPC.daligner_pbs.sh
while IFS='' read -r line || [[ -n "$line" ]]; do
  cmd=$line 

  #cat <<EOF
  qsub <<EOF
#!/bin/bash -l

#PBS -N HPCdaligner
#PBS -l walltime=10:00:00
#PBS -j oe
#PBS -l mem=5G
#PBS -l ncpus=1
#PBS -M m.lorenc@qut.edu.au
###PBS -m bea   

cd \$PBS_O_WORKDIR

source activate thegenemyers
echo $cmd
$cmd

EOF

done < "$1"

find . -name "*.DAStrim" -type f -exec cat {} + > DAStrim-cmds
sh HPC.parallel_pbs.sh DAStrim-cmds           #MEM:1.1GB; CPU time:00:00:26
Catrack -v plantDB trim  

#DASpatch_pbs.sh

#!/bin/sh
db="$1"

# ... so we can loop over the remaining filenames (1 or more) on the command line
for filename in $(find . -type f -name "*.*.las"); do
  #cat << EOF
  qsub <<EOF
#!/bin/bash -l

#PBS -N DASpatch
#PBS -l walltime=48:00:00
#PBS -j oe
#PBS -l mem=10G
#PBS -l ncpus=1
#PBS -M m.lorenc@qut.edu.au
##PBS -m bea

cd "\$PBS_O_WORKDIR"

source activate thegenemyers

DASpatch -v $db $filename

EOF

done

sh DASpatch_pbs.sh plantDB                 #MEM:1.4GB; CPU time:00:00:16
Catrack -v plantDB patch  

#DASedit

DASedit -v plantDB plantDB-DASedit                         #MEM:0.9GB; CPU time:02:04:50

So far I have got the following files:

plantDB.100.las  plantDB.133.las  plantDB.166.las  plantDB.199.las  plantDB.230.las  plantDB.263.las  plantDB.296.las  plantDB.3.las   plantDB.72.las      daligner.01.OVL
plantDB.101.las  plantDB.134.las  plantDB.167.las  plantDB.19.las   plantDB.231.las  plantDB.264.las  plantDB.297.las  plantDB.40.las  plantDB.73.las      daligner.02.CHECK.OPT
plantDB.102.las  plantDB.135.las  plantDB.168.las  plantDB.1.las    plantDB.232.las  plantDB.265.las  plantDB.298.las  plantDB.41.las  plantDB.74.las      daligner.03.MERGE
plantDB.103.las  plantDB.136.las  plantDB.169.las  plantDB.200.las  plantDB.233.las  plantDB.266.las  plantDB.299.las  plantDB.42.las  plantDB.75.las      daligner.04.CHECK.OPT
plantDB.104.las  plantDB.137.las  plantDB.16.las   plantDB.201.las  plantDB.234.las  plantDB.267.las  plantDB.29.las   plantDB.43.las  plantDB.76.las      daligner.05.RM.OPT
plantDB.105.las  plantDB.138.las  plantDB.170.las  plantDB.202.las  plantDB.235.las  plantDB.268.las  plantDB.2.las    plantDB.44.las  plantDB.77.las      daligner.06.MERGE
plantDB.106.las  plantDB.139.las  plantDB.171.las  plantDB.203.las  plantDB.236.las  plantDB.269.las  plantDB.300.las  plantDB.45.las  plantDB.78.las      daligner.07.CHECK.OPT
plantDB.107.las  plantDB.13.las   plantDB.172.las  plantDB.204.las  plantDB.237.las  plantDB.26.las   plantDB.301.las  plantDB.46.las  plantDB.79.las      daligner.08.RM
plantDB.108.las  plantDB.140.las  plantDB.173.las  plantDB.205.las  plantDB.238.las  plantDB.270.las  plantDB.302.las  plantDB.47.las  plantDB.7.las       Dascrubber_pbs.sh
plantDB.109.las  plantDB.141.las  plantDB.174.las  plantDB.206.las  plantDB.239.las  plantDB.271.las  plantDB.303.las  plantDB.48.las  plantDB.80.las      DASedit.o2777006
plantDB.10.las   plantDB.142.las  plantDB.175.las  plantDB.207.las  plantDB.23.las   plantDB.272.las  plantDB.304.las  plantDB.49.las  plantDB.81.las      DASedit.o2777039
plantDB.110.las  plantDB.143.las  plantDB.176.las  plantDB.208.las  plantDB.240.las  plantDB.273.las  plantDB.305.las  plantDB.4.las   plantDB.82.las      DASedit_pbs.sh
plantDB.111.las  plantDB.144.las  plantDB.177.las  plantDB.209.las  plantDB.241.las  plantDB.274.las  plantDB.306.las  plantDB.50.las  plantDB.83.las      DASmap_pbs.sh
plantDB.112.las  plantDB.145.las  plantDB.178.las  plantDB.20.las   plantDB.242.las  plantDB.275.las  plantDB.307.las  plantDB.51.las  plantDB.84.las      DASpatch_pbs.sh
plantDB.113.las  plantDB.146.las  plantDB.179.las  plantDB.210.las  plantDB.243.las  plantDB.276.las  plantDB.308.las  plantDB.52.las  plantDB.85.las      DASqv_pbs.sh
plantDB.114.las  plantDB.147.las  plantDB.17.las   plantDB.211.las  plantDB.244.las  plantDB.277.las  plantDB.309.las  plantDB.53.las  plantDB.86.las      DAStrim-cmds
plantDB.115.las  plantDB.148.las  plantDB.180.las  plantDB.212.las  plantDB.245.las  plantDB.278.las  plantDB.30.las   plantDB.54.las  plantDB.87.las      DBstats.out
plantDB.116.las  plantDB.149.las  plantDB.181.las  plantDB.213.las  plantDB.246.las  plantDB.279.las  plantDB.310.las  plantDB.55.las  plantDB.88.las      fasta2DB_input.fofn
plantDB.117.las  plantDB.14.las   plantDB.182.las  plantDB.214.las  plantDB.247.las  plantDB.27.las   plantDB.311.las  plantDB.56.las  plantDB.89.las      fasta2DB_pbs.sh
plantDB.118.las  plantDB.150.las  plantDB.183.las  plantDB.215.las  plantDB.248.las  plantDB.280.las  plantDB.312.las  plantDB.57.las  plantDB.8.las       HPC.parallel_pbs.sh
plantDB.119.las  plantDB.151.las  plantDB.184.las  plantDB.216.las  plantDB.249.las  plantDB.281.las  plantDB.313.las  plantDB.58.las  plantDB.90.las      REPmask.01.OVL
plantDB.11.las   plantDB.152.las  plantDB.185.las  plantDB.217.las  plantDB.24.las   plantDB.282.las  plantDB.314.las  plantDB.59.las  plantDB.91.las      REPmask.02.CHECK.OPT
plantDB.120.las  plantDB.153.las  plantDB.186.las  plantDB.218.las  plantDB.250.las  plantDB.283.las  plantDB.315.las  plantDB.5.las   plantDB.92.las      REPmask.03.MASK
plantDB.121.las  plantDB.154.las  plantDB.187.las  plantDB.219.las  plantDB.251.las  plantDB.284.las  plantDB.316.las  plantDB.60.las  plantDB.93.las      REPmask.04.RM
plantDB.122.las  plantDB.155.las  plantDB.188.las  plantDB.21.las   plantDB.252.las  plantDB.285.las  plantDB.317.las  plantDB.61.las  plantDB.94.las      TANmask.01.OVL
plantDB.123.las  plantDB.156.las  plantDB.189.las  plantDB.220.las  plantDB.253.las  plantDB.286.las  plantDB.318.las  plantDB.62.las  plantDB.95.las      TANmask.02.CHECK.OPT
plantDB.124.las  plantDB.157.las  plantDB.18.las   plantDB.221.las  plantDB.254.las  plantDB.287.las  plantDB.319.las  plantDB.63.las  plantDB.96.las      TANmask.03.MASK
plantDB.125.las  plantDB.158.las  plantDB.190.las  plantDB.222.las  plantDB.255.las  plantDB.288.las  plantDB.31.las   plantDB.64.las  plantDB.97.las      TANmask.04.RM
plantDB.126.las  plantDB.159.las  plantDB.191.las  plantDB.223.las  plantDB.256.las  plantDB.289.las  plantDB.32.las   plantDB.65.las  plantDB.98.las      tmp
plantDB.127.las  plantDB.15.las   plantDB.192.las  plantDB.224.las  plantDB.257.las  plantDB.28.las   plantDB.33.las   plantDB.66.las  plantDB.99.las
plantDB.128.las  plantDB.160.las  plantDB.193.las  plantDB.225.las  plantDB.258.las  plantDB.290.las  plantDB.34.las   plantDB.67.las  plantDB.9.las
plantDB.129.las  plantDB.161.las  plantDB.194.las  plantDB.226.las  plantDB.259.las  plantDB.291.las  plantDB.35.las   plantDB.68.las  plantDB-DASedit.db
plantDB.12.las   plantDB.162.las  plantDB.195.las  plantDB.227.las  plantDB.25.las   plantDB.292.las  plantDB.36.las   plantDB.69.las  plantDB.db
plantDB.130.las  plantDB.163.las  plantDB.196.las  plantDB.228.las  plantDB.260.las  plantDB.293.las  plantDB.37.las   plantDB.6.las   bwa_pbs.sh
plantDB.131.las  plantDB.164.las  plantDB.197.las  plantDB.229.las  plantDB.261.las  plantDB.294.las  plantDB.38.las   plantDB.70.las  Catrack.o2678948
plantDB.132.las  plantDB.165.las  plantDB.198.las  plantDB.22.las   plantDB.262.las  plantDB.295.las  plantDB.39.las   plantDB.71.las  catrackTAN_pbs.sh

Thank you in advance,

Michal