* dDocentHPC v4.6 Forked by cbird@tamucc.edu * Running dDocentHPC fltrBAM in OVERWRITE mode. \n Sample names, file indexing, etc will be overwritten to clear errors... Fr 22. Dez 10:36:55 CET 2023 Files output to: /home/bt140047/Dokumente/FK1/dD-Fp3-HPC/mkBAM Fr 22. Dez 10:36:55 CET 2023 Reading config file... Settings for dDocentHPC #Made in notepad++, use for best viewing These default settings assume ddRAD, no overlapping 151 bp reads All Number of Processors (Not Adjustable) All Free Memory (Not Adjustable) ----------trimFQ: Settings for Trimming FASTQ Files--------------------------------------------------------------- 146 trimmomatic MINLEN (integer, mkREF only) Drop the read if it is below a specified length. Set to the length of the Read1 reads. 75 trimmomatic MINLEN (integer, mkBAM only) Drop the read if it is below a specified length. Set to the minimum frag length you want mapped to the reference. 20 trimmomatic LEADING: (integer, mkBAM only) Specifies the minimum quality required to keep a base. 15 trimmomatic TRAILING: (integer, mkREF only) Specifies the minimum quality required to keep a base. 20 trimmomatic TRAILING: (integer, mkBAM only) Specifies the minimum quality required to keep a base. TruSeq3-PE-2.fa trimmomatic ILLUMINACLIP: (0, fasta file name) Specifies the trimmomatic adapter file to use. entering a 0 (zero) will turn off adapter trimming. Options are: TruSeq3-PE-2.fa, TruSeq3-PE.fa, TruSeq3-SE.fa, TruSeq2-PE.fa, TruSeq2-SE.fa, any other files included with trimmomatic. Entering a custom path here will break the script. If you want a customized file, you have to put it where the default trimmomatic files are located on your computer. If you have trouble finding this location, run dDocentHPC trimREF and it will be included in the output. 2 trimmomatic ILLUMINACLIP: (integer) specifies the maximum mismatch count which will still allow a full match to be performed 30 trimmomatic ILLUMINACLIP: (integer) specifies how accurate the match between the two 'adapter ligated' reads must be for PE palindrome read alignment 10 trimmomatic ILLUMINACLIP: (integer) specifies how accurate the match between any adapter etc. sequence must be against a read. 20 trimmomatic SLIDINGWINDOW: (integer) specifies the number of bases to average across 20 trimmomatic SLIDINGWINDOW: (integer) specifies the average quality required. 0 trimmomatic CROP: (integer, mkBAM only) Trim read sequences down to this length. Enter 0 for no cropping 0 trimmomatic HEADCROP: (integer, only Read1 for ezRAD) The number of bases to remove from the start of the read. 0 for ddRAD, 5 for ezRAD no FixStacks (yes,no) Demultiplexing with stacks introduces anomolies. This removes them. ------------------------------------------------------------------------------------------------------------------ ----------mkREF: Settings for de novo assembly of the reference genome-------------------------------------------- PE Type of reads for assembly (PE, SE, OL, RPE) PE=ddRAD & ezRAD pairedend, non-overlapping reads; SE=singleend reads; OL=ddRAD & ezRAD overlapping reads, miseq; RPE=oregonRAD, restriction site + random shear 0.9 cdhit Clustering_Similarity_Pct (0-1) Use cdhit to cluster and collapse uniq reads by similarity threshold 2 Cutoff1 (integer) Use unique reads that have at least this much coverage for making the reference genome 2 Cutoff2 (integer) Use unique reads that occur in at least this many individuals for making the reference genome 0.05 rainbow merge -r (decimal 0-1) Percentile-based minimum number of seqs to assemble in a precluster 0.95 rainbow merge -R (decimal 0-1) Percentile-based maximum number of seqs to assemble in a precluster ------------------------------------------------------------------------------------------------------------------ ----------mkBAM: Settings for mapping the reads to the reference genome------------------------------------------- Make sure the cutoffs above match the reference*fasta! 1 bwa mem -A Mapping_Match_Value (integer) bwa mem default is 1 6 bwa mem -B Mapping_MisMatch_Value (integer) bwa mem default is 4 10 bwa mem -O Mapping_GapOpen_Penalty (integer) bwa mem default is 6 50 bwa mem -T Mapping_Minimum_Alignment_Score (integer) bwa mem default is 30. Remove reads that have an alignment score less than this. don't go lower than 1 or else the resulting file will be huge. NOTE! in fltrBAM settings (below) there is an alignment score filter that uses a threshold relative to read length. This -T setting here affects which reads the relative alignment score threshold will be applied to. 30,5 bwa mem -L Mapping_Clipping_Penalty (integer,integer) bwa mem default is 5 ------------------------------------------------------------------------------------------------------------------ ----------fltrBAM: Settings for filtering mapping alignments in the *bam files--------------- 20 samtools view -q Mapping_Min_Quality (integer) Remove reads with mapping qual less than this value no samtools view -F 4 Remove_unmapped_reads? (yes,no) Since the reads aren't mapped, we generally don't need to filter them yes samtools view -F 8 Remove_read_pair_if_one_is_unmapped? (yes,no) If either read in a pair does not map, then the other is also removed yes samtools view -F 256 Remove_secondary_alignments? (yes,no) Secondary alignments are reads that also map to other contigs in the reference genome no samtools view -F 512 Remove_reads_not_passing_platform_vendor_filters (yes,no) We generally don't see any of these no samtools view -F 1024 Remove_PCR_or_optical_duplicates? (yes,no) You probably don't want to set this to yes no samtools view -F 2048 Remove_supplementary_alignments? (yes,no) We generally don't see any of these yes samtools view -f 2 Keep_only_properly_aligned_read_pairs? (yes,no) Set to no if OL mode 0 samtools view -F Custom_samtools_view_F_bit_value? (integer) performed separately from the above, consult samtools man 0 samtools view -f Custom_samtools_view_f_bit_value? (integer) performed separately from the above, consult samtools man 20 Remove_reads_with_excessive_soft_clipping? (no, integers) minimum number of soft clipped bases in a read, summed between the beginning and end, that are unacceptable 50 Remove_reads_with_alignment_score_below_relative_threshold (integer) Alignment score thresholds are calculated based on this value adjusted by a factor (actual read length relative the assumed read length value in next setting). RelativeThreshold = as_threshold * actual_read_length / assumed_read_length, where this setting controls as_threshold. NOTE! bwa mem -T affects which reads are mapped based on alignment score, and therefore this filter cannot save reads elimated by bwa mem -T, but if the -T setting is too low then the RAW bam files can be huge. 150 Read_length_assumed_by_relative_alignment_score_threshold (integer) Alignment score thresholds are calculated based on the threshold in the previous setting adjusted by a factor (actual read length relative the assumed read length value here). RelativeThreshold= as_threshold * actual_read_length / assumed_read_length, where this setting controls assumed_read_length yes Remove_reads_orphaned_by_filters? (yes,no) ------------------------------------------------------------------------------------------------------------------ ----------mkVCF: Settings for variant calling/ genotyping--------------------------------------------------------- no freebayes -J --pooled-discrete (yes|no) If yes, a pool of individuals is assumed to be the statistical unit of observation. no freebayes -A --cnv-map (filename.bed or no) If the pools have different numbers of individuals, then you should provide a copy number variation (cnv) *.bed file with the "ploidy" of each pool. the bed file should be in the working directory and formatted as follows: popmap_column_1 ploidy_of_pool. If that doesn't work, try the basenames of the files in popmap column 1. 2 freebayes -p --ploidy (integer) Whether pooled or not, if no cnv-map file is provided, then what is the ploidy of the samples? for pools, this number should be the number of individuals * ploidy no freebayes -r --region (filename.bed or no) Limit analysis to specified region. Bed file format: :- 0 only genotype read 1 (integer) Limit analysis to only Read 1 positions, integer is maximum Read1 bp position 2 Minimum Mean Depth of Coverage Per Individual Limit analysis to contigs with at least the specified mean depth of coverage per individual 0 freebayes -n --use-best-n-alleles (integer) reduce the number of alleles considered to n, zero means all, set to 2 or more if you run out of memory 30 freebayes -m --min-mapping-quality (integer) 20 freebayes -q --min-base-quality (integer) -1 freebayes -E --haplotype-length (-1, 3, or integer) Set to -1 to avoid multi nucleotide polymorphisms and force calling MNPs as SNPs. Can be set up to half the read length, or more. 0 freebayes --min-repeat-entropy (0, 1, or integer) Set to 0 to avoid multi nucleotide polymorphisms and force calling MNPs as SNPs. To detect interrupted repeats, build across sequence until it has entropy > N bits per bp. 10 freebayes --min-coverage (integer) Require at least this coverage to process a site 0.375 freebayes -F --min-alternate-fraction (decimal 0-1) There must be at least 1 individual with this fraction of alt reads to evaluate the position. If your individuals are barcoded, then use 0.2. If your data is pooled, then set based upon ~ 1/(numIndivids * ploidy) and average depth of coverage 20 freebayes -C --min-alternate-count (integer) Require at least this count of observations supporting an alternate allele within a single individual in order to evaluate the position. default: 2 20 freebayes -G --min-alternate-total (integer) Require at least this count of observations supporting an alternate allele within the total population in order to use the allele in analysis. default: 1 0.33 freebayes -z --read-max-mismatch-fraction (decimal 0-1) Exclude reads with more than N [0,1] fraction of mismatches where each mismatch has base quality >= mismatch-base-quality-threshold default: 1.0 20 freebayes -Q --mismatch-base-quality-threshold (integer) Count mismatches toward --read-mismatch-limit if the base quality of the mismatch is >= Q. default: 10 30 freebayes -U --read-mismatch-limit (integer) Exclude reads with more than N mismatches where each mismatch has base quality >= mismatch-base-quality-threshold. default: ~unbounded 30 freebayes ~3 ~~min-alternate-qsum (integer) This value is the mean base quality score for alternate reads and will be multiplied by -C to set -3. Description of -3: Require at least this count of observations supporting an alternate allele within a single individual in order to evaluate the position. default: 2 30 freebayes -$ --read-snp-limit (integer) Exclude reads with more than N base mismatches, ignoring gaps with quality >= mismatch-base-quality-threshold. default: ~unbounded 20 freebayes -e --read-indel-limit (integer) Exclude reads with more than N separate gaps. default: ~unbounded no freebayes -w --hwe-priors-off (no|yes) Disable estimation of the probability of the combination arising under HWE given the allele frequency as estimated by observation frequency. no freebayes -V --binomial-obs-priors-off (no|yes) Disable incorporation of prior expectations about observations. Uses read placement probability, strand balance probability, and read position (5'-3') probability. no freebayes -a --allele-balance-priors-off (no|yes) Disable use of aggregate probability of observation balance between alleles as a component of the priors. no freebayes --no-partial-observations (no|yes) Exclude observations which do not fully span the dynamically-determined detection window. (default, use all observations, dividing partial support across matching haplotypes when generating haplotypes.) no freebayes --report-monomorphic (no|yes) Report even loci which appear to be monomorphic, and report allconsidered alleles, even those which are not in called genotypes. Loci which do not have any potential alternates have '.' for ALT. ------------------------------------------------------------------------------------------------------------------ Email user@tamucc.edu Fr 22. Dez 10:36:55 CET 2023 Reading in variables from config file... Fr 22. Dez 10:36:55 CET 2023 Checking for all required dDocent software... Running CheckDependencies Function... The dependency trimmomatic is installed! The dependency freebayes is installed! The dependency mawk is installed! The dependency bwa is installed! The dependency samtools is installed! The dependency vcftools is installed! The dependency rainbow is installed! The dependency gnuplot is installed! The dependency gawk is installed! The dependency seqtk is installed! The dependency cd-hit-est is installed! The dependency bamToBed is installed! The dependency bedtools is installed! The dependency coverageBed is installed! The dependency parallel is installed! The dependency vcfcombine is installed! The dependency bamtools is installed! The dependency pearRM is installed! All dependencies are installed and up to date! Fr 22. Dez 10:36:55 CET 2023 Begin main ddocent function The HPC version of dDocent will only digest files with particular extensions for particular tasks untouched files for trimming must be *.F.fq.gz and *.R.fq.gz files trimmed for assembly (mkREF) must be *r1.fq.gz *r2.fq.gz files trimmed for mapping (mkBAM) must be *R1.fq.gz *R2.fq.gz extensions selected: *.R1.fq.gz *.R2.fq.gz The samples being processed are: AT_AT1Fp AT_AT2Fp CZ_CZ14Fp CZ_CZ15Fp CZ_CZ16Fp CZ_CZ17Fp CZ_CZ19Fp CZ_CZ1Fp CZ_CZ20Fp CZ_CZ21Fp CZ_CZ22Fp CZ_CZ23Fp CZ_CZ24Fp CZ_CZ2Fp CZ_CZ4Fp CZ_CZ5Fp DE_DE13Fp DE_DE1Fp DE_DE2Fp DE_DE8Fp DE_DE9Fp EE_EE1Fp EE_EE2Fp EE_EE3Fp EE_EE4Fp EE_EE5Fp FI_FI1Fp FI_FI2Fp FI_FI3Fp HR_HR12Fp HR_HR1Fp HR_HR5Fp LT_LT1Fp LT_LT2Fp LT_LT3Fp LT_LT4Fp SK_SK1Fp SK_SK2Fp SK_SK3Fp SK_SK4Fp SK_SK5Fp SOFT_CLIP_CUT=20 FILTER_ORPHANS=yes MAPPING_MIN_QUALITY=20 SAMTOOLS_VIEW_F=264 SAMTOOLS_VIEW_f=2 SAMTOOLS_VIEW_Fcustom=0 SAMTOOLS_VIEW_fcustom=0 FILTER_MIN_AS=50 FILTER_MIN_AS_LEN=150 Fr 22. Dez 10:36:56 CET 2023 Filtering bam files by Samtools Flags, Soft Clipped Bases, Alignment Scores relative to Read Length, and Orphans... Fr 22. Dez 10:37:31 CET 2023 Indexing the filtered BAM files... Fr 22. Dez 10:37:32 CET 2023 Filtering complete! Compress large filesFr 22. Dez 10:37:32 CET 2023