==================================== Bulk data analysis pipeline will run ============================================================== ## Input FASTQ folder: /dartfs-hpc/rc/lab/L/LeachLab/HIMANSHU/CutNrun/CutNRun_6-18-21/human ## Sample name: SL_04_S4 ## Workdir folder: /dartfs-hpc/rc/lab/L/LeachLab/HIMANSHU/CutNrun/CutNRun_6-18-21/SL_04_S4 ## Experiment name: SL_04_S4 ## Experiment type: CUT&RUN ## Reference genome: hg38 ## Spike-in genome: FALSE ## Spike-in normalization: FALSE ## Fragment 120 filtration: TRUE ================================================================================================================================= [info] Input file is SL_04_S4_R1_001.fastq.gz and SL_04_S4_R2_001.fastq.gz Wed Jul 21 19:04:53 EDT 2021 [info] Trimming file SL_04_S4 ... Wed Jul 21 19:04:53 EDT 2021 [info] Second stage trimming SL_04_S4 ... Wed Jul 21 19:21:11 EDT 2021 [info] Aligning file SL_04_S4 to reference genome... Wed Jul 21 19:33:18 EDT 2021 [info] Bowtie2 command: --dovetail --phred33 [info] The dovetail mode is enabled [as parameter frag_120 is on] [info] FASTQ files won't be aligned to the spike-in genome [info] Filtering unmapped fragments... SL_04_S4.bam Wed Jul 21 19:44:38 EDT 2021 [info] Sorting BAM... SL_04_S4.bam Wed Jul 21 19:46:42 EDT 2021 INFO 2021-07-21 19:46:43 SortSam ********** NOTE: Picard's command line syntax is changing. ********** ********** For more information, please see: ********** https://github.com/broadinstitute/picard/wiki/Command-Line-Syntax-Transition-For-Users-(Pre-Transition) ********** ********** The command line looks like this in the new syntax: ********** ********** SortSam -INPUT sorted/SL_04_S4.step1.bam -OUTPUT sorted/SL_04_S4.bam -SORT_ORDER coordinate -VALIDATION_STRINGENCY SILENT ********** 19:46:43.981 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/dartfs-hpc/rc/lab/L/LeachLab/HIMANSHU/CutNrun/CUT-RUNTools-2.0/install/picard-2.8.0.jar!/com/intel/gkl/native/libgkl_compression.so [Wed Jul 21 19:46:44 EDT 2021] SortSam INPUT=sorted/SL_04_S4.step1.bam OUTPUT=sorted/SL_04_S4.bam SORT_ORDER=coordinate VALIDATION_STRINGENCY=SILENT VERBOSITY=INFO QUIET=false COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false [Wed Jul 21 19:46:44 EDT 2021] Executing as f0042dk@n07.hpcc.dartmouth.edu on Linux 3.10.0-1160.31.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_292-b10; Deflater: Intel; Inflater: Intel; Provider GCS is not available; Picard version: 2.21.7-SNAPSHOT INFO 2021-07-21 19:46:44 SortSam Seen many non-increasing record positions. Printing Read-names as well. INFO 2021-07-21 19:47:16 SortSam Read 10,000,000 records. Elapsed time: 00:00:32s. Time for last 10,000,000: 32s. Last read position: chr6:108,935,318. Last read name: NB501031:621:HWYF2AFX2:2:11311:19284:7828 INFO 2021-07-21 19:47:49 SortSam Read 20,000,000 records. Elapsed time: 00:01:05s. Time for last 10,000,000: 32s. Last read position: chr1:19,153,505. Last read name: NB501031:621:HWYF2AFX2:3:21612:11919:16901 INFO 2021-07-21 19:48:10 SortSam Finished reading inputs, merging and writing to output now. INFO 2021-07-21 19:49:46 SortSam Wrote 10,000,000 records from a sorting collection. Elapsed time: 00:03:02s. Time for last 10,000,000: 95s. Last read position: chr18:1,909,991 INFO 2021-07-21 19:51:21 SortSam Wrote 20,000,000 records from a sorting collection. Elapsed time: 00:04:36s. Time for last 10,000,000: 94s. Last read position: chr6:10,694,018 [Wed Jul 21 19:52:23 EDT 2021] picard.sam.SortSam done. Elapsed time: 5.66 minutes. Runtime.totalMemory()=10343153664 [info] Marking duplicates... SL_04_S4.bam Wed Jul 21 19:52:24 EDT 2021 [info] Removing duplicates... SL_04_S4.bam Wed Jul 21 20:00:33 EDT 2021 [info] Filtering to <120bp... dup.marked and dedup BAMs Wed Jul 21 20:02:32 EDT 2021 [info] Creating bam index files... SL_04_S4.bam Wed Jul 21 20:03:34 EDT 2021 [info] Reads shifting Wed Jul 21 20:03:38 EDT 2021 [info] Your data won't be shifted as the experiment_type is specified as CUT&RUN... [info] Peak calling using MACS2... SL_04_S4.bam [info] Logs are stored in /dartfs-hpc/rc/lab/L/LeachLab/HIMANSHU/CutNrun/CutNRun_6-18-21/SL_04_S4/logs Wed Jul 21 20:03:38 EDT 2021 [info] Peak calling with BAM file with NO duplications [info] macs2 narrow peak calling [info] macs2 broad peak calling [info] Getting broad peak summits [info] SEACR stringent peak calling Calling enriched regions without control file Proceeding without normalization of control to experimental bedgraph Using stringent threshold Creating experimental AUC file: Wed Jul 21 20:05:01 EDT 2021 Calculating optimal AUC threshold: Wed Jul 21 20:05:01 EDT 2021 Using user-provided threshold: Wed Jul 21 20:05:01 EDT 2021 Creating thresholded feature file: Wed Jul 21 20:05:05 EDT 2021 Empirical false discovery rate = 0.01 Merging nearby features and eliminating control-enriched features: Wed Jul 21 20:05:06 EDT 2021 Removing temporary files: Wed Jul 21 20:05:06 EDT 2021 Done: Wed Jul 21 20:05:06 EDT 2021 [info] Generating the normalized signal file with BigWig format... Wed Jul 21 20:05:06 EDT 2021 [info] Your bigwig file won't be normalized with spike-in reads [info] Input file is /dartfs-hpc/rc/lab/L/LeachLab/HIMANSHU/CutNrun/CutNRun_6-18-21/SL_04_S4/peakcalling/macs2.narrow/SL_04_S4_peaks.narrowPeak [info] Get randomized [1000] peaks from the top [2000] peaks... [info] Filtering the blacklist regions for the selected peak files [info] Getting Fasta sequences Warning: the index file is older than the FASTA file. Feature (chrUn_KI270438v1:112311-112512) beyond the length of chrUn_KI270438v1 size (112505 bp). Skipping. [info] Start MEME analysis for de novo motif finding ... [info] Up to 10 will be output ... Log::Log4perl configuration looks suspicious: No loggers defined at /dartfs-hpc/rc/home/k/f0042dk/miniconda3/envs/meme/lib/site_perl/5.26.2/Log/Log4perl/Config.pm line 325. Starting getsize: getsize random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa 1> $metrics getsize ran successfully in 0.082524 seconds Starting fasta-most: fasta-most -min 50 < random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa 1> $metrics fasta-most ran successfully in 0.244248 seconds Starting fasta-center: fasta-center -dna -len 100 < random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa 1> random1000/MEME_SL_04_S4_shuf/seqs-centered fasta-center ran successfully in 0.294666 seconds Starting fasta-shuffle-letters: fasta-shuffle-letters random1000/MEME_SL_04_S4_shuf/seqs-centered random1000/MEME_SL_04_S4_shuf/seqs-shuffled -kmer 2 -tag -dinuc -dna -seed 1 fasta-shuffle-letters ran successfully in 0.043727 seconds Starting fasta-get-markov: fasta-get-markov -nostatus -nosummary -dna -m 1 random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa random1000/MEME_SL_04_S4_shuf/background fasta-get-markov ran successfully in 0.047089 seconds Starting meme: meme random1000/MEME_SL_04_S4_shuf/seqs-centered -oc random1000/MEME_SL_04_S4_shuf/meme_out -mod zoops -nmotifs 10 -minw 6 -maxw 30 -bfile random1000/MEME_SL_04_S4_shuf/background -dna -revcomp -nostatus meme ran successfully in 121.292254 seconds Starting dreme: dreme -verbosity 1 -oc random1000/MEME_SL_04_S4_shuf/dreme_out -png -dna -p random1000/MEME_SL_04_S4_shuf/seqs-centered -n random1000/MEME_SL_04_S4_shuf/seqs-shuffled -m 10 File "/dartfs-hpc/rc/home/k/f0042dk/miniconda3/envs/meme/bin/dreme", line 765 print "Finding secondary RE in left flank..." ^ SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Finding secondary RE in left flank...")? dreme exited with error code 1 Starting centrimo: centrimo -seqlen 201 -verbosity 1 -oc random1000/MEME_SL_04_S4_shuf/centrimo_out -bfile random1000/MEME_SL_04_S4_shuf/background random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml Bad file name. Bad file name. Bad file name. Bad file name. Bad file name. Bad file name. Bad file name. Bad file name. Bad file name. Bad file name. Bad file name. Bad file name. Bad file name. Bad file name. Bad file name. FATAL: Template does not contain data section. centrimo exited with error code 1 Starting tomtom: tomtom -verbosity 1 -text -thresh 0.1 random1000/MEME_SL_04_S4_shuf/combined.meme random1000/MEME_SL_04_S4_shuf/combined.meme 1> random1000/MEME_SL_04_S4_shuf/motif_alignment.txt tomtom ran successfully in 2.053908 seconds Starting spamo: spamo -verbosity 1 -oc random1000/MEME_SL_04_S4_shuf/spamo_out_1 -bgfile random1000/MEME_SL_04_S4_shuf/background -keepprimary -primary GGAATBGAATGGAATCAWCAC random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml spamo exited with error code 1 Starting spamo: spamo -verbosity 1 -oc random1000/MEME_SL_04_S4_shuf/spamo_out_2 -bgfile random1000/MEME_SL_04_S4_shuf/background -keepprimary -primary TATATATATAWATATATATATATATATATA random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml spamo exited with error code 1 Starting spamo: spamo -verbosity 1 -oc random1000/MEME_SL_04_S4_shuf/spamo_out_3 -bgfile random1000/MEME_SL_04_S4_shuf/background -keepprimary -primary RGKTWCTKWGAWWKYTTMTGTMYAGTWT random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml spamo exited with error code 1 Starting spamo: spamo -verbosity 1 -oc random1000/MEME_SL_04_S4_shuf/spamo_out_4 -bgfile random1000/MEME_SL_04_S4_shuf/background -keepprimary -primary TGTCTAGGCTCTGCCTACAGGGGMHTTK random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml spamo exited with error code 1 Starting spamo: spamo -verbosity 1 -oc random1000/MEME_SL_04_S4_shuf/spamo_out_5 -bgfile random1000/MEME_SL_04_S4_shuf/background -keepprimary -primary TATCAAAGTACCCAAAGTA random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml spamo exited with error code 1 Starting fimo: fimo --parse-genomic-coord --verbosity 1 --oc random1000/MEME_SL_04_S4_shuf/fimo_out_1 --bgfile random1000/MEME_SL_04_S4_shuf/background --motif GGAATBGAATGGAATCAWCAC random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa fimo ran successfully in 0.367951 seconds Starting fimo: fimo --parse-genomic-coord --verbosity 1 --oc random1000/MEME_SL_04_S4_shuf/fimo_out_2 --bgfile random1000/MEME_SL_04_S4_shuf/background --motif TATATATATAWATATATATATATATATATA random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa fimo ran successfully in 0.299063 seconds Starting fimo: fimo --parse-genomic-coord --verbosity 1 --oc random1000/MEME_SL_04_S4_shuf/fimo_out_3 --bgfile random1000/MEME_SL_04_S4_shuf/background --motif RGKTWCTKWGAWWKYTTMTGTMYAGTWT random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa fimo ran successfully in 0.23248 seconds Starting fimo: fimo --parse-genomic-coord --verbosity 1 --oc random1000/MEME_SL_04_S4_shuf/fimo_out_4 --bgfile random1000/MEME_SL_04_S4_shuf/background --motif TGTCTAGGCTCTGCCTACAGGGGMHTTK random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa fimo ran successfully in 0.26177 seconds Starting fimo: fimo --parse-genomic-coord --verbosity 1 --oc random1000/MEME_SL_04_S4_shuf/fimo_out_5 --bgfile random1000/MEME_SL_04_S4_shuf/background --motif TATCAAAGTACCCAAAGTA random1000/MEME_SL_04_S4_shuf/meme_out/meme.xml random1000/MEME_SL_04_S4_shuf/SL_04_S4_summits_padded.fa fimo ran successfully in 0.234876 seconds Starting meme-chip_html_to_tsv: meme-chip_html_to_tsv random1000/MEME_SL_04_S4_shuf/meme-chip.html random1000/MEME_SL_04_S4_shuf/summary.tsv "meme-chip -oc random1000/MEME_SL_04_S4_shuf -dreme-m 10 -meme-nmotifs 10 random1000/padded.fa/SL_04_S4_summits_padded.fa" 5.0.5 "Mon Mar 18 20\:12\:19 2019 -0700" meme-chip_html_to_tsv ran successfully in 1.103435 seconds [info] De Novo motifs can be found: random1000/MEME_SL_04_S4_shuf ... [info] Loading the De Novo motifs ... Traceback (most recent call last): File "/dartfs-hpc/rc/lab/L/LeachLab/HIMANSHU/CutNrun/CUT-RUNTools-2.0/install/read.meme.py", line 94, in dreme_matrices = read_dreme(this_dir + "/dreme_out/dreme.txt") File "/dartfs-hpc/rc/lab/L/LeachLab/HIMANSHU/CutNrun/CUT-RUNTools-2.0/install/read.meme.py", line 47, in read_dreme f = open(n) FileNotFoundError: [Errno 2] No such file or directory: 'random1000/MEME_SL_04_S4_shuf/dreme_out/dreme.txt' [info] The signficance cutoff of Fimo scaning is 0.0005... [info] Motif files can be found: random1000/MEME_SL_04_S4_shuf/motifs [info] Filtering the blacklist regions for the selected peak files [info] Getting Fasta sequences Warning: the index file is older than the FASTA file. [info] Scaning the De Novo motifs for each peak ls: cannot access 'random1000/MEME_SL_04_S4_shuf/motifs': No such file or directory [info] Output can be found: fimo.result/SL_04_S4 # # Congrats! The bulk data analysis is complete! # --------------------------------------------------------------------------