____ ____ __ / ___)( __) / \ \___ \ ) _) ( O ) (____/(____) \__\) ____ (___ \ / __/ (____) ____ ___ __ ____ __ _ ___ ____ / ___) / __)( )( __)( ( \ / __)( __) \___ \( (__ )( ) _) / /( (__ ) _) (____/ \___)(__)(____)\_)__) \___)(____) version: 0.5.1 docs: https://vanheeringen-lab.github.io/seq2science Building DAG of jobs... Using shell: /usr/bin/bash Provided cores: 24 Rules claiming more threads will be scaled down. Provided resources: parallel_downloads=3, deeptools_limit=16, R_scripts=1, mem_gb=126 Job counts: count jobs 1 bedgraph_bigwig 1 bedtools_slop 1 bwa_mem 1 chipseeker 1 combine_peaks 1 combine_qc_files 1 computeMatrix 1 coverage_table 3 edgeR_normalization 1 fastp_SE 1 featureCounts 4 log_normalization 1 macs2_callpeak 1 mark_duplicates 4 mean_center 1 mt_nuc_ratio_calculator 1 multiqc 1 onehot_peaks 1 peak_bigpeak 1 plotFingerprint 1 plotProfile 1 quantile_normalization 1 samtools_index 1 samtools_presort 2 samtools_stats 1 seq2science 1 sieve_bam 1 trackhub 37 [Wed Apr 21 10:00:37 2021] rule fastp_SE: input: /exports/humgen/jihed/seq2science/results/fastq/GSM1555120.fastq.gz output: /exports/humgen/jihed/seq2science/results/fastq_trimmed/GSM1555120_trimmed.fastq.gz, /exports/humgen/jihed/seq2science/results/qc/trimming/GSM1555120.fastp.json, /exports/humgen/jihed/seq2science/results/qc/trimming/GSM1555120.fastp.html log: /exports/humgen/jihed/seq2science/results/log/fastp_SE/GSM1555120.log jobid: 35 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/fastp_SE/GSM1555120.benchmark.txt wildcards: sample=GSM1555120 threads: 4 [Wed Apr 21 10:00:37 2021] Finished job 35. 1 of 37 steps (3%) done [Wed Apr 21 10:00:37 2021] group job 8a4b96c2-aac6-4f7b-8571-8a20547e3d02 (jobs in lexicogr. order): [Wed Apr 21 10:00:37 2021] rule bwa_mem: input: /exports/humgen/jihed/seq2science/results/fastq_trimmed/GSM1555120_trimmed.fastq.gz, /exports/humgen/jihed/seq2science/genomes/mm10/index/bwa-mem/ output: /exports/humgen/jihed/seq2science/results/bwa-mem/mm10-GSM1555120.samtools-coordinate.pipe (pipe) log: /exports/humgen/jihed/seq2science/results/log/bwa-mem_align/mm10-GSM1555120.log jobid: 47 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/bwa-mem_align/mm10-GSM1555120.benchmark.txt wildcards: assembly=mm10, sample=GSM1555120 threads: 10 resources: mem_gb=13 [Wed Apr 21 10:00:37 2021] rule samtools_presort: input: /exports/humgen/jihed/seq2science/results/bwa-mem/mm10-GSM1555120.samtools-coordinate.pipe output: /exports/humgen/jihed/seq2science/results/bwa-mem/mm10-GSM1555120.samtools-coordinate-unsieved.bam log: /exports/humgen/jihed/seq2science/results/log/samtools_presort/mm10-GSM1555120.log jobid: 43 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/samtools_presort/mm10-GSM1555120.benchmark.txt wildcards: assembly=mm10, sample=GSM1555120 threads: 2 resources: mem_gb=2 Removing temporary output file /exports/humgen/jihed/seq2science/results/fastq_trimmed/GSM1555120_trimmed.fastq.gz. [Wed Apr 21 10:00:37 2021] Finished job 47. [Wed Apr 21 10:00:37 2021] Finished job 43. 3 of 37 steps (8%) done [Wed Apr 21 10:00:37 2021] rule sieve_bam: input: /exports/humgen/jihed/seq2science/results/bwa-mem/mm10-GSM1555120.samtools-coordinate-unsieved.bam, /exports/humgen/jihed/seq2science/genomes/mm10/mm10.customblacklist_complement.bed, /exports/humgen/jihed/seq2science/genomes/mm10/mm10.fa.sizes output: /exports/humgen/jihed/seq2science/results/bwa-mem/mm10-GSM1555120.samtools-coordinate-sieved.bam log: /exports/humgen/jihed/seq2science/results/log/sieve_bam/mm10-GSM1555120.log jobid: 36 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/sieve_bam/mm10-GSM1555120.benchmark.txt wildcards: assembly=mm10, sample=GSM1555120 threads: 2 [Wed Apr 21 10:00:37 2021] Finished job 36. 4 of 37 steps (11%) done [Wed Apr 21 10:00:37 2021] rule mt_nuc_ratio_calculator: input: /exports/humgen/jihed/seq2science/results/bwa-mem/mm10-GSM1555120.samtools-coordinate-unsieved.bam, /exports/humgen/jihed/seq2science/genomes/mm10/mm10.fa.sizes output: /exports/humgen/jihed/seq2science/results/bwa-mem/mm10-GSM1555120.samtools-coordinate-unsieved.bam.mtnucratiomtnuc.json log: /exports/humgen/jihed/seq2science/results/log/MTNucRatioCalculator/mm10-GSM1555120.log jobid: 33 wildcards: assembly=mm10, sample=GSM1555120 resources: time=0-06:00:00 [Wed Apr 21 10:00:37 2021] Finished job 33. 5 of 37 steps (14%) done [Wed Apr 21 10:00:37 2021] rule samtools_stats: input: /exports/humgen/jihed/seq2science/results/bwa-mem/mm10-GSM1555120.samtools-coordinate-unsieved.bam output: /exports/humgen/jihed/seq2science/results/qc/samtools_stats/bwa-mem/mm10-GSM1555120.samtools-coordinate.samtools_stats.txt log: /exports/humgen/jihed/seq2science/results/log/samtools_stats/bwa-mem/mm10-GSM1555120-samtools-coordinate.log jobid: 29 wildcards: directory=bwa-mem, assembly=mm10, sample=GSM1555120, sorter=samtools, sorting=coordinate resources: time=0-06:00:00 Removing temporary output file /exports/humgen/jihed/seq2science/results/bwa-mem/mm10-GSM1555120.samtools-coordinate-unsieved.bam. [Wed Apr 21 10:00:38 2021] Finished job 29. 6 of 37 steps (16%) done [Wed Apr 21 10:00:38 2021] rule mark_duplicates: input: /exports/humgen/jihed/seq2science/results/bwa-mem/mm10-GSM1555120.samtools-coordinate-sieved.bam output: /exports/humgen/jihed/seq2science/results/final_bam/mm10-GSM1555120.samtools-coordinate.bam, /exports/humgen/jihed/seq2science/results/qc/markdup/mm10-GSM1555120.samtools-coordinate.metrics.txt log: /exports/humgen/jihed/seq2science/results/log/mark_duplicates/mm10-GSM1555120-samtools-coordinate.log jobid: 17 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/mark_duplicates/mm10-GSM1555120-samtools-coordinate.benchmark.txt wildcards: assembly=mm10, sample=GSM1555120, sorter=samtools, sorting=coordinate resources: mem_gb=8 Removing temporary output file /exports/humgen/jihed/seq2science/results/bwa-mem/mm10-GSM1555120.samtools-coordinate-sieved.bam. [Wed Apr 21 10:00:38 2021] Finished job 17. 7 of 37 steps (19%) done [Wed Apr 21 10:00:38 2021] rule samtools_index: input: /exports/humgen/jihed/seq2science/results/final_bam/mm10-GSM1555120.samtools-coordinate.bam output: /exports/humgen/jihed/seq2science/results/final_bam/mm10-GSM1555120.samtools-coordinate.bam.bai jobid: 18 wildcards: filepath=/exports/humgen/jihed/seq2science/results/final_bam/mm10-GSM1555120.samtools-coordinate [Wed Apr 21 10:00:38 2021] Finished job 18. 8 of 37 steps (22%) done [Wed Apr 21 10:00:38 2021] rule macs2_callpeak: input: /exports/humgen/jihed/seq2science/results/final_bam/mm10-GSM1555120.samtools-coordinate.bam, /exports/humgen/jihed/seq2science/results/qc/trimming/GSM1555120.fastp.json output: /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120_control_lambda.bdg, /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120_peaks.xls, /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120_treat_pileup.bdg, /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120_summits.bed, /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120_peaks.narrowPeak log: /exports/humgen/jihed/seq2science/results/log/macs2_callpeak/mm10-GSM1555120.log jobid: 23 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/macs2_callpeak/mm10-GSM1555120.benchmark.txt wildcards: assembly=mm10, sample=GSM1555120 resources: mem_gb=4 [Wed Apr 21 10:00:38 2021] Finished job 23. 9 of 37 steps (24%) done [Wed Apr 21 10:00:38 2021] rule samtools_stats: input: /exports/humgen/jihed/seq2science/results/final_bam/mm10-GSM1555120.samtools-coordinate.bam output: /exports/humgen/jihed/seq2science/results/qc/samtools_stats/final_bam/mm10-GSM1555120.samtools-coordinate.samtools_stats.txt log: /exports/humgen/jihed/seq2science/results/log/samtools_stats/final_bam/mm10-GSM1555120-samtools-coordinate.log jobid: 34 wildcards: directory=final_bam, assembly=mm10, sample=GSM1555120, sorter=samtools, sorting=coordinate resources: time=0-06:00:00 [Wed Apr 21 10:00:38 2021] Finished job 34. 10 of 37 steps (27%) done [Wed Apr 21 10:00:38 2021] rule bedgraph_bigwig: input: /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120_treat_pileup.bdg, /exports/humgen/jihed/seq2science/genomes/mm10/mm10.fa.sizes output: /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120.bw, /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120.bedgraphtmp log: /exports/humgen/jihed/seq2science/results/log/bedgraph_bigwig/macs2/mm10-GSM1555120.log jobid: 10 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/bedgraphish_to_bedgraph/mm10-GSM1555120-macs2.log wildcards: peak_caller=macs2, assembly=mm10, sample=GSM1555120 Removing temporary output file /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120.bedgraphtmp. [Wed Apr 21 10:00:38 2021] Finished job 10. 11 of 37 steps (30%) done [Wed Apr 21 10:00:38 2021] rule featureCounts: input: /exports/humgen/jihed/seq2science/results/final_bam/mm10-GSM1555120.samtools-coordinate.bam, /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120_peaks.narrowPeak output: /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120.saf, /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120_featureCounts.txt, /exports/humgen/jihed/seq2science/results/qc/macs2/mm10-GSM1555120_featureCounts.txt.summary log: /exports/humgen/jihed/seq2science/results/log/featureCounts/mm10-GSM1555120-macs2.log jobid: 30 wildcards: peak_caller=macs2, assembly=mm10, sample=GSM1555120 threads: 4 Removing temporary output file /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120.saf. [Wed Apr 21 10:00:38 2021] Finished job 30. 12 of 37 steps (32%) done [Wed Apr 21 10:00:38 2021] rule peak_bigpeak: input: /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120_peaks.narrowPeak, /exports/humgen/jihed/seq2science/genomes/mm10/mm10.fa.sizes output: /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120.bignarrowPeak, /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120.tmp.narrowPeak log: /exports/humgen/jihed/seq2science/results/log/narrowpeak_bignarrowpeak/macs2/mm10-GSM1555120-narrowPeak.log jobid: 9 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/bedgraphish_to_bedgraph/mm10-GSM1555120-macs2-narrowPeak.log wildcards: peak_caller=macs2, assembly=mm10, sample=GSM1555120, peak=narrowPeak Removing temporary output file /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120.tmp.narrowPeak. [Wed Apr 21 10:00:39 2021] Finished job 9. 13 of 37 steps (35%) done [Wed Apr 21 10:00:39 2021] rule chipseeker: input: /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120_summits.bed, /exports/humgen/jihed/seq2science/genomes/mm10/mm10.annotation.gtf output: /exports/humgen/jihed/seq2science/results/qc/chipseeker/mm10-macs2_img1_mqc.png, /exports/humgen/jihed/seq2science/results/qc/chipseeker/mm10-macs2_img2_mqc.png log: /exports/humgen/jihed/seq2science/results/log/chipseeker/mm10-macs2.log jobid: 28 wildcards: assembly=mm10, peak_caller=macs2 resources: R_scripts=1 [Wed Apr 21 10:00:39 2021] Finished job 28. 14 of 37 steps (38%) done [Wed Apr 21 10:00:39 2021] rule plotFingerprint: input: /exports/humgen/jihed/seq2science/results/final_bam/mm10-GSM1555120.samtools-coordinate.bam, /exports/humgen/jihed/seq2science/results/final_bam/mm10-GSM1555120.samtools-coordinate.bam.bai output: /exports/humgen/jihed/seq2science/results/qc/plotFingerprint/mm10.tsv log: /exports/humgen/jihed/seq2science/results/log/plotFingerprint/mm10.log jobid: 31 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/plotFingerprint/mm10.benchmark.txt wildcards: assembly=mm10 threads: 16 resources: mem_gb=5 [Wed Apr 21 10:00:39 2021] Finished job 31. 15 of 37 steps (41%) done [Wed Apr 21 10:00:39 2021] rule combine_peaks: input: /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120_summits.bed, /exports/humgen/jihed/seq2science/genomes/mm10/mm10.fa.sizes output: /exports/humgen/jihed/seq2science/results/macs2/mm10_combinedsummits.bed log: /exports/humgen/jihed/seq2science/results/log/combine_peaks/mm10-macs2.log jobid: 24 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/combine_peaks/mm10-macs2.benchmark.txt wildcards: peak_caller=macs2, assembly=mm10 [Wed Apr 21 10:00:39 2021] Finished job 24. 16 of 37 steps (43%) done [Wed Apr 21 10:00:39 2021] rule onehot_peaks: input: /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120_peaks.narrowPeak, /exports/humgen/jihed/seq2science/results/macs2/mm10_combinedsummits.bed output: /exports/humgen/jihed/seq2science/results/counts/macs2/mm10_onehotpeaks.tsv, /exports/humgen/jihed/seq2science/results/counts/macs2/mm10_onehotpeaks.tsv.tmp log: /exports/humgen/jihed/seq2science/results/log/onehot_peaks/mm10-macs2.log jobid: 8 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/onehot_peaks/mm10-macs2.benchmark.txt wildcards: peak_caller=macs2, assembly=mm10 Removing temporary output file /exports/humgen/jihed/seq2science/results/counts/macs2/mm10_onehotpeaks.tsv.tmp. [Wed Apr 21 10:00:39 2021] Finished job 8. 17 of 37 steps (46%) done [Wed Apr 21 10:00:39 2021] rule bedtools_slop: input: /exports/humgen/jihed/seq2science/results/macs2/mm10_combinedsummits.bed, /exports/humgen/jihed/seq2science/genomes/mm10/mm10.fa.sizes output: /exports/humgen/jihed/seq2science/results/macs2/mm10_combinedpeaks.bed log: /exports/humgen/jihed/seq2science/results/log/bedtools_slop/mm10-macs2.log jobid: 16 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/bedtools_slop/mm10-macs2.benchmark.txt wildcards: peak_caller=macs2, assembly=mm10 Removing temporary output file /exports/humgen/jihed/seq2science/results/macs2/mm10_combinedsummits.bed. [Wed Apr 21 10:00:39 2021] Finished job 16. 18 of 37 steps (49%) done [Wed Apr 21 10:00:39 2021] rule trackhub: input: /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120.bignarrowPeak, /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120.bw output: /exports/humgen/jihed/seq2science/results/trackhub log: /exports/humgen/jihed/seq2science/results/log/trackhub/trackhub.log jobid: 1 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/trackhub/trackhub.benchmark.txt ImproperOutputException in line 658 of /exports/humgen/jihed/miniconda3/envs/seq2science/lib/python3.8/site-packages/seq2science/rules/trackhub.smk: Outputs of incorrect type (directories when expecting files or vice versa). Output directories must be flagged with directory(). for rule trackhub: /exports/humgen/jihed/seq2science/results/trackhub File "/exports/humgen/jihed/miniconda3/envs/seq2science/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 339, in handle_job_success File "/exports/humgen/jihed/miniconda3/envs/seq2science/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 225, in handle_job_success [Wed Apr 21 10:00:39 2021] rule computeMatrix: input: /exports/humgen/jihed/seq2science/results/macs2/mm10-GSM1555120.bw output: /exports/humgen/jihed/seq2science/results/qc/computeMatrix/mm10-macs2.mat.gz log: /exports/humgen/jihed/seq2science/results/log/computeMatrix/mm10-macs2.log jobid: 42 benchmark: /exports/humgen/jihed/seq2science/results/benchmark/computeMatrix/mm10-macs2.benchmark.txt wildcards: assembly=mm10, peak_caller=macs2 threads: 16 resources: deeptools_limit=16, mem_gb=50 [Wed Apr 21 10:00:40 2021] Finished job 42. 19 of 37 steps (51%) done Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message __ __ __ ____ ____ _ / \ / \ / \( _ \/ ___)/ \ ( O )( O )( O )) __/\___ \\_/ \__/ \__/ \__/(__) (____/(_) One or more rules did not finish as expected! Please take a look at the log files of the failed rule(s), and our Frequently Asked Questions: https://vanheeringen-lab.github.io/seq2science/content/faq.html If that does not help you, don't be afraid to reach out to us. The easiest way would be to make an issue on our github page: https://github.com/vanheeringen-lab/seq2science/issues Complete log: /exports/humgen/jihed/seq2science/seq2science.2021-04-21T100027.917233.log