-
Notifications
You must be signed in to change notification settings - Fork 412
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
First Release #19
First Release #19
Conversation
- update README - update `.travis.yml` - add `Jenkinsfile` - update conda `environment.yml` file
* update README * update .travis.yml * add Jenkinsfile * update conda environment file * update software_version collect * remove lib/SarekUtils.groovy * add References to .gitignore * improve software versions gathering * fix docker owner * update tests * sort params * add preprocessing * improve multiple TSV * add parrallelized BaseRecalibrator
* Update allelecount to 4.0.2 * Update GATK to 4.1.2.0 * Update docs * Add HaplotypeCaller * Add single Strelka and single Manta
Add somatic Variant Callers
* add containers for annotation
* Add Annotation * Annotation containers are tagged version.genomeversion * Better indentation * Replace `--sampleDir` by `--sample` * add MultiQC * update Tests * Add parallelized testing on Travis * Add parallelized building of Docker Images for annotation on CircleCI * Update docs
* improve tests + add some docs
* update tests * code polishing * add some docs about QC tools
* better docs * add docs about annotation * move helper scripts to scripts * add helper script to filter FreeBayes output * add docs about Variant Calling and Preprocessing * add scripts for ASCAT * fix publishDir for MergeMpileup * fix intra link * completed docs about ASCAT * add building container for dogs * add Abstracts & Posters * improve tests * update docs
* Update CI scripts * Add possibility to download singularity images with helper script * Update Docs * Improve build.nf * Use label for processes configuration * Update configuration files * Disable Docker in singularity profile * Disable Singularity in docker profile * Disable Docker and Singularity in conda profile * Simplify check_max() function
* Add BamQC, CompressVCFsnpEff and CompressVCFvep processes * Add Citation documentation * Fix merge in annotation * Merge BamQCmapped and BamQCrecalibrated processes into BamQC process * Removed BamQCmapped, BamQCrecalibrated and CompressVCF processes * Split CompressVCF process into CompressVCFsnpEff and CompressVCFvep processes
fix output name for vcf
* chmod +x * add correct path to Rscript * update CHANGELOG
🚨 Build log output:Branch indexing Connecting to https://api.github.com using nf-core-bot/****** (nf-core) Obtained Jenkinsfile from 10eff4e+7a93b668665336cfd7cd80fbf17a93314e9fa090 (0f07bf2) Running in Durability level: MAX_SURVIVABILITY [Pipeline] Start of Pipeline [Pipeline] node Running on ship-1 in /share/jenkins_home/workspace/nf-core_sarek_PR-19 [Pipeline] { [Pipeline] stage [Pipeline] { (Declarative: Checkout SCM) [Pipeline] checkout using credential 3d82ac34-e01a-4592-a628-ce6732c445a3 Cloning the remote Git repository Cloning with configured refspecs honoured and without tags Cloning repository https://github.com/nf-core/sarek.git > git init /share/jenkins_home/workspace/nf-core_sarek_PR-19 # timeout=10 Fetching upstream changes from https://github.com/nf-core/sarek.git > git --version # timeout=10 using GIT_ASKPASS to set credentials nf-core > git fetch --no-tags --progress https://github.com/nf-core/sarek.git +refs/pull/19/head:refs/remotes/origin/PR-19 +refs/heads/master:refs/remotes/origin/master Fetching without tags Merging remotes/origin/master commit 7a93b66 into PR head commit 10eff4e > git config remote.origin.url https://github.com/nf-core/sarek.git # timeout=10 > git config --add remote.origin.fetch +refs/pull/19/head:refs/remotes/origin/PR-19 # timeout=10 > git config --add remote.origin.fetch +refs/heads/master:refs/remotes/origin/master # timeout=10 > git config remote.origin.url https://github.com/nf-core/sarek.git # timeout=10 Fetching upstream changes from https://github.com/nf-core/sarek.git using GIT_ASKPASS to set credentials nf-core > git fetch --no-tags --progress https://github.com/nf-core/sarek.git +refs/pull/19/head:refs/remotes/origin/PR-19 +refs/heads/master:refs/remotes/origin/master > git config core.sparsecheckout # timeout=10 > git checkout -f 10eff4e Merge succeeded, producing 10eff4e Checking out Revision 10eff4e (PR-19) Commit message: "Fix R scripts (#16)" First time build. Skipping changelog. [Pipeline] } [Pipeline] // stage > git merge 7a93b66 # timeout=10 > git rev-parse HEAD^{commit} # timeout=10 > git config core.sparsecheckout # timeout=10 > git checkout -f 10eff4e [Pipeline] withEnv [Pipeline] { [Pipeline] withCredentials Masking only exact matches of $JENKINS_API or $JENKINS_API_USR or $JENKINS_API_PSW [Pipeline] { [Pipeline] stage [Pipeline] { (Docker setup) [Pipeline] sh + ./scripts/download_image.sh -n docker -t ALL + ENGINE=docker + NXF_SINGULARITY_CACHEDIR=work/singularity/. + TEST=ALL + [[ 4 -gt 0 ]] + key=-n + case $key in + ENGINE=docker + shift + shift + [[ 2 -gt 0 ]] + key=-t + case $key in + TEST=ALL + shift + shift + [[ 0 -gt 0 ]] + [[ ALL,ANNOTATEALL,ANNOTATESNPEFF =~ ALL ]] + [[ docker =~ docker ]] + docker pull nfcore/sareksnpeff:dev.GRCh37 dev.GRCh37: Pulling from nfcore/sareksnpeff Digest: sha256:734b4ba5645f8b88a18ea37c33cc4cedef97a53dc389810bfb3d5c4281650f92 Status: Image is up to date for nfcore/sareksnpeff:dev.GRCh37 + docker tag nfcore/sareksnpeff:dev.GRCh37 nfcore/sareksnpeff:dev.smallGRCh37 + [[ ALL,ANNOTATEALL,ANNOTATEVEP =~ ALL ]] + [[ docker =~ docker ]] + docker pull nfcore/sarekvep:dev.GRCh37 dev.GRCh37: Pulling from nfcore/sarekvep Digest: sha256:0f051af98e8360e8f01f5aad56d1d6dce96c3ad4dc59b3627873acaa0bb729f9 Status: Image is up to date for nfcore/sarekvep:dev.GRCh37 + docker tag nfcore/sarekvep:dev.GRCh37 nfcore/sarekvep:dev.smallGRCh37 + [[ ANNOTATEALL,ANNOTATEVEP,ANNOTATESNPEFF != ALL ]] + [[ docker =~ docker ]] + docker pull nfcore/sarek:dev dev: Pulling from nfcore/sarek Digest: sha256:d2ac7fc24278fb0ce21952a7ed7318dde0dda1988423ac8d72b86dff5a338810 Status: Image is up to date for nfcore/sarek:dev + docker tag nfcore/sarek:dev nfcore/sarek:dev [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Build references) [Pipeline] sh + rm -rf references/ [Pipeline] sh + ./scripts/build_reference.sh + NXF_SINGULARITY_CACHEDIR=work/singularity/. + OFFLINE= + PROFILE=docker + TEST=ALL + TRAVIS_BUILD_DIR=. + TRAVIS=false + VERBOSE= + [[ 0 -gt 0 ]] + [[ ALL != ANNOTATESNPEFF ]] + [[ ALL != ANNOTATEVEP ]] + rm -rf references + nextflow run ./build.nf -profile test,docker --build --outdir references N E X T F L O W ~ version 19.04.0 Launching `./build.nf` [spontaneous_borg] - revision: 712bf0811e WARN: There's no process matching config selector: BamQC Run Name : spontaneous_borg Max Resources : 7 GB memory, 2 cpus, 2d time per job Container : docker - nfcore/sarek:dev Output dir : references Launch dir : /share/****_home/workspace/nf-core_sarek_PR-19 Working dir : /share/****_home/workspace/nf-core_sarek_PR-19/work Script dir : /share/****_home/workspace/nf-core_sarek_PR-19 User : **** Config Profile : test,docker Config Description: Minimal test dataset to check pipeline function executor > local (1) [74/b0b54b] process > DecompressFile [ 0%] 0 of 1 executor > local (2) [b1/4f8c63] process > DecompressFile [ 0%] 0 of 2 executor > local (2) [74/b0b54b] process > DecompressFile [ 50%] 1 of 2 executor > local (3) [02/bcc797] process > DecompressFile [ 33%] 1 of 3 executor > local (3) [b1/4f8c63] process > DecompressFile [ 67%] 2 of 3 executor > local (5) [fb/2e7000] process > DecompressFile [ 50%] 2 of 4 [f0/8b2f37] process > BuildSAMToolsIndex [ 0%] 0 of 1 executor > local (8) [fb/2e7000] process > DecompressFile [ 50%] 2 of 4 [f0/8b2f37] process > BuildSAMToolsIndex [ 0%] 0 of 1 [e7/1b54c8] process > BuildBWAindexes [ 0%] 0 of 1 [b7/b92b2b] process > BuildReferenceIndex [ 0%] 0 of 1 [ee/692355] process > BuildVCFIndex [ 0%] 0 of 1 executor > local (9) [02/bcc797] process > DecompressFile [ 75%] 3 of 4 [f0/8b2f37] process > BuildSAMToolsIndex [ 0%] 0 of 1 [e7/1b54c8] process > BuildBWAindexes [ 0%] 0 of 1 [b7/b92b2b] process > BuildReferenceIndex [ 0%] 0 of 1 [d8/e66ced] process > BuildVCFIndex [ 0%] 0 of 2 executor > local (10) [fb/2e7000] process > DecompressFile [100%] 4 of 4 ✔ [f0/8b2f37] process > BuildSAMToolsIndex [ 0%] 0 of 1 [e7/1b54c8] process > BuildBWAindexes [ 0%] 0 of 1 [b7/b92b2b] process > BuildReferenceIndex [ 0%] 0 of 1 [6f/0a2b76] process > BuildVCFIndex [ 0%] 0 of 3 executor > local (10) [fb/2e7000] process > DecompressFile [100%] 4 of 4 ✔ [f0/8b2f37] process > BuildSAMToolsIndex [100%] 1 of 1 ✔ [e7/1b54c8] process > BuildBWAindexes [ 0%] 0 of 1 [b7/b92b2b] process > BuildReferenceIndex [ 0%] 0 of 1 [6f/0a2b76] process > BuildVCFIndex [ 0%] 0 of 3 executor > local (10) [fb/2e7000] process > DecompressFile [100%] 4 of 4 ✔ [f0/8b2f37] process > BuildSAMToolsIndex [100%] 1 of 1 ✔ [e7/1b54c8] process > BuildBWAindexes [ 0%] 0 of 1 [b7/b92b2b] process > BuildReferenceIndex [ 0%] 0 of 1 [ee/692355] process > BuildVCFIndex [ 33%] 1 of 3 executor > local (10) [fb/2e7000] process > DecompressFile [100%] 4 of 4 ✔ [f0/8b2f37] process > BuildSAMToolsIndex [100%] 1 of 1 ✔ [e7/1b54c8] process > BuildBWAindexes [100%] 1 of 1 ✔ [b7/b92b2b] process > BuildReferenceIndex [ 0%] 0 of 1 [ee/692355] process > BuildVCFIndex [ 33%] 1 of 3 executor > local (10) [fb/2e7000] process > DecompressFile [100%] 4 of 4 ✔ [f0/8b2f37] process > BuildSAMToolsIndex [100%] 1 of 1 ✔ [e7/1b54c8] process > BuildBWAindexes [100%] 1 of 1 ✔ [b7/b92b2b] process > BuildReferenceIndex [ 0%] 0 of 1 [d8/e66ced] process > BuildVCFIndex [ 67%] 2 of 3 executor > local (10) [fb/2e7000] process > DecompressFile [100%] 4 of 4 ✔ [f0/8b2f37] process > BuildSAMToolsIndex [100%] 1 of 1 ✔ [e7/1b54c8] process > BuildBWAindexes [100%] 1 of 1 ✔ [b7/b92b2b] process > BuildReferenceIndex [ 0%] 0 of 1 [6f/0a2b76] process > BuildVCFIndex [100%] 3 of 3 ✔ executor > local (10) [fb/2e7000] process > DecompressFile [100%] 4 of 4 ✔ [f0/8b2f37] process > BuildSAMToolsIndex [100%] 1 of 1 ✔ [e7/1b54c8] process > BuildBWAindexes [100%] 1 of 1 ✔ [b7/b92b2b] process > BuildReferenceIndex [100%] 1 of 1 ✔ [6f/0a2b76] process > BuildVCFIndex [100%] 3 of 3 ✔ executor > local (10) [fb/2e7000] process > DecompressFile [100%] 4 of 4 ✔ [f0/8b2f37] process > BuildSAMToolsIndex [100%] 1 of 1 ✔ [e7/1b54c8] process > BuildBWAindexes [100%] 1 of 1 ✔ [b7/b92b2b] process > BuildReferenceIndex [100%] 1 of 1 ✔ [6f/0a2b76] process > BuildVCFIndex [100%] 3 of 3 ✔ WARN: To render the execution DAG in the required format it is required to install Graphviz -- See http://www.graphviz.org for more info. Completed at: 04-Jul-2019 13:00:44 Duration : 14s CPU hours : (a few seconds) Succeeded : 10 + rm -rf .nextflow .nextflow.log references/pipeline_info work [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Germline) [Pipeline] sh + rm -rf data/ [Pipeline] sh + git clone --single-branch --branch sarek https://github.com/nf-core/test-datasets.git data Cloning into 'data'... [Pipeline] sh + ./scripts/run_tests.sh --test GERMLINE + CPUS=2 + LOGS= + NXF_SINGULARITY_CACHEDIR=work/singularity/. + OFFLINE=false + PROFILE=docker + TEST=ALL + TRAVIS_BUILD_DIR=. + TRAVIS=false + VERBOSE= + [[ 2 -gt 0 ]] + key=--test + case $key in + TEST=GERMLINE + shift + shift + [[ 0 -gt 0 ]] + [[ ALL,GERMLINE =~ GERMLINE ]] + [[ false == false ]] + rm -rf data + git clone --single-branch --branch sarek https://github.com/nf-core/test-datasets.git data Cloning into 'data'... + run_sarek --tools=false --sample data/testdata/tiny/normal --noReports + nextflow run ./main.nf -profile test,docker --monochrome_logs --tools=false --sample data/testdata/tiny/normal --noReports N E X T F L O W ~ version 19.04.0 Launching `./main.nf` [soggy_heyrovsky] - revision: fb7cd02724 No TSV file Reading data/testdata/tiny/normal directory ---------------------------------------------------- ,--./,-. ___ __ __ __ ___ /,-._.--~' |\ | |__ __ / ` / \ |__) |__ } { | \| | \__, \__/ | \ |___ \`-._,-`-, `._,._,' ____ _____ _ .' _ `. / ____| | | / |\`-_ \ | (___ ___ _ __ __ | | __ | | \ `-| \___ \/__ \| ´__/ _\| |/ / \ | \ / ____) | __ | | | __| < `|____\' |_____/\____|_| \__/|_|\_\ nf-core/sarek v2.5dev ---------------------------------------------------- Run Name : soggy_heyrovsky Max Resources : 7 GB memory, 2 cpus, 2d time per job Container : docker - nfcore/sarek:dev Sample : data/testdata/tiny/normal Step : mapping No Reports : true Nucleotides/s : 1000.0 Output dir : ./results Launch dir : /share/****_home/workspace/nf-core_sarek_PR-19 Working dir : /share/****_home/workspace/nf-core_sarek_PR-19/work Script dir : /share/****_home/workspace/nf-core_sarek_PR-19 User : **** Config Profile : test,docker Config Description: Minimal test dataset to check pipeline function ---------------------------------------------------- executor > local (2) [6d/c6e37d] process > CreateIntervalBeds [ 0%] 0 of 1 [cb/4b4feb] process > MapReads [ 0%] 0 of 1 executor > local (6) [6d/c6e37d] process > CreateIntervalBeds [ 0%] 0 of 1 [b0/fe04e4] process > MapReads [ 0%] 0 of 5 executor > local (6) [6d/c6e37d] process > CreateIntervalBeds [ 0%] 0 of 1 [cb/4b4feb] process > MapReads [ 20%] 1 of 5 executor > local (6) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [64/0a5a48] process > MapReads [ 40%] 2 of 5 executor > local (6) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [10/1e8516] process > MapReads [ 60%] 3 of 5 executor > local (6) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [66/c1979b] process > MapReads [ 80%] 4 of 5 executor > local (7) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [ 0%] 0 of 1 executor > local (8) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [ 0%] 0 of 1 executor > local (14) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [23/8b006d] process > BaseRecalibrator [ 0%] 0 of 6 executor > local (14) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [c1/c749c3] process > BaseRecalibrator [ 17%] 1 of 6 executor > local (14) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [23/8b006d] process > BaseRecalibrator [ 33%] 2 of 6 executor > local (14) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [ca/bd3099] process > BaseRecalibrator [ 50%] 3 of 6 executor > local (14) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [af/efe42f] process > BaseRecalibrator [ 83%] 5 of 6 executor > local (15) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [f8/a0bc4b] process > BaseRecalibrator [100%] 6 of 6 ✔ [f1/135459] process > GatherBQSRReports [ 0%] 0 of 1 executor > local (17) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [f8/a0bc4b] process > BaseRecalibrator [100%] 6 of 6 ✔ [f1/135459] process > GatherBQSRReports [100%] 1 of 1 ✔ [15/2c3be8] process > ApplyBQSR [ 0%] 0 of 2 executor > local (21) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [f8/a0bc4b] process > BaseRecalibrator [100%] 6 of 6 ✔ [f1/135459] process > GatherBQSRReports [100%] 1 of 1 ✔ [7d/d4b9c4] process > ApplyBQSR [ 0%] 0 of 6 executor > local (21) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [f8/a0bc4b] process > BaseRecalibrator [100%] 6 of 6 ✔ [f1/135459] process > GatherBQSRReports [100%] 1 of 1 ✔ [15/2c3be8] process > ApplyBQSR [ 17%] 1 of 6 executor > local (21) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [f8/a0bc4b] process > BaseRecalibrator [100%] 6 of 6 ✔ [f1/135459] process > GatherBQSRReports [100%] 1 of 1 ✔ [93/9fedc6] process > ApplyBQSR [ 33%] 2 of 6 executor > local (21) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [f8/a0bc4b] process > BaseRecalibrator [100%] 6 of 6 ✔ [f1/135459] process > GatherBQSRReports [100%] 1 of 1 ✔ [54/d6814e] process > ApplyBQSR [ 67%] 4 of 6 executor > local (21) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [f8/a0bc4b] process > BaseRecalibrator [100%] 6 of 6 ✔ [f1/135459] process > GatherBQSRReports [100%] 1 of 1 ✔ [7d/d4b9c4] process > ApplyBQSR [ 83%] 5 of 6 executor > local (22) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [f8/a0bc4b] process > BaseRecalibrator [100%] 6 of 6 ✔ [f1/135459] process > GatherBQSRReports [100%] 1 of 1 ✔ [24/aa115e] process > ApplyBQSR [100%] 6 of 6 ✔ [a1/85fe1c] process > MergeBamRecal [ 0%] 0 of 1 executor > local (22) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [f8/a0bc4b] process > BaseRecalibrator [100%] 6 of 6 ✔ [f1/135459] process > GatherBQSRReports [100%] 1 of 1 ✔ [24/aa115e] process > ApplyBQSR [100%] 6 of 6 ✔ [a1/85fe1c] process > MergeBamRecal [100%] 1 of 1 ✔ executor > local (22) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [f8/a0bc4b] process > BaseRecalibrator [100%] 6 of 6 ✔ [f1/135459] process > GatherBQSRReports [100%] 1 of 1 ✔ [24/aa115e] process > ApplyBQSR [100%] 6 of 6 ✔ [a1/85fe1c] process > MergeBamRecal [100%] 1 of 1 ✔ Warning, pipeline completed, but with errored process(es) Number of ignored errored process(es) : 0 Number of successfully ran process(es) : 22 WARN: [nf-core/sarek] Could not attach MultiQC report to summary email executor > local (22) [6d/c6e37d] process > CreateIntervalBeds [100%] 1 of 1 ✔ [b0/fe04e4] process > MapReads [100%] 5 of 5 ✔ [64/59cc75] process > MergeBamMapped [100%] 1 of 1 ✔ [34/c7e2d1] process > MarkDuplicates [100%] 1 of 1 ✔ [f8/a0bc4b] process > BaseRecalibrator [100%] 6 of 6 ✔ [f1/135459] process > GatherBQSRReports [100%] 1 of 1 ✔ [24/aa115e] process > ApplyBQSR [100%] 6 of 6 ✔ [a1/85fe1c] process > MergeBamRecal [100%] 1 of 1 ✔ Warning, pipeline completed, but with errored process(es) Number of ignored errored process(es) : 0 Number of successfully ran process(es) : 22 WARN: [nf-core/sarek] Could not attach MultiQC report to summary email WARN: To render the execution DAG in the required format it is required to install Graphviz -- See http://www.graphviz.org for more info. Completed at: 04-Jul-2019 13:01:50 Duration : 55.9s CPU hours : 0.1 Succeeded : 22 + run_sarek --tools=false --sample results/Preprocessing/TSV/duplicateMarked.tsv --step recalibrate --noReports + nextflow run ./main.nf -profile test,docker --monochrome_logs --tools=false --sample results/Preprocessing/TSV/duplicateMarked.tsv --step recalibrate --noReports N E X T F L O W ~ version 19.04.0 Launching `./main.nf` [small_wilson] - revision: fb7cd02724 ---------------------------------------------------- ,--./,-. ___ __ __ __ ___ /,-._.--~' |\ | |__ __ / ` / \ |__) |__ } { | \| | \__, \__/ | \ |___ \`-._,-`-, `._,._,' ____ _____ _ .' _ `. / ____| | | / |\`-_ \ | (___ ___ _ __ __ | | __ | | \ `-| \___ \/__ \| ´__/ _\| |/ / \ | \ / ____) | __ | | | __| < `|____\' |_____/\____|_| \__/|_|\_\ nf-core/sarek v2.5dev ---------------------------------------------------- Run Name : small_wilson Max Resources : 7 GB memory, 2 cpus, 2d time per job Container : docker - nfcore/sarek:dev Sample : results/Preprocessing/TSV/duplicateMarked.tsv Step : recalibrate No Reports : true Nucleotides/s : 1000.0 Output dir : ./results Launch dir : /share/****_home/workspace/nf-core_sarek_PR-19 Working dir : /share/****_home/workspace/nf-core_sarek_PR-19/work Script dir : /share/****_home/workspace/nf-core_sarek_PR-19 User : **** Config Profile : test,docker Config Description: Minimal test dataset to check pipeline function ---------------------------------------------------- executor > local (1) [28/a7838b] process > CreateIntervalBeds [ 0%] 0 of 1 executor > local (1) [28/a7838b] process > CreateIntervalBeds [100%] 1 of 1 ✔ executor > local (7) [28/a7838b] process > CreateIntervalBeds [100%] 1 of 1 ✔ [4a/e15971] process > ApplyBQSR [ 0%] 0 of 6 executor > local (7) [28/a7838b] process > CreateIntervalBeds [100%] 1 of 1 ✔ [84/779aec] process > ApplyBQSR [ 17%] 1 of 6 executor > local (7) [28/a7838b] process > CreateIntervalBeds [100%] 1 of 1 ✔ [4a/e15971] process > ApplyBQSR [ 33%] 2 of 6 executor > local (7) [28/a7838b] process > CreateIntervalBeds [100%] 1 of 1 ✔ [90/7581aa] process > ApplyBQSR [ 50%] 3 of 6 executor > local (7) [28/a7838b] process > CreateIntervalBeds [100%] 1 of 1 ✔ [9c/a847c2] process > ApplyBQSR [ 83%] 5 of 6 executor > local (8) [28/a7838b] process > CreateIntervalBeds [100%] 1 of 1 ✔ [43/9529ba] process > ApplyBQSR [100%] 6 of 6 ✔ [6e/476a8d] process > MergeBamRecal [ 0%] 0 of 1 executor > local (8) [28/a7838b] process > CreateIntervalBeds [100%] 1 of 1 ✔ [43/9529ba] process > ApplyBQSR [100%] 6 of 6 ✔ [6e/476a8d] process > MergeBamRecal [100%] 1 of 1 ✔ executor > local (8) [28/a7838b] process > CreateIntervalBeds [100%] 1 of 1 ✔ [43/9529ba] process > ApplyBQSR [100%] 6 of 6 ✔ [6e/476a8d] process > MergeBamRecal [100%] 1 of 1 ✔ Warning, pipeline completed, but with errored process(es) Number of ignored errored process(es) : 0 Number of successfully ran process(es) : 8 WARN: [nf-core/sarek] Could not attach MultiQC report to summary email executor > local (8) [28/a7838b] process > CreateIntervalBeds [100%] 1 of 1 ✔ [43/9529ba] process > ApplyBQSR [100%] 6 of 6 ✔ [6e/476a8d] process > MergeBamRecal [100%] 1 of 1 ✔ Warning, pipeline completed, but with errored process(es) Number of ignored errored process(es) : 0 Number of successfully ran process(es) : 8 WARN: [nf-core/sarek] Could not attach MultiQC report to summary email WARN: To render the execution DAG in the required format it is required to install Graphviz -- See http://www.graphviz.org for more info. Completed at: 04-Jul-2019 13:02:13 Duration : 18.7s CPU hours : (a few seconds) Succeeded : 8 + run_sarek --tools HaplotypeCaller --sample results/Preprocessing/TSV/recalibrated.tsv --step variantCalling --noReports + nextflow run ./main.nf -profile test,docker --monochrome_logs --tools HaplotypeCaller --sample results/Preprocessing/TSV/recalibrated.tsv --step variantCalling --noReports N E X T F L O W ~ version 19.04.0 Launching `./main.nf` [soggy_gilbert] - revision: fb7cd02724 ---------------------------------------------------- ,--./,-. ___ __ __ __ ___ /,-._.--~' |\ | |__ __ / ` / \ |__) |__ } { | \| | \__, \__/ | \ |___ \`-._,-`-, `._,._,' ____ _____ _ .' _ `. / ____| | | / |\`-_ \ | (___ ___ _ __ __ | | __ | | \ `-| \___ \/__ \| ´__/ _\| |/ / \ | \ / ____) | __ | | | __| < `|____\' |_____/\____|_| \__/|_|\_\ nf-core/sarek v2.5dev ---------------------------------------------------- Run Name : soggy_gilbert Max Resources : 7 GB memory, 2 cpus, 2d time per job Container : docker - nfcore/sarek:dev Sample : results/Preprocessing/TSV/recalibrated.tsv Step : variantCalling Tools : haplotypecaller No Reports : true Nucleotides/s : 1000.0 Output dir : ./results Launch dir : /share/****_home/workspace/nf-core_sarek_PR-19 Working dir : /share/****_home/workspace/nf-core_sarek_PR-19/work Script dir : /share/****_home/workspace/nf-core_sarek_PR-19 User : **** Config Profile : test,docker Config Description: Minimal test dataset to check pipeline function ---------------------------------------------------- executor > local (1) [9d/9e292f] process > CreateIntervalBeds [ 0%] 0 of 1 executor > local (2) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [8b/26de48] process > HaplotypeCaller [ 0%] 0 of 1 executor > local (7) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [47/0cddc1] process > HaplotypeCaller [ 0%] 0 of 6 executor > local (8) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [8b/26de48] process > HaplotypeCaller [ 17%] 1 of 6 [a7/86e87d] process > GenotypeGVCFs [ 0%] 0 of 1 executor > local (9) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [5a/2d6d60] process > HaplotypeCaller [ 33%] 2 of 6 [3b/1bd347] process > GenotypeGVCFs [ 0%] 0 of 2 executor > local (11) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [50/4bfc19] process > HaplotypeCaller [ 67%] 4 of 6 [ff/966091] process > GenotypeGVCFs [ 0%] 0 of 4 executor > local (12) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [d3/7cdb5c] process > HaplotypeCaller [ 83%] 5 of 6 [24/32a2dc] process > GenotypeGVCFs [ 0%] 0 of 5 executor > local (14) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [47/0cddc1] process > HaplotypeCaller [100%] 6 of 6 ✔ [97/127f2a] process > GenotypeGVCFs [ 0%] 0 of 6 [f6/c87ba1] process > ConcatVCF [ 0%] 0 of 1 executor > local (14) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [47/0cddc1] process > HaplotypeCaller [100%] 6 of 6 ✔ [97/127f2a] process > GenotypeGVCFs [ 0%] 0 of 6 [f6/c87ba1] process > ConcatVCF [100%] 1 of 1 executor > local (14) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [47/0cddc1] process > HaplotypeCaller [100%] 6 of 6 ✔ [a7/86e87d] process > GenotypeGVCFs [ 17%] 1 of 6 [f6/c87ba1] process > ConcatVCF [100%] 1 of 1 executor > local (14) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [47/0cddc1] process > HaplotypeCaller [100%] 6 of 6 ✔ [ff/966091] process > GenotypeGVCFs [ 33%] 2 of 6 [f6/c87ba1] process > ConcatVCF [100%] 1 of 1 executor > local (14) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [47/0cddc1] process > HaplotypeCaller [100%] 6 of 6 ✔ [7e/773221] process > GenotypeGVCFs [ 83%] 5 of 6 [f6/c87ba1] process > ConcatVCF [100%] 1 of 1 executor > local (15) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [47/0cddc1] process > HaplotypeCaller [100%] 6 of 6 ✔ [97/127f2a] process > GenotypeGVCFs [100%] 6 of 6 ✔ [5d/19a5ac] process > ConcatVCF [ 50%] 1 of 2 executor > local (15) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [47/0cddc1] process > HaplotypeCaller [100%] 6 of 6 ✔ [97/127f2a] process > GenotypeGVCFs [100%] 6 of 6 ✔ [5d/19a5ac] process > ConcatVCF [100%] 2 of 2 ✔ executor > local (15) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [47/0cddc1] process > HaplotypeCaller [100%] 6 of 6 ✔ [97/127f2a] process > GenotypeGVCFs [100%] 6 of 6 ✔ [5d/19a5ac] process > ConcatVCF [100%] 2 of 2 ✔ Warning, pipeline completed, but with errored process(es) Number of ignored errored process(es) : 0 Number of successfully ran process(es) : 15 WARN: [nf-core/sarek] Could not attach MultiQC report to summary email executor > local (15) [9d/9e292f] process > CreateIntervalBeds [100%] 1 of 1 ✔ [47/0cddc1] process > HaplotypeCaller [100%] 6 of 6 ✔ [97/127f2a] process > GenotypeGVCFs [100%] 6 of 6 ✔ [5d/19a5ac] process > ConcatVCF [100%] 2 of 2 ✔ Warning, pipeline completed, but with errored process(es) Number of ignored errored process(es) : 0 Number of successfully ran process(es) : 15 WARN: [nf-core/sarek] Could not attach MultiQC report to summary email WARN: To render the execution DAG in the required format it is required to install Graphviz -- See http://www.graphviz.org for more info. Completed at: 04-Jul-2019 13:02:53 Duration : 35.3s CPU hours : 0.1 Succeeded : 15 + [[ false == false ]] + rm -rf data + manage_logs + [[ -n '' ]] + [[ ALL,SOMATIC =~ GERMLINE ]] + [[ ALL,TARGETED =~ GERMLINE ]] + [[ false == false ]] + pathToSample=https://github.com/nf-core/test-datasets/raw/sarek/testdata + [[ ALL,ANNOTATEALL,ANNOTATESNPEFF,ANNOTATEVEP =~ GERMLINE ]] + [[ MULTIPLE =~ GERMLINE ]] [Pipeline] sh + rm -rf data/ [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Somatic) [Pipeline] sh + ./scripts/run_tests.sh --test SOMATIC + CPUS=2 + LOGS= + NXF_SINGULARITY_CACHEDIR=work/singularity/. + OFFLINE=false + PROFILE=docker + TEST=ALL + TRAVIS_BUILD_DIR=. + TRAVIS=false + VERBOSE= + [[ 2 -gt 0 ]] + key=--test + case $key in + TEST=SOMATIC + shift + shift + [[ 0 -gt 0 ]] + [[ ALL,GERMLINE =~ SOMATIC ]] + [[ ALL,SOMATIC =~ SOMATIC ]] + [[ false == false ]] + run_sarek --tools FreeBayes,HaplotypeCaller,Manta,Strelka,Mutect2 --noReports + nextflow run ./main.nf -profile test,docker --monochrome_logs --tools FreeBayes,HaplotypeCaller,Manta,Strelka,Mutect2 --noReports N E X T F L O W ~ version 19.04.0 Launching `./main.nf` [infallible_shaw] - revision: fb7cd02724 ---------------------------------------------------- ,--./,-. ___ __ __ __ ___ /,-._.--~' |\ | |__ __ / ` / \ |__) |__ } { | \| | \__, \__/ | \ |___ \`-._,-`-, `._,._,' ____ _____ _ .' _ `. / ____| | | / |\`-_ \ | (___ ___ _ __ __ | | __ | | \ `-| \___ \/__ \| ´__/ _\| |/ / \ | \ / ____) | __ | | | __| < `|____\' |_____/\____|_| \__/|_|\_\ nf-core/sarek v2.5dev ---------------------------------------------------- Run Name : infallible_shaw Max Resources : 7 GB memory, 2 cpus, 2d time per job Container : docker - nfcore/sarek:dev Sample : https://github.com/nf-core/test-datasets/raw/sarek/testdata/tsv/tiny-manta-https.tsv Step : mapping Tools : freebayes, haplotypecaller, manta, strelka, mutect2 No Reports : true Nucleotides/s : 1000.0 Output dir : ./results Launch dir : /share/****_home/workspace/nf-core_sarek_PR-19 Working dir : /share/****_home/workspace/nf-core_sarek_PR-19/work Script dir : /share/****_home/workspace/nf-core_sarek_PR-19 User : **** Config Profile : test,docker Config Description: Minimal test dataset to check pipeline function ---------------------------------------------------- executor > local (1) [a5/c4014e] process > CreateIntervalBeds [ 0%] 0 of 1 [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Targeted) Stage "Targeted" skipped due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Annotation) Stage "Annotation" skipped due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Multiple) Stage "Multiple" skipped due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Declarative: Post Actions) [Pipeline] script [Pipeline] { [Pipeline] sh |
Relaunched the Jenkins tests, and it worked, I have no clue why it failed the first time... |
* Improve cpu usage * Use same font for nf-core and sarek in ascii art * Improve syntax * Add --no-reports option for tests + add snpEff,VEP,merge to MULTIPLE test * Use --no-reports for TravisCI testing * Add possibility to download other genome for sareksnpeff and sarekvep containers * Update docs * Add --no-reports for all tests but MULTIPLE * Use sensible label selectors for configs * Enhance the old --noReports with --skip to skip QC on demand * Replace --noReports by --skip all * Update logos with new nf-core/sarek logo * Add logo to MultiQC report * Remove createMultiQCconfig() * Improve docs about offline usage with bianca * Code polishing * Update CHANGELOG
* fix: fix logo * fix: Fix path to reference genomes * feat: add markdownlint config file * fix: lint all md files + small typos corrected * feat: update CHANGELOG
* add tiddit to container * add process for TIDDIT * update TIDDIT to 2.7.1 * fix merge issues * feat: add tests for TIDDIT * feat: update CHANGELOG * feat: Add documentation for TIDDIT * fix: arrange TIDDIT output * feat: filter out low quality calls from TIDDIT * feat: complete TIDDIT docs * fix: syntac error, bad grep
* feat: merge markdownlint config files * fix: more linting on md files * feat: add md linting to travisCI tests * feat: sort tests * feat: improve travis testing * feat: improve Travis tests * feat: sort tests * feat: use YAML anchors to improve/simplify circle ci docker deployment * fix: remove with stroke over aple core in nf-core logo * feat: use direct link to slack sarek channel * fix: code spacing * feat: remove ALL from tests and simplify/improve script * feat: use kraken profile to run tests * feat: remove unnecessary Channel.create() * fix: removed links to deleted files * feat: moved smallGRCh37 to new genomes.config * feat: update CHANGELOG
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Co-Authored-By: Alexander Peltzer <apeltzer@users.noreply.github.com>
Co-Authored-By: Alexander Peltzer <apeltzer@users.noreply.github.com>
By the way, now that I finally have one circleCI job complete (even for VEP GRCh37 and GRCh38), we now have all images built for annotation, and can merge even if circleCI is not complete or failing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks OK now, means I could not find a major problem in 2 hours. Would go through once extensive tests are over.
* enforcing file names to be idsample_idRun pre merging to avoid collision * more consistency for fastqc and bamqc output * update CHANGELOG
* fix typo for bwaIndex in params.genomes[params.genome] * add more logs for reference Files * fix params.dbsnpIndex * remove bwaIndex from smallGRCh37 * fix ch_dbsnp * update CHANGELOG
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Really great job @maxulysse! I have done a few test runs and it looks very good. I will continue my review but if I find something it will have to wait for the next release. 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice! Also removes the strange "issue PR".
Major PR with first changes for first nf-core release. Pipeline ported from https://github.com/SciLifeLab/Sarek by @maxulysse
Minor things to do before release:
'copy
which is the default for other nf-core pipelines