Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Solved] Bactopia stucks or hangs [Because .nextflow directory accumulates] #390

Closed
llk578496 opened this issue Feb 7, 2023 · 24 comments
Closed
Labels
bug Something isn't working

Comments

@llk578496
Copy link

llk578496 commented Feb 7, 2023

Description
Hello @rpetit3 ,

We are currently using the shigeifinder wf in the bactopia tools, but it stucks for many hours. Please find the logs below.

(base) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ conda activate bactopia
(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ bactopia --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir
2023-02-07 11:34:16:root:INFO - Checking if environment pre-builds are needed, use --verbose to see full details.

Steps to Reproduce
Steps to reproduce the behavior:

conda activate bactopia
bactopia --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir

Expected Behavior
A description of what is expected to happen.

Execution Environment

  • Bactopia Version: 2.2.0
  • OS: Ubuntu
  • Environment: conda, docker

Additional Information
Any other information you think might be helpful in squashing this bug.

@llk578496 llk578496 added the bug Something isn't working label Feb 7, 2023
@rpetit3
Copy link
Member

rpetit3 commented Feb 8, 2023

Hi @llk578496

That's definitely not expected behavior. Is this only happening with shigeifinder? or all workflows?

Thank you,
Robert

@llk578496
Copy link
Author

llk578496 commented Feb 9, 2023

Yes, we just tried running the mcroni workflow and it hangs.

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ bactopia --wf mcroni --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00001-20MB006312 -profile docker --cleanup_workdir
2023-02-09 08:47:23:root:INFO - Checking if environment pre-builds are needed, use --verbose to see full details.

Also, we just tried running the main bactopia workflow and it also hangs

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ bactopia --ont --SE /mnt/data2/Food/REAL/02_fastq/Batch049_P_08_S_CRE_01_Sub_01_Sub.fastq.gz --sample Batch049_P_08_S_CRE_01_Sub_01_Sub --datasets /mnt/data2/Food/REAL/datasets --outdir /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/Batch049_P_08_S_CRE_01_Sub_01_Sub -profile docker --max_cpus 10 --max_memory 110 --max_time 360 --dragonflye_opts="--nanohq" --medaka_model r941_min_sup_g507 --genome_size 5000000 --min_coverage 25 --medaka_steps 1 --cleanup_workdir
2023-02-09 08:55:42:root:INFO - Checking if environment pre-builds are needed, use --verbose to see full details.

We have been using bactopia successfully previously, but just a few days ago (5 Aug 2023 20:44 UTC +8), it suddenly stucks.

@rpetit3
Copy link
Member

rpetit3 commented Feb 9, 2023

Can you try adding a --verbose to the command? Let's see if we can get some more info

@llk578496
Copy link
Author

Yes. It stops here.

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ bactopia --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir --verbose
2023-02-09 09:08:29:executor.process:DEBUG - Executing external command: bash -c 'nextflow config -flat /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/main.nf'
2023-02-09 09:08:29:executor.process:DEBUG - Constructing subprocess.Popen object ..
2023-02-09 09:08:29:executor.process:DEBUG - Joining synchronous process using subprocess.Popen.communicate() ..
2023-02-09 09:08:31:executor.process:DEBUG - Got return code 0 from synchronous process (bash -c 'nextflow config -flat /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/main.nf').
2023-02-09 09:08:31:root:STDOUT - manifest.author = 'Robert A. Petit III'
manifest.name = 'bactopia'
manifest.homePage = 'https://github.com/bactopia/bactopia'
manifest.description = 'An extensive workflow for processing Illumina sequencing of bacterial genomes.'
manifest.mainScript = 'main.nf'
manifest.version = '2.2.0'
manifest.nextflowVersion = '>=19'
container_version = '2.2.x'
params.wf = 'bactopia'
params.list_wfs = false
params.outdir = './'
params.run_name = 'bactopia'
params.skip_compression = false
params.keep_all_files = false
params.schema_ignore_params = 'build_all,use_mamba,empty_proteins,empty_tf,empty_adapters,empty_phix,help_all,include_tools,min_time,publish_dir,silent,test_data,test_data_dir,verbose,schema_inputs,empty_extra,empty_r2,empty_r1,workflows,available_workflows,is_subworkflow,enable_conda,force_rebuild,config_profile_url,config_profile_contact,config_profile_description,config_profile_name'
params.schema_inputs = ['fastqs', 'R1,R2,SE,sample,hybrid', 'R1,R2,sample', 'SE,sample', 'assembly,sample', 'accessions', 'accession']
params.max_retry = 3
params.min_time = 60
params.max_time = 120
params.max_memory = 32
params.max_cpus = 4
params.max_downloads = 3
params.nfconfig = null
params.publish_dir_mode = 'copy'
params.force = false
params.cleanup_workdir = false
params.condadir = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/conda/envs'
params.use_mamba = false
params.registry = 'quay'
params.singularity_cache = '/home/gilmansiu3/.bactopia/singularity'
params.singularity_pull_docker_container = false
params.force_rebuild = false
params.help = null
params.help_all = false
params.sleep_time = 5
params.validate_params = true
params.monochrome_logs = false
params.enable_conda = true
params.show_hidden_params = false
params.build_all = false
params.include_tools = null
params.verbose = null
params.silent = null
params.test_data_dir = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data'
params.available_workflows.bactopia = ['bactopia', 'staphopia']
params.available_workflows.bactopiatools.subworkflows = ['ariba', 'bakta', 'eggnog', 'gtdb', 'merlin', 'pangenome', 'snippy', 'staphtyper']
params.available_workflows.bactopiatools.modules = ['abricate', 'agrvate', 'amrfinderplus', 'busco', 'checkm', 'ectyper', 'emmtyper', 'fastani', 'gamma', 'genotyphi', 'hicap', 'hpsuissero', 'ismapper', 'kleborate', 'kraken2', 'legsta', 'lissero', 'mashdist', 'mashtree', 'mcroni', 'meningotype', 'mlst', 'mobsuite', 'mykrobe', 'ngmaster', 'pasty', 'pbptyper', 'plasmidfinder', 'prokka', 'rgi', 'seqsero2', 'seroba', 'shigatyper', 'shigeifinder', 'sistr', 'spatyper', 'ssuissero', 'staphopiasccmec', 'tbprofiler']
params.workflows.bactopia.description = 'Bactopia is a flexible pipeline for complete analysis of bacterial genomes.'
params.workflows.bactopia.includes = ['bactopia']
params.workflows.bactopia.is_workflow = true
params.workflows.bactopia.modules = ['gather_samples', 'qc_reads', 'assemble_genome', 'assembly_qc', 'annotate_genome', 'minmer_sketch', 'minmer_query', 'antimicrobial_resistance', 'minmer_query', 'blast', 'call_variants', 'mapping_query', 'sequence_type']
params.workflows.staphopia.description = 'Staphopia is a flexible pipeline for complete analysis of Staphylococcus aureus genomes.'
params.workflows.staphopia.includes = ['bactopia', 'staphtyper']
params.workflows.staphopia.is_workflow = true
params.workflows.ariba.description = 'Gene identification through local assemblies'
params.workflows.ariba.ext = 'fastq'
params.workflows.ariba.modules = ['ariba_getref', 'ariba_run']
params.workflows.bakta.description = 'Rapid annotation of bacterial genomes & plasmids'
params.workflows.bakta.ext = 'fna'
params.workflows.bakta.modules = ['bakta_download', 'bakta_run']
params.workflows.eggnog.description = 'Functional annotation of proteins using orthologous groups and phylogenies'
params.workflows.eggnog.ext = 'faa'
params.workflows.eggnog.modules = ['eggnog_download', 'eggnog_mapper']
params.workflows.fastani.description = 'Fast alignment-free computation of whole-genome Average Nucleotide Identity (ANI)'
params.workflows.fastani.ext = 'fna'
params.workflows.fastani.path = 'modules/nf-core/fastani'
params.workflows.fastani.modules = ['ncbigenomedownload']
params.workflows.fastani.use_local = 'assembly'
params.workflows.gtdb.description = 'Identify marker genes and assign taxonomic classifications'
params.workflows.gtdb.ext = 'fna'
params.workflows.gtdb.modules = ['gtdbtk_setupdb', 'gtdbtk_classifywf']
params.workflows.merlin.description = 'MinmER assisted species-specific bactopia tool seLectIoN'
params.workflows.merlin.ext = 'fna_fastq'
params.workflows.merlin.modules = ['mashdist', 'agrvate', 'ectyper', 'emmtyper', 'genotyphi', 'hicap', 'hpsuissero', 'kleborate', 'legsta', 'lissero', 'meningotype', 'ngmaster', 'pasty', 'pbptyper', 'seqsero2', 'seroba', 'shigatyper', 'shigeifinder', 'sistr', 'spatyper', 'ssuissero', 'staphopiasccmec', 'tbprofiler']
params.workflows.pangenome.description = 'Pangenome analysis with optional core-genome phylogeny'
params.workflows.pangenome.ext = 'gff'
params.workflows.pangenome.modules = ['ncbigenomedownload', 'pirate', 'roary', 'panaroo', 'prokka', 'iqtree', 'clonalframeml', 'snpdists', 'scoary']
params.workflows.pangenome.use_local = 'gff'
params.workflows.snippy.description = 'Rapid variant calling from Illumina sequence reads with optional core-SNP phylogeny'
params.workflows.snippy.ext = 'fastq'
params.workflows.snippy.modules = ['snippy_run', 'snippy_core', 'gubbins', 'iqtree', 'snpdists']
params.workflows.staphtyper.description = 'Includes AgrVATE, SpaTyper and Staphopia SCCmec for Staphylococcus aureus'
params.workflows.staphtyper.ext = 'fna'
params.workflows.staphtyper.modules = ['agrvate', 'spatyper', 'staphopiasccmec']
params.workflows.annotate_genome.path = 'modules/local/bactopia/annotate_genome'
params.workflows.antimicrobial_resistance.path = 'modules/local/bactopia/antimicrobial_resistance'
params.workflows.assemble_genome.path = 'modules/local/bactopia/assemble_genome'
params.workflows.assembly_qc.path = 'modules/local/bactopia/assembly_qc'
params.workflows.blast.path = 'modules/local/bactopia/blast'
params.workflows.call_variants.path = 'modules/local/bactopia/call_variants'
params.workflows.gather_samples.path = 'modules/local/bactopia/gather_samples'
params.workflows.mapping_query.path = 'modules/local/bactopia/mapping_query'
params.workflows.minmer_query.path = 'modules/local/bactopia/minmer_query'
params.workflows.minmer_sketch.path = 'modules/local/bactopia/minmer_sketch'
params.workflows.qc_reads.path = 'modules/local/bactopia/qc_reads'
params.workflows.sequence_type.path = 'modules/local/bactopia/sequence_type'
params.workflows.abricate.description = 'Mass screening of contigs for antimicrobial and virulence genes'
params.workflows.abricate.ext = 'fna'
params.workflows.abricate.path = 'modules/nf-core/abricate/run'
params.workflows.agrvate.description = 'Rapid identification of Staphylococcus aureus agr locus type and agr operon variants'
params.workflows.agrvate.ext = 'fna'
params.workflows.agrvate.path = 'modules/nf-core/agrvate'
params.workflows.amrfinderplus.description = 'Identify antimicrobial resistance in genes or proteins'
params.workflows.amrfinderplus.ext = 'fna_faa'
params.workflows.amrfinderplus.path = 'modules/nf-core/amrfinderplus/run'
params.workflows.ariba_getref.path = 'modules/nf-core/ariba/getref'
params.workflows.ariba_run.path = 'modules/nf-core/ariba/run'
params.workflows.bakta_download.path = 'modules/nf-core/bakta/download'
params.workflows.bakta_run.path = 'modules/nf-core/bakta/run'
params.workflows.busco.description = 'Assembly completeness based on evolutionarily informed expectations'
params.workflows.busco.ext = 'fna'
params.workflows.busco.path = 'modules/nf-core/busco'
params.workflows.checkm.description = 'Assess the assembly quality of your microbial samples'
params.workflows.checkm.ext = 'fna'
params.workflows.checkm.path = 'modules/nf-core/checkm/lineagewf'
params.workflows.clonalframeml.description = 'Predict recomination events in bacterial genomes'
params.workflows.clonalframeml.path = 'modules/nf-core/clonalframeml'
params.workflows.csvtk_concat.path = 'modules/nf-core/csvtk/concat'
params.workflows.custom_dumpsoftwareversions.path = 'modules/nf-core/custom/dumpsoftwareversions'
params.workflows.ectyper.description = 'In-silico prediction of Escherichia coli serotype'
params.workflows.ectyper.ext = 'fna'
params.workflows.ectyper.path = 'modules/nf-core/ectyper'
params.workflows.eggnog_download.path = 'modules/nf-core/eggnog/download'
params.workflows.eggnog_mapper.path = 'modules/nf-core/eggnog/mapper'
params.workflows.emmtyper.description = 'emm-typing of Streptococcus pyogenes assemblies'
params.workflows.emmtyper.ext = 'fna'
params.workflows.emmtyper.path = 'modules/nf-core/emmtyper'
params.workflows.gtdbtk_setupdb.path = 'modules/nf-core/gtdbtk/setupdb'
params.workflows.gtdbtk_classifywf.path = 'modules/nf-core/gtdbtk/classifywf'
params.workflows.gamma.description = 'Identification, classification, and annotation of translated gene matches'
params.workflows.gamma.ext = 'fna'
params.workflows.gamma.path = 'modules/nf-core/gamma'
params.workflows.genotyphi.description = 'Salmonella Typhi genotyping with Mykrobe outputs'
params.workflows.genotyphi.ext = 'fastq'
params.workflows.genotyphi.path = 'modules/nf-core/genotyphi/parse'
params.workflows.gubbins.description = 'Rapid phylogenetic analysis of recombinant bacterial sequences'
params.workflows.gubbins.path = 'modules/nf-core/gubbins'
params.workflows.hicap.description = 'Identify cap locus serotype and structure in your Haemophilus influenzae assemblies'
params.workflows.hicap.ext = 'fna'
params.workflows.hicap.path = 'modules/nf-core/hicap'
params.workflows.hpsuissero.description = 'Rapid Haemophilus parasuis Serotyping of assemblies'
params.workflows.hpsuissero.ext = 'fna'
params.workflows.hpsuissero.path = 'modules/nf-core/hpsuissero'
params.workflows.iqtree.description = 'Phylogeny from a multiple sequence alignment using the maxium likelihood algorithm.'
params.workflows.iqtree.path = 'modules/nf-core/iqtree'
params.workflows.ismapper.description = 'Identify insertion sites positions in bacterial genomes'
params.workflows.ismapper.ext = 'fastq'
params.workflows.ismapper.path = 'modules/nf-core/ismapper'
params.workflows.kleborate.description = 'Screen for MLST, sub-species, and other Klebsiella related genes of interest'
params.workflows.kleborate.ext = 'fna'
params.workflows.kleborate.path = 'modules/nf-core/kleborate'
params.workflows.kraken2.description = 'Taxonomic classification of sequence reads'
params.workflows.kraken2.ext = 'fastq'
params.workflows.kraken2.path = 'modules/nf-core/kraken2'
params.workflows.legsta.description = 'Typing of Legionella pneumophila assemblies'
params.workflows.legsta.ext = 'fna'
params.workflows.legsta.path = 'modules/nf-core/legsta'
params.workflows.lissero.description = 'Serogroup typing prediction for Listeria monocytogenes'
params.workflows.lissero.ext = 'fna'
params.workflows.lissero.path = 'modules/nf-core/lissero'
params.workflows.mashdist.description = 'Calculate Mash distances between sequences'
params.workflows.mashdist.ext = 'fna'
params.workflows.mashdist.path = 'modules/nf-core/mash/dist'
params.workflows.mashtree.description = 'Create a trees using Mash distance'
params.workflows.mashtree.ext = 'fna'
params.workflows.mashtree.path = 'modules/nf-core/mashtree'
params.workflows.mcroni.description = 'Sequence variation in mobilized colistin resistance (mcr-1) genes'
params.workflows.mcroni.ext = 'fna'
params.workflows.mcroni.path = 'modules/nf-core/mcroni'
params.workflows.meningotype.description = 'Serotyping of Neisseria meningitidis'
params.workflows.meningotype.ext = 'fna'
params.workflows.meningotype.path = 'modules/nf-core/meningotype'
params.workflows.mlst.description = 'Scan contig files against PubMLST typing schemes'
params.workflows.mlst.ext = 'fna'
params.workflows.mlst.path = 'modules/nf-core/mlst'
params.workflows.mobsuite.description = 'Reconstruct and annotate plasmids in bacterial assemblies'
params.workflows.mobsuite.ext = 'fna'
params.workflows.mobsuite.path = 'modules/nf-core/mobsuite/recon'
params.workflows.mykrobe.description = 'Antimicrobial resistance detection for specific species'
params.workflows.mykrobe.ext = 'fastq'
params.workflows.mykrobe.path = 'modules/nf-core/mykrobe/predict'
params.workflows.ncbigenomedownload.description = 'Quickly download assemblies from NCBI's Assembly database'
params.workflows.ncbigenomedownload.path = 'modules/nf-core/ncbigenomedownload'
params.workflows.ngmaster.description = 'Multi-antigen sequence typing for Neisseria gonorrhoeae'
params.workflows.ngmaster.ext = 'fna'
params.workflows.ngmaster.path = 'modules/nf-core/ngmaster'
params.workflows.panaroo.description = 'Pipeline for pangenome investigations'
params.workflows.panaroo.ext = 'gff'
params.workflows.panaroo.path = 'modules/nf-core/panaroo/run'
params.workflows.pasty.description = 'Serogrouping of Pseudomonas aeruginosa isolates'
params.workflows.pasty.ext = 'fna'
params.workflows.pasty.path = 'modules/nf-core/pasty'
params.workflows.pbptyper.description = 'Penicillin Binding Protein (PBP) typer for Streptococcus pneumoniae'
params.workflows.pbptyper.ext = 'fna'
params.workflows.pbptyper.path = 'modules/nf-core/pbptyper'
params.workflows.pirate.description = 'Pangenome toolbox for bacterial genomes'
params.workflows.pirate.ext = 'gff'
params.workflows.pirate.path = 'modules/nf-core/pirate'
params.workflows.plasmidfinder.description = 'Plasmid identification from assemblies'
params.workflows.plasmidfinder.ext = 'fna'
params.workflows.plasmidfinder.path = 'modules/nf-core/plasmidfinder'
params.workflows.prokka.description = 'Whole genome annotation of small genomes (bacterial, archeal, viral)'
params.workflows.prokka.ext = 'fna'
params.workflows.prokka.path = 'modules/nf-core/prokka'
params.workflows.rgi.description = 'Predict antibiotic resistance from assemblies'
params.workflows.rgi.ext = 'fna'
params.workflows.rgi.path = 'modules/nf-core/rgi/main'
params.workflows.roary.description = 'Rapid large-scale prokaryote pangenome analysis'
params.workflows.roary.ext = 'gff'
params.workflows.roary.path = 'modules/nf-core/roary'
params.workflows.scoary.description = 'GWAS analysis using pangenome outputs'
params.workflows.scoary.ext = 'csv'
params.workflows.scoary.path = 'modules/nf-core/scoary'
params.workflows.seqsero2.description = 'Salmonella serotype prediction from reads or assemblies'
params.workflows.seqsero2.ext = 'fna'
params.workflows.seqsero2.path = 'modules/nf-core/seqsero2'
params.workflows.seroba.description = 'Serotyping of Streptococcus pneumoniae from sequence reads'
params.workflows.seroba.ext = 'fastq'
params.workflows.seroba.path = 'modules/nf-core/seroba/run'
params.workflows.shigatyper.description = 'Shigella serotype from Illumina or Oxford Nanopore reads'
params.workflows.shigatyper.ext = 'fastq'
params.workflows.shigatyper.path = 'modules/nf-core/shigatyper'
params.workflows.shigeifinder.description = 'Shigella and EIEC serotyping from assemblies'
params.workflows.shigeifinder.ext = 'fna'
params.workflows.shigeifinder.path = 'modules/nf-core/shigeifinder'
params.workflows.sistr.description = 'Serovar prediction of Salmonella assemblies'
params.workflows.sistr.ext = 'fna'
params.workflows.sistr.path = 'modules/nf-core/sistr'
params.workflows.snippy_run.description = 'Rapid variant calling from Illumina sequence reads'
params.workflows.snippy_run.path = 'modules/nf-core/snippy/run'
params.workflows.snippy_core.description = 'Core-SNP alignment from Snippy outputs'
params.workflows.snippy_core.path = 'modules/nf-core/snippy/core'
params.workflows.snpdists.description = 'Pairwise SNP distance matrix from a FASTA sequence alignment'
params.workflows.snpdists.ext = 'fna'
params.workflows.snpdists.path = 'modules/nf-core/snpdists'
params.workflows.spatyper.description = 'Computational method for finding spa types in Staphylococcus aureus'
params.workflows.spatyper.ext = 'fna'
params.workflows.spatyper.path = 'modules/nf-core/spatyper'
params.workflows.staphopiasccmec.description = 'Primer based SCCmec typing of Staphylococcus aureus genomes'
params.workflows.staphopiasccmec.ext = 'fna'
params.workflows.staphopiasccmec.path = 'modules/nf-core/staphopiasccmec'
params.workflows.ssuissero.description = 'Rapid Streptococcus suis Serotyping of assemblies'
params.workflows.ssuissero.ext = 'fna'
params.workflows.ssuissero.path = 'modules/nf-core/ssuissero'
params.workflows.tbprofiler.description = 'Detect resistance and lineages of Mycobacterium tuberculosis genomes'
params.workflows.tbprofiler.ext = 'fastq'
params.workflows.tbprofiler.path = 'modules/nf-core/tbprofiler/profile'
params.is_subworkflow = false
params.available_datasets = null
params.empty_r1 = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/data/EMPTY_R1'
params.empty_r2 = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/data/EMPTY_R2'
params.empty_extra = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/data/EMPTY_EXTRA'
params.empty_proteins = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/data/EMPTY_PROTEINS'
params.empty_tf = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/data/EMPTY_TF'
params.empty_adapters = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/data/EMPTY_ADAPTERS'
params.empty_phix = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/data/EMPTY_PHIX'
params.samples = null
params.check_samples = false
params.R1 = null
params.R2 = null
params.SE = null
params.sample = null
params.ont = false
params.hybrid = false
params.short_polish = false
params.accessions = null
params.accession = null
params.assembly = null
params.reassemble = false
params.datasets = null
params.species = null
params.ask_merlin = false
params.coverage = 100
params.genome_size = '0'
params.skip_fastq_check = false
params.min_basepairs = 2241820
params.min_reads = 7472
params.min_coverage = 10
params.min_proportion = 0.5
params.min_genome_size = 100000
params.max_genome_size = 18040666
params.attempts = 3
params.use_ena = false
params.no_cache = false
params.skip_qc = false
params.skip_qc_plots = false
params.skip_error_correction = false
params.adapters = null
params.adapter_k = 23
params.phix = null
params.phix_k = 31
params.ktrim = 'r'
params.mink = 11
params.hdist = 1
params.tpe = 't'
params.tbo = 't'
params.qtrim = 'rl'
params.trimq = 6
params.maq = 10
params.minlength = 35
params.ftm = 5
params.tossjunk = 't'
params.qout = '33'
params.maxcor = 1
params.sampleseed = 42
params.ain = 'f'
params.bbduk_opts = ''
params.ont_minqual = 0
params.ont_minlength = 1000
params.porechop_opts = ''
params.nanoplot_opts = ''
params.use_fastp = false
params.fastp_opts = ''
params.shovill_assembler = 'skesa'
params.dragonflye_assembler = 'flye'
params.use_unicycler = false
params.min_contig_len = 500
params.min_contig_cov = 2
params.contig_namefmt = null
params.shovill_opts = null
params.shovill_kmers = null
params.dragonflye_opts = null
params.trim = null
params.no_stitch = null
params.no_corr = null
params.unicycler_mode = 'normal'
params.min_component_size = 1000
params.min_dead_end_size = 1000
params.no_miniasm = false
params.no_polish = false
params.no_rotate = false
params.racon_steps = 1
params.medaka_steps = 0
params.medaka_model = ''
params.polypolish_rounds = 1
params.pilon_rounds = 0
params.run_checkm = false
params.checkm_unique = 10
params.checkm_multi = 10
params.aai_strain = 0.9
params.checkm_length = 0.7
params.full_tree = null
params.skip_pseudogene_correction = null
params.ignore_thresholds = null
params.checkm_ali = null
params.checkm_nt = null
params.force_domain = null
params.no_refinement = null
params.individual_markers = null
params.skip_adj_correction = null
params.contig_thresholds = '0,1000,10000,100000,250000,1000000'
params.plots_format = 'pdf'
params.compliant = false
params.centre = 'Bactopia'
params.addmrna = false
params.rawproduct = null
params.cdsrnaolap = null
params.prokka_evalue = '1e-09'
params.prokka_coverage = 80
params.nogenes = false
params.norrna = null
params.notrna = null
params.rnammer = null
params.rfam = null
params.skip_prodigal_tf = false
params.use_bakta = false
params.count_31mers = false
params.keep_singletons = false
params.sketch_size = 10000
params.sourmash_scale = 10000
params.no_winner_take_all = false
params.screen_i = 0.8
params.skip_amr = false
params.amr_plus = false
params.amr_report_common = false
params.amr_organism = null
params.amr_ident_min = -1
params.amr_coverage_min = 0.5
params.amr_translation_table = 11
params.perc_identity = 50
params.qcov_hsp_perc = 50
params.max_target_seqs = 2000
params.mapqual = 60
params.basequal = 13
params.mincov = 10
params.minfrac = 0
params.minqual = 100
params.maxsoft = 10
params.bwaopt = null
params.fbopt = null
params.random_tie_break = false
params.disable_auto_variants = false
params.keep_unmapped_reads = false
params.bwa_mem_opts = null
params.bwa_aln_opts = null
params.bwa_samse_opts = null
params.bwa_sampe_opts = null
params.bwa_n = 9999
params.test_data.accessions.gcf = 'GCF_000292685'
params.test_data.accessions.gcfs = ['GCF_002849995', 'GCF_000292685', 'GCF_002849975']
params.test_data.accessions.srx = 'SRX1390609'
params.test_data.accessions.srxs = ['SRX1390609', 'SRX1390608', 'SRX849002']
params.test_data.datasets.amrdb.amrfinder = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/amrdb/amrfinderdb.tar.gz'
params.test_data.datasets.ariba.card = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/ariba/card.tar.gz'
params.test_data.datasets.ariba.vfdb_core = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/ariba/vfdb_core.tar.gz'
params.test_data.datasets.bakta = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/bakta'
params.test_data.datasets.bakta_tarball = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/bakta/bakta.tar.gz'
params.test_data.datasets.blast.genes = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/blast/genes'
params.test_data.datasets.blast.proteins = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/blast/proteins'
params.test_data.datasets.blast.primers = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/blast/primers'
params.test_data.datasets.eggnog = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/eggnog'
params.test_data.datasets.eggnog_tarball = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/eggnog/eggnog.tar.gz'
params.test_data.datasets.gtdb = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/gtdb'
params.test_data.datasets.gtdb_tarball = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/gtdb/gtdb.tar.gz'
params.test_data.datasets.kraken2 = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/kraken2'
params.test_data.datasets.kraken2_tarball = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/kraken2/kraken2.tar.gz'
params.test_data.datasets.mapping = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/mapping-sequences'
params.test_data.datasets.minmer = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/minmer'
params.test_data.datasets.mlst = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/mlst'
params.test_data.datasets.refseq_genomes = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/variants/refseq-genomes.msh'
params.test_data.datasets.refseq_msh = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/datasets/minmer/mash-refseq-k21.msh'
params.test_data.empty.adapters = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/empty/EMPTY_ADAPTERS'
params.test_data.empty.fna = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/empty/EMPTY.fna.gz'
params.test_data.empty.fastq = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/empty/EMPTY.fastq.gz'
params.test_data.empty.phix = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/empty/EMPTY_PHIX'
params.test_data.empty.proteins = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/empty/EMPTY_PROTEINS'
params.test_data.empty.prodigal_tf = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/empty/EMPTY_TF'
params.test_data.empty.r1 = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/empty/EMPTY_R1.fastq.gz'
params.test_data.empty.r2 = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/empty/EMPTY_R2.fastq.gz'
params.test_data.species.portiera.genome.name = 'GCF_000292685'
params.test_data.species.portiera.genome.length = '358242'
params.test_data.species.portiera.genome.blastdb = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/blastdb'
params.test_data.species.portiera.genome.aln_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/genome.aln.gz'
params.test_data.species.portiera.genome.faa = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/GCF_000292685.faa'
params.test_data.species.portiera.genome.faa_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/GCF_000292685.faa.gz'
params.test_data.species.portiera.genome.faa_single_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/GCF_000292685-single.faa.gz'
params.test_data.species.portiera.genome.ffn = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/GCF_000292685.ffn'
params.test_data.species.portiera.genome.ffn_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/GCF_000292685.ffn.gz'
params.test_data.species.portiera.genome.fna = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/GCF_000292685.fna'
params.test_data.species.portiera.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/GCF_000292685.fna.gz'
params.test_data.species.portiera.genome.gbk = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/GCF_000292685.gbk'
params.test_data.species.portiera.genome.gbk_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/GCF_000292685.gbk.gz'
params.test_data.species.portiera.genome.gff = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/gff/GCF_000292685.gff'
params.test_data.species.portiera.genome.gene_presence_absence = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/gene_presence_absence.csv'
params.test_data.species.portiera.genome.gene_traits = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/gene_traits.csv'
params.test_data.species.portiera.genome.gff_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/gff/GCF_000292685.gff.gz'
params.test_data.species.portiera.genome.gff2_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/gff/GCF_000298385.gff.gz'
params.test_data.species.portiera.genome.gff3_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/gff/GCF_002849995.gff.gz'
params.test_data.species.portiera.genome.gff4_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/gff/GCF_000300075.gff.gz'
params.test_data.species.portiera.genome.gff5_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/gff/GCF_902713755.gff.gz'
params.test_data.species.portiera.genome.msh = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/GCF_000292685.msh'
params.test_data.species.portiera.genome.genome_size = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/GCF_000292685-genome-size.txt'
params.test_data.species.portiera.genome.total_contigs = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/genome/total_contigs_1'
params.test_data.species.portiera.illumina.name = 'SRR2838702'
params.test_data.species.portiera.illumina.msh = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/illumina/SRR2838702.msh'
params.test_data.species.portiera.illumina.sig = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/illumina/SRR2838702.sig'
params.test_data.species.portiera.illumina.r1 = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/illumina/SRR2838702_R1.fastq.gz'
params.test_data.species.portiera.illumina.r1_low_depth = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/illumina/SRR2838702_R1-1x.fastq.gz'
params.test_data.species.portiera.illumina.r1_low_read_count = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/illumina/SRR2838702_R1-1-read.fastq.gz'
params.test_data.species.portiera.illumina.r2 = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/illumina/SRR2838702_R2.fastq.gz'
params.test_data.species.portiera.illumina.r2_low_depth = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/illumina/SRR2838702_R2-1x.fastq.gz'
params.test_data.species.portiera.illumina.r2_low_proportion = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/illumina/SRR2838702_R2-low-proportion.fastq.gz'
params.test_data.species.portiera.illumina.r2_low_read_count = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/illumina/SRR2838702_R2-1-read.fastq.gz'
params.test_data.species.portiera.illumina.se = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/illumina/SRR2838702SE.fastq.gz'
params.test_data.species.portiera.illumina.se_low_depth = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/illumina/SRR2838702SE-1x.fastq.gz'
params.test_data.species.portiera.illumina.se_low_read_count = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/illumina/SRR2838702SE-1-read.fastq.gz'
params.test_data.species.portiera.nanopore.name = 'ERR3772599'
params.test_data.species.portiera.nanopore.se = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/portiera/nanopore/ERR3772599.fastq.gz'
params.test_data.species.escherichia_coli.genome.name = 'GCF_001695515'
params.test_data.species.escherichia_coli.genome.length = '5866200'
params.test_data.species.escherichia_coli.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/escherichia_coli/genome/GCF_001695515.fna.gz'
params.test_data.species.escherichia_coli_mcr1.genome.name = 'GCF_001682305'
params.test_data.species.escherichia_coli_mcr1.genome.length = '4617703'
params.test_data.species.escherichia_coli_mcr1.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/escherichia_coli/genome/GCF_001682305.fna.gz'
params.test_data.species.glaesserella_parasuis.genome.name = 'GCF_002777395'
params.test_data.species.glaesserella_parasuis.genome.length = '2479163'
params.test_data.species.glaesserella_parasuis.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/glaesserella_parasuis/genome/GCF_002777395.fna.gz'
params.test_data.species.haemophilus_influenzae.genome.name = 'GCF_900478275'
params.test_data.species.haemophilus_influenzae.genome.length = '1890469'
params.test_data.species.haemophilus_influenzae.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/haemophilus_influenzae/genome/GCF_900478275.fna.gz'
params.test_data.species.haemophilus_influenzae.genome.gbk_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/haemophilus_influenzae/genome/GCF_900478275.gbk.gz'
params.test_data.species.haemophilus_influenzae.genome.msa_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/haemophilus_influenzae/genome/genome_msa.fa.gz'
params.test_data.species.haemophilus_influenzae.genome.nwk = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/haemophilus_influenzae/genome/genome_msa.nwk'
params.test_data.species.haemophilus_influenzae.genome.is1016 = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/haemophilus_influenzae/genome/is1016.fasta'
params.test_data.species.haemophilus_influenzae.nthi.name = 'GCA_000027305'
params.test_data.species.haemophilus_influenzae.nthi.length = '1830138'
params.test_data.species.haemophilus_influenzae.nthi.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/haemophilus_influenzae/nthi/GCA_000027305.fna.gz'
params.test_data.species.klebsiella_pneumoniae.genome.name = 'GCF_000009885'
params.test_data.species.klebsiella_pneumoniae.genome.length = '5472672'
params.test_data.species.klebsiella_pneumoniae.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/klebsiella_pneumoniae/genome/GCF_000009885.fna.gz'
params.test_data.species.legionella_pneumophila.genome.name = 'GCF_000048645'
params.test_data.species.legionella_pneumophila.genome.length = '3635495'
params.test_data.species.legionella_pneumophila.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/legionella_pneumophila/genome/GCF_000048645.fna.gz'
params.test_data.species.listeria_monocytogenes.genome.name = 'GCF_002285835'
params.test_data.species.listeria_monocytogenes.genome.length = '3148183'
params.test_data.species.listeria_monocytogenes.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/listeria_monocytogenes/genome/GCF_002285835.fna.gz'
params.test_data.species.neisseria_meningitidis.genome.name = 'GCF_003355215'
params.test_data.species.neisseria_meningitidis.genome.length = '2314390'
params.test_data.species.neisseria_meningitidis.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/neisseria_meningitidis/genome/GCF_003355215.fna.gz'
params.test_data.species.neisseria_gonorrhoeae.genome.name = 'GCF_001047255'
params.test_data.species.neisseria_gonorrhoeae.genome.length = '2168698'
params.test_data.species.neisseria_gonorrhoeae.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/neisseria_gonorrhoeae/genome/GCF_001047255.fna.gz'
params.test_data.species.pseudomonas_aeruginosa.genome.name = 'GCF_000006765'
params.test_data.species.pseudomonas_aeruginosa.genome.length = '6264404'
params.test_data.species.pseudomonas_aeruginosa.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/pseudomonas_aeruginosa/genome/GCF_000006765.fna.gz'
params.test_data.species.salmonella_enterica.genome.name = 'GCF_016028495'
params.test_data.species.salmonella_enterica.genome.length = '4968720'
params.test_data.species.salmonella_enterica.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/salmonella_enterica/genome/GCF_016028495.fna.gz'
params.test_data.species.shigella_boydii.genome.name = 'GCF_016726285'
params.test_data.species.shigella_boydii.genome.length = '4548972'
params.test_data.species.shigella_boydii.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/shigella_boydii/genome/GCF_016726285.fna.gz'
params.test_data.species.shigella_boydii.illumina.name = ''
params.test_data.species.shigella_boydii.illumina.r1 = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/shigella_boydii/illumina/XXXXXX_R1.fastq.gz'
params.test_data.species.shigella_boydii.illumina.r2 = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/shigella_boydii/illumina/XXXXXX_R2.fastq.gz'
params.test_data.species.shigella_boydii.illumina.se = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/shigella_boydii/illumina/XXXXXXSE.fastq.gz'
params.test_data.species.shigella_boydii.nanopore.name = ''
params.test_data.species.shigella_boydii.nanopore.se = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/shigella_boydii/nanopore/XXXXXX.fastq.gz'
params.test_data.species.shigella_dysenteriae.genome.name = 'GCF_002949675'
params.test_data.species.shigella_dysenteriae.genome.length = '4578459'
params.test_data.species.shigella_dysenteriae.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/shigella_dysenteriae/genome/GCF_002949675.fna.gz'
params.test_data.species.shigella_dysenteriae.illumina.name = 'ERR6005894'
params.test_data.species.shigella_dysenteriae.illumina.r1 = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/shigella_dysenteriae/illumina/ERR6005894_R1.fastq.gz'
params.test_data.species.shigella_dysenteriae.illumina.r2 = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/shigella_dysenteriae/illumina/ERR6005894_R2.fastq.gz'
params.test_data.species.shigella_dysenteriae.illumina.se = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/shigella_dysenteriae/illumina/ERR6005894SE.fastq.gz'
params.test_data.species.shigella_dysenteriae.nanopore.name = 'SRR13039589'
params.test_data.species.shigella_dysenteriae.nanopore.se = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/shigella_dysenteriae/nanopore/SRR13039589.fastq.gz'
params.test_data.species.staphylococcus_aureus.genome.name = 'GCF_000017085'
params.test_data.species.staphylococcus_aureus.genome.length = '2903081'
params.test_data.species.staphylococcus_aureus.genome.fna = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/staphylococcus_aureus/genome/GCF_000017085.fna'
params.test_data.species.staphylococcus_aureus.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/staphylococcus_aureus/genome/GCF_000017085.fna.gz'
params.test_data.species.streptococcus_pyogenes.genome.name = 'GCF_006364235'
params.test_data.species.streptococcus_pyogenes.genome.length = '1914862'
params.test_data.species.streptococcus_pyogenes.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/streptococcus_pyogenes/genome/GCF_006364235.fna.gz'
params.test_data.species.streptococcus_pneumoniae.genome.name = 'GCF_001457635'
params.test_data.species.streptococcus_pneumoniae.genome.length = '2110968'
params.test_data.species.streptococcus_pneumoniae.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/streptococcus_pneumoniae/genome/GCF_001457635.fna.gz'
params.test_data.species.streptococcus_suis.genome.name = 'GCF_002285535'
params.test_data.species.streptococcus_suis.genome.length = '2066888'
params.test_data.species.streptococcus_suis.genome.fna_gz = 'https://raw.githubusercontent.com/bactopia/bactopia-tests/main/data/species/streptococcus_suis/genome/GCF_002285535.fna.gz'
params.publish_dir.annotate_genome = 'annotation'
params.publish_dir.antimicrobial_resistance = 'antimicrobial-resistance'
params.publish_dir.assemble_genome = 'assembly'
params.publish_dir.assembly_qc = 'assembly'
params.publish_dir.blast = 'blast'
params.publish_dir.call_variants = 'variants'
params.publish_dir.count_31mers = 'minmers'
params.publish_dir.mapping_query = 'mapping'
params.publish_dir.minmer_query = 'minmers'
params.publish_dir.minmer_sketch = 'minmers'
params.publish_dir.qc_reads = 'quality-control'
params.publish_dir.sequence_type = 'mlst'
params.publish_dir.gather_samples = ''
docker_repo = 'quay.io/bactopia'
singularity_cache = '/home/gilmansiu3/.bactopia/singularity'
infodir = './/bactopia/nf-reports'
timeline.enabled = true
timeline.overwrite = true
timeline.file = './/bactopia/nf-reports/bactopia-timeline.html'
report.enabled = true
report.overwrite = true
report.file = './/bactopia/nf-reports/bactopia-report.html'
trace.enabled = true
trace.overwrite = true
trace.file = './/bactopia/nf-reports/bactopia-trace.txt'
trace.fields = 'task_id,hash,native_id,process,tag,name,status,exit,module,container,cpus,time,disk,memory,attempt,start,complete,duration,realtime,queue,%cpu,%mem,rss,vmem'
dag.enabled = true
dag.overwrite = true
dag.file = './/bactopia/nf-reports/bactopia-dag.svg'
conda.enabled = true
conda.createOptions = '-c conda-forge -c bioconda'
conda.useMamba = false
conda.cacheDir = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/conda/envs'
process.cpus = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure2@40faff12
process.memory = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure3@734fbae3
process.time = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure4@223967ea
process.errorStrategy = 'retry'
process.maxRetries = 3
process.'withLabel:max_cpus_1'.cpus = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure5$_closure19@4a6a6a69
process.'withLabel:max_cpu_75'.cpus = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure6$_closure20@5d7f1e59
process.'withLabel:max_cpu_50'.cpus = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure7$_closure21@53e93fb7
process.'withLabel:base_mem_4gb'.memory = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure8$_closure22@3a861417
process.'withLabel:base_mem_8gb'.memory = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure9$_closure23@2326180c
process.'withLabel:assemble_genome'.memory = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure10$_closure24@68e47e7
process.'withLabel:assemble_genome'.time = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure10$_closure25@1c00d406
process.'withLabel:assemble_genome'.conda = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/conda/envs/assemble_genome-2.2.x'
process.'withLabel:gather_samples'.maxForks = 3
process.'withLabel:gather_samples'.maxRetries = 20
process.'withLabel:process_low'.cpus = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure12$_closure26@16ac4d3d
process.'withLabel:process_low'.memory = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure12$_closure27@559d19c
process.'withLabel:process_low'.time = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure12$_closure28@719c1faf
process.'withLabel:process_medium'.cpus = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure13$_closure29@6ba6557e
process.'withLabel:process_medium'.memory = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure13$_closure30@1f172892
process.'withLabel:process_medium'.time = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure13$_closure31@5409dfdd
process.'withLabel:process_high'.cpus = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure14$_closure32@45f9d394
process.'withLabel:process_high'.memory = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure14$_closure33@1c2d63f0
process.'withLabel:process_high'.time = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure14$_closure34@3a588b5f
process.'withLabel:process_long'.time = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure15$_closure35@20b54cfe
process.'withLabel:process_high_memory'.memory = ScriptE105F011BDD35A6D5531E17486D7CB4B$_run_closure1$_closure16$_closure36@2bdb5e0f
process.'withLabel:error_ignore'.errorStrategy = 'ignore'
process.'withLabel:error_retry'.errorStrategy = 'retry'
process.'withLabel:error_retry'.maxRetries = 2
process.'withLabel:annotate_genome|blast|antimicrobial_resistance|sequence_type'.conda = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/conda/envs/annotate_genome-2.2.x'
process.'withLabel:assembly_qc'.conda = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/conda/envs/assembly_qc-2.2.x'
process.'withLabel:call_variants|mapping_query'.conda = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/conda/envs/call_variants-2.2.x'
process.'withLabel:minmer_sketch|minmer_query'.conda = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/conda/envs/minmers-2.2.x'
process.'withLabel:gather_samples|qc_reads'.conda = '/home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/conda/envs/gather_samples-2.2.x'

2023-02-09 09:08:31:root:STDERR -
2023-02-09 09:08:31:root:INFO - Checking if environment pre-builds are needed
2023-02-09 09:08:31:root:DEBUG - Working on shigeifinder
2023-02-09 09:08:31:executor.process:DEBUG - Executing external command: bash -c 'docker inspect quay.io/biocontainers/shigeifinder:1.3.2--pyhdfd78af_0 || true'
2023-02-09 09:08:31:executor.process:DEBUG - Constructing subprocess.Popen object ..
2023-02-09 09:08:31:executor.process:DEBUG - Joining synchronous process using subprocess.Popen.communicate() ..
2023-02-09 09:08:31:executor.process:DEBUG - Got return code 0 from synchronous process (bash -c 'docker inspect quay.io/biocontainers/shigeifinder:1.3.2--pyhdfd78af_0 || true').
2023-02-09 09:08:31:root:STDOUT - [
{
"Id": "sha256:0c8d9971c1d41e5c584faf745abbd9ae4559e32cf0d9b820f4a60895a10fa377",
"RepoTags": [
"quay.io/biocontainers/shigeifinder:1.3.2--pyhdfd78af_0"
],
"RepoDigests": [
"quay.io/biocontainers/shigeifinder@sha256:9fdc17b500161739a1a84e01de72699c8b65600eb8388cfa9508331a0d4eace8"
],
"Parent": "",
"Comment": "Created with Involucro",
"Created": "2022-04-21T00:30:13.873052277Z",
"Container": "c85a5b53c83733e3689647b86ec33f1b827c235415cbeb68ed7d3db6f47a9139",
"ContainerConfig": {
"Hostname": "c85a5b53c837",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"LANG=C.UTF-8"
],
"Cmd": [
"/bin/sh"
],
"Image": "quay.io/bioconda/base-glibc-busybox-bash:2.1.0",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/env-execute"
],
"OnBuild": null,
"Labels": {
"busybox": "BusyBox v1.32.1 (2021-04-13 11:15:36 UTC) multi-call binary.",
"deb-list": "gcc-8-base_8.3.0-6_amd64.deb\nlibc6_2.28-10_amd64.deb\nlibgcc1_1%3a8.3.0-6_amd64.deb\nbash_5.0-4_amd64.deb\nlibc-bin_2.28-10_amd64.deb\nlibtinfo6_6.1+20181013-2+deb10u2_amd64.deb\nncurses-base_6.1+20181013-2+deb10u2_all.deb\nbase-files_10.3+deb10u9_amd64.deb",
"glibc": "GNU C Library (Debian GLIBC 2.28-10) stable release version 2.28.",
"io.buildah.version": "1.19.6",
"pkg-list": "gcc-8-base\nlibc6\nlibgcc1\nbash\nlibc-bin\nlibtinfo6\nncurses-base\nbase-files"
}
},
"DockerVersion": "20.10.14+azure-1",
"Author": "",
"Config": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"LANG=C.UTF-8"
],
"Cmd": [
"/bin/sh"
],
"Image": "",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/env-execute"
],
"OnBuild": null,
"Labels": {
"busybox": "BusyBox v1.32.1 (2021-04-13 11:15:36 UTC) multi-call binary.",
"deb-list": "gcc-8-base_8.3.0-6_amd64.deb\nlibc6_2.28-10_amd64.deb\nlibgcc1_1%3a8.3.0-6_amd64.deb\nbash_5.0-4_amd64.deb\nlibc-bin_2.28-10_amd64.deb\nlibtinfo6_6.1+20181013-2+deb10u2_amd64.deb\nncurses-base_6.1+20181013-2+deb10u2_all.deb\nbase-files_10.3+deb10u9_amd64.deb",
"glibc": "GNU C Library (Debian GLIBC 2.28-10) stable release version 2.28.",
"io.buildah.version": "1.19.6",
"pkg-list": "gcc-8-base\nlibc6\nlibgcc1\nbash\nlibc-bin\nlibtinfo6\nncurses-base\nbase-files"
}
},
"Architecture": "amd64",
"Os": "linux",
"Size": 388249219,
"VirtualSize": 388249219,
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/f82d869abba49d1c8a64329ab744a38ec27ad66edd521ab99032cd876c7ad1e3/diff:/var/lib/docker/overlay2/c9fea4dc3f811836e43cebaaeebb841b48e3cf550c8827b22803719a8b507120/diff",
"MergedDir": "/var/lib/docker/overlay2/2642b123fe09a895c935929c333e7f3fd00f8b6bf0acb7eae2285f665d6e3482/merged",
"UpperDir": "/var/lib/docker/overlay2/2642b123fe09a895c935929c333e7f3fd00f8b6bf0acb7eae2285f665d6e3482/diff",
"WorkDir": "/var/lib/docker/overlay2/2642b123fe09a895c935929c333e7f3fd00f8b6bf0acb7eae2285f665d6e3482/work"
},
"Name": "overlay2"
},
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:968f331861f0ab0535ba337d320dac4aaca7ee4b11048be407a4dbd4915004f5",
"sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef",
"sha256:d7a68540181de214202b6dd3d86c20eb7e35d8f0c295ebb52b84db9068356d26"
]
},
"Metadata": {
"LastTagTime": "0001-01-01T00:00:00Z"
}
}
]

2023-02-09 09:08:31:root:STDERR -
2023-02-09 09:08:31:root:DEBUG - Existing container (quay.io/biocontainers/shigeifinder:1.3.2--pyhdfd78af_0) found, skipping unless manually removed
2023-02-09 09:08:31:root:DEBUG - Found Docker container for quay.io/biocontainers/shigeifinder:1.3.2--pyhdfd78af_0, if a complete rebuild is needed please manually remove the containers
2023-02-09 09:08:31:root:DEBUG - Working on shigeifinder
2023-02-09 09:08:31:executor.process:DEBUG - Executing external command: bash -c 'docker inspect quay.io/biocontainers/multiqc:1.11--pyhdfd78af_0 || true'
2023-02-09 09:08:31:executor.process:DEBUG - Constructing subprocess.Popen object ..
2023-02-09 09:08:31:executor.process:DEBUG - Joining synchronous process using subprocess.Popen.communicate() ..
2023-02-09 09:08:31:executor.process:DEBUG - Got return code 0 from synchronous process (bash -c 'docker inspect quay.io/biocontainers/multiqc:1.11--pyhdfd78af_0 || true').
2023-02-09 09:08:31:root:STDOUT - [
{
"Id": "sha256:3e1158ef22f173c985ec7169bcf627af4bd626f24d2be0efc8b1509fbc1eac40",
"RepoTags": [
"quay.io/biocontainers/multiqc:1.11--pyhdfd78af_0"
],
"RepoDigests": [
"quay.io/biocontainers/multiqc@sha256:88df23fac5b9eecda9943d922f81b68e30188eb4dd7cbfe9554e952ff5a3b0ee"
],
"Parent": "",
"Comment": "Created with Involucro",
"Created": "2021-07-05T07:49:03.081083835Z",
"Container": "fc0de0abc7bc9688c98380fa3eb87fcc07e8736b9c4136cfd2b262ea26d46bf5",
"ContainerConfig": {
"Hostname": "fc0de0abc7bc",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"LANG=C.UTF-8"
],
"Cmd": [
"/bin/sh"
],
"Image": "quay.io/bioconda/base-glibc-debian-bash:2.1.0",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/env-execute"
],
"OnBuild": null,
"Labels": {
"debian": "10.9",
"glibc": "GNU C Library (Debian GLIBC 2.28-10) stable release version 2.28.",
"io.buildah.version": "1.19.6"
}
},
"DockerVersion": "17.09.0-ce",
"Author": "",
"Config": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"LANG=C.UTF-8"
],
"Cmd": [
"/bin/sh"
],
"Image": "",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/env-execute"
],
"OnBuild": null,
"Labels": {
"debian": "10.9",
"glibc": "GNU C Library (Debian GLIBC 2.28-10) stable release version 2.28.",
"io.buildah.version": "1.19.6"
}
},
"Architecture": "amd64",
"Os": "linux",
"Size": 429811474,
"VirtualSize": 429811474,
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/442fae63e8a896521f7a201580da6375588a0f688c60492cea4c57e751ee4762/diff:/var/lib/docker/overlay2/45e4f9101d06fd24d63aa3bd282f485cfd47e2f796872860cb56b96ba1770bde/diff:/var/lib/docker/overlay2/b3aae48e3d58878ccb41693c5bd60d591ee454b1b20ed96a083e1ca1df2caa0b/diff:/var/lib/docker/overlay2/c17b8b7ca0e9be2f6e351f92b49c2d9d8b7d38848b4308becf3bc2e835acff16/diff:/var/lib/docker/overlay2/846b344adc789768a098d920fdac0d5fcdcd7351c9a21acc782149cc6d17f70f/diff",
"MergedDir": "/var/lib/docker/overlay2/c69c792eccf29fb9d29585aaa2fdd117ed3ce7bf554ddc4c54936d303b2d90a3/merged",
"UpperDir": "/var/lib/docker/overlay2/c69c792eccf29fb9d29585aaa2fdd117ed3ce7bf554ddc4c54936d303b2d90a3/diff",
"WorkDir": "/var/lib/docker/overlay2/c69c792eccf29fb9d29585aaa2fdd117ed3ce7bf554ddc4c54936d303b2d90a3/work"
},
"Name": "overlay2"
},
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:7e718b9c0c8c2e6420fe9c4d1d551088e314fe923dce4b2caf75891d82fb227d",
"sha256:4747c2394004fcaeee513b3585049555b5dc6e179be570116694aa15297d8242",
"sha256:ab614ec5cb98907a97af4b2747e99eb4c27432b5a77b932bef9d105c35ae92a7",
"sha256:898a1347bf3f01f21b2bbfddbf450aef5fe7aed72626973d46c972f0487279d1",
"sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef",
"sha256:5e252093c4e58f7edb4fb7081d0c4e9052b5f5a4b65321d9d3da523bd4031b5a"
]
},
"Metadata": {
"LastTagTime": "0001-01-01T00:00:00Z"
}
}
]

2023-02-09 09:08:31:root:STDERR -
2023-02-09 09:08:31:root:DEBUG - Existing container (quay.io/biocontainers/multiqc:1.11--pyhdfd78af_0) found, skipping unless manually removed
2023-02-09 09:08:31:root:DEBUG - Found Docker container for quay.io/biocontainers/multiqc:1.11--pyhdfd78af_0, if a complete rebuild is needed please manually remove the containers
2023-02-09 09:08:31:root:DEBUG - Working on shigeifinder
2023-02-09 09:08:31:executor.process:DEBUG - Executing external command: bash -c 'docker inspect quay.io/biocontainers/csvtk:0.25.0--h9ee0642_0 || true'
2023-02-09 09:08:31:executor.process:DEBUG - Constructing subprocess.Popen object ..
2023-02-09 09:08:31:executor.process:DEBUG - Joining synchronous process using subprocess.Popen.communicate() ..
2023-02-09 09:08:31:executor.process:DEBUG - Got return code 0 from synchronous process (bash -c 'docker inspect quay.io/biocontainers/csvtk:0.25.0--h9ee0642_0 || true').
2023-02-09 09:08:31:root:STDOUT - [
{
"Id": "sha256:c99e6a11ac8a63ed7ed7ff541024095cb2610a18aaf80c749d218d57c907033c",
"RepoTags": [
"quay.io/biocontainers/csvtk:0.25.0--h9ee0642_0"
],
"RepoDigests": [
"quay.io/biocontainers/csvtk@sha256:f07592f60d3749bfb3df81a23f57d5e709cda30b3c818ac0be699e586bff01b2"
],
"Parent": "",
"Comment": "Created with Involucro",
"Created": "2022-09-22T09:56:46.995652569Z",
"Container": "5eb57ccd9f263fcd30a7f13f86655b82e9f0c63e88585c2f6e9c8f941326d09c",
"ContainerConfig": {
"Hostname": "5eb57ccd9f26",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"LANG=C.UTF-8"
],
"Cmd": [
"/bin/sh"
],
"Image": "quay.io/bioconda/base-glibc-busybox-bash:2.1.0",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/env-execute"
],
"OnBuild": null,
"Labels": {
"busybox": "BusyBox v1.32.1 (2021-04-13 11:15:36 UTC) multi-call binary.",
"deb-list": "gcc-8-base_8.3.0-6_amd64.deb\nlibc6_2.28-10_amd64.deb\nlibgcc1_1%3a8.3.0-6_amd64.deb\nbash_5.0-4_amd64.deb\nlibc-bin_2.28-10_amd64.deb\nlibtinfo6_6.1+20181013-2+deb10u2_amd64.deb\nncurses-base_6.1+20181013-2+deb10u2_all.deb\nbase-files_10.3+deb10u9_amd64.deb",
"glibc": "GNU C Library (Debian GLIBC 2.28-10) stable release version 2.28.",
"io.buildah.version": "1.19.6",
"pkg-list": "gcc-8-base\nlibc6\nlibgcc1\nbash\nlibc-bin\nlibtinfo6\nncurses-base\nbase-files"
}
},
"DockerVersion": "20.10.17+azure-3",
"Author": "",
"Config": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"LANG=C.UTF-8"
],
"Cmd": [
"/bin/sh"
],
"Image": "",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/env-execute"
],
"OnBuild": null,
"Labels": {
"busybox": "BusyBox v1.32.1 (2021-04-13 11:15:36 UTC) multi-call binary.",
"deb-list": "gcc-8-base_8.3.0-6_amd64.deb\nlibc6_2.28-10_amd64.deb\nlibgcc1_1%3a8.3.0-6_amd64.deb\nbash_5.0-4_amd64.deb\nlibc-bin_2.28-10_amd64.deb\nlibtinfo6_6.1+20181013-2+deb10u2_amd64.deb\nncurses-base_6.1+20181013-2+deb10u2_all.deb\nbase-files_10.3+deb10u9_amd64.deb",
"glibc": "GNU C Library (Debian GLIBC 2.28-10) stable release version 2.28.",
"io.buildah.version": "1.19.6",
"pkg-list": "gcc-8-base\nlibc6\nlibgcc1\nbash\nlibc-bin\nlibtinfo6\nncurses-base\nbase-files"
}
},
"Architecture": "amd64",
"Os": "linux",
"Size": 28937321,
"VirtualSize": 28937321,
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/f82d869abba49d1c8a64329ab744a38ec27ad66edd521ab99032cd876c7ad1e3/diff:/var/lib/docker/overlay2/c9fea4dc3f811836e43cebaaeebb841b48e3cf550c8827b22803719a8b507120/diff",
"MergedDir": "/var/lib/docker/overlay2/5d20c9eb16af23375faccdf141a83304b8cc7515b09ebf5bd7116c875b3b681d/merged",
"UpperDir": "/var/lib/docker/overlay2/5d20c9eb16af23375faccdf141a83304b8cc7515b09ebf5bd7116c875b3b681d/diff",
"WorkDir": "/var/lib/docker/overlay2/5d20c9eb16af23375faccdf141a83304b8cc7515b09ebf5bd7116c875b3b681d/work"
},
"Name": "overlay2"
},
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:968f331861f0ab0535ba337d320dac4aaca7ee4b11048be407a4dbd4915004f5",
"sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef",
"sha256:eafd6e62113d590d64ede1311faac7886f88a6c9e8dfc777171f67365f8917f9"
]
},
"Metadata": {
"LastTagTime": "0001-01-01T00:00:00Z"
}
}
]

2023-02-09 09:08:31:root:STDERR -
2023-02-09 09:08:31:root:DEBUG - Existing container (quay.io/biocontainers/csvtk:0.25.0--h9ee0642_0) found, skipping unless manually removed
2023-02-09 09:08:31:root:DEBUG - Found Docker container for quay.io/biocontainers/csvtk:0.25.0--h9ee0642_0, if a complete rebuild is needed please manually remove the containers

@rpetit3
Copy link
Member

rpetit3 commented Feb 9, 2023

Ok, let's take out the --verbose, and try replacing bactopia with bash -x $(which bactopia)

@llk578496
Copy link
Author

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ bash -x $(which bactopia) --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir

  • VERSION=2.2.0
  • CONTAINER_VERSION=2.2.x
    ++ which bactopia
    ++ sed s=bin/bactopia==
  • CONDA_ENV=/home/gilmansiu3/miniconda3/envs/bactopia/
  • BACTOPIA_NF=/home/gilmansiu3/miniconda3/envs/bactopia//share/bactopia-2.2.x
  • [[ 7 == 0 ]]
  • [[ --wf == \d\a\t\a\s\e\t\s ]]
  • [[ --wf == \p\r\e\p\a\r\e ]]
  • [[ --wf == \s\e\a\r\c\h ]]
  • [[ --wf == \d\o\w\n\l\o\a\d ]]
  • [[ --wf == \c\i\t\a\t\i\o\n ]]
  • [[ --wf == --\c\i\t\a\t\i\o\n ]]
  • [[ --wf == \c\i\t\a\t\i\o\n\s ]]
  • [[ --wf == \v\e\r\s\i\o\n ]]
  • [[ --wf == --\v\e\r\s\i\o\n ]]
  • PREBUILD=1
  • BUILD_TYPE=conda
  • [[ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir == \s\i\n\g\u\l\a\r\i\t\y ]]
  • [[ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir == \s\l\u\r\m ]]
  • [[ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir == \s\g\e ]]
  • [[ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir == \a\r\c\c ]]
  • [[ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir == \d\o\c\k\e\r ]]
  • BUILD_TYPE=docker
  • CHECKS=("help" "help_all" "nfconfig" "list_wfs")
  • for check in "${CHECKS[@]}"
  • [[ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir == \h\e\l\p ]]
  • for check in "${CHECKS[@]}"
  • [[ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir == \h\e\l\p_\a\l\l ]]
  • for check in "${CHECKS[@]}"
  • [[ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir == \n\f\c\o\n\f\i\g ]]
  • for check in "${CHECKS[@]}"
  • [[ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir == \l\i\s\t_\w\f\s ]]
  • [[ 1 -eq 1 ]]
  • BUILD_OPTS=
  • [[ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir == \a\r\c\c ]]
  • bactopia-download.py /home/gilmansiu3/miniconda3/envs/bactopia//share/bactopia-2.2.x --envtype docker --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir
    2023-02-09 09:23:25:root:INFO - Checking if environment pre-builds are needed, use --verbose to see full details.
  • WORK_DIR=
  • WORK_ARG=
  • CAN_CLEAN_UP=1
  • echo '--wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir'
  • sed -r 's/ ([-]+)/\n\1/g'
  • grep '^-w |-work-dir'
    ++ pwd
  • WORK_DIR=/mnt/data2/Food/REAL/work/
  • WORK_ARG='-w /mnt/data2/Food/REAL/work/'
  • OPTS='-w /mnt/data2/Food/REAL/work/'
  • nextflow run /home/gilmansiu3/miniconda3/envs/bactopia//share/bactopia-2.2.x/main.nf -w /mnt/data2/Food/REAL/work/ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir

@rpetit3
Copy link
Member

rpetit3 commented Feb 9, 2023

We're making progress, let's try manually running

nextflow run /home/gilmansiu3/miniconda3/envs/bactopia//share/bactopia-2.2.x/main.nf \
    -w /mnt/data2/Food/REAL/work/ \
    --wf shigeifinder \
    --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 \
    -profile docker \
    --cleanup_workdir

@llk578496
Copy link
Author

llk578496 commented Feb 9, 2023

Seems it stops here.

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ nextflow run /home/gilmansiu3/miniconda3/envs/bactopia//share/bactopia-2.2.x/main.nf
-w /mnt/data2/Food/REAL/work/
--wf shigeifinder
--bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673
-profile docker
--cleanup_workdir

@rpetit3
Copy link
Member

rpetit3 commented Feb 9, 2023

What do you get with:

nextflow -v
java --version
nextflow run -h

@llk578496
Copy link
Author

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ nextflow -v
nextflow version 22.10.4.5836
(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ java --version
openjdk 17.0.3-internal 2022-04-19
OpenJDK Runtime Environment (build 17.0.3-internal+0-adhoc..src)
OpenJDK 64-Bit Server VM (build 17.0.3-internal+0-adhoc..src, mixed mode, sharing)
(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ nextflow run -h
Execute a pipeline project
Usage: run [options] Project name or repository url
Options:
-E
Exports all current system environment
Default: false
-ansi-log
Enable/disable ANSI console logging
-bucket-dir
Remote bucket where intermediate result files are stored
-cache
Enable/disable processes caching
-disable-jobs-cancellation
Prevent the cancellation of child jobs on execution termination
-dsl1
Execute the workflow using DSL1 syntax
Default: false
-dsl2
Execute the workflow using DSL2 syntax
Default: false
-dump-channels
Dump channels for debugging purpose
-dump-hashes
Dump task hash keys for debugging purpose
Default: false
-e.
Add the specified variable to execution environment
Syntax: -e.key=value
Default: {}
-entry
Entry workflow name to be executed
-h, -help
Print the command usage
Default: false
-hub
Service hub where the project is hosted
-latest
Pull latest changes before run
Default: false
-lib
Library extension path
-main-script
The script file to be executed when launching a project directory or
repository
-name
Assign a mnemonic name to the a pipeline run
-offline
Do not check for remote project updates
Default: false
-params-file
Load script parameters from a JSON/YAML file
-plugins
Specify the plugins to be applied for this run e.g. nf-amazon,nf-tower
-preview
Run the workflow script skipping the execution of all processes
Default: false
-process.
Set process options
Syntax: -process.key=value
Default: {}
-profile
Choose a configuration profile
-qs, -queue-size
Max number of processes that can be executed in parallel by each executor
-resume
Execute the script using the cached results, useful to continue
executions that was stopped by an error
-r, -revision
Revision of the project to run (either a git branch, tag or commit SHA
number)
-stub-run, -stub
Execute the workflow replacing process scripts with command stubs
Default: false
-test
Test a script function with the name specified
-user
Private repository user name
-with-charliecloud
Enable process execution in a Charliecloud container runtime
-with-conda
Use the specified Conda environment package or file (must end with
.yml|.yaml suffix)
-with-dag
Create pipeline DAG file
-with-docker
Enable process execution in a Docker container
-N, -with-notification
Send a notification email on workflow completion to the specified
recipients
-with-podman
Enable process execution in a Podman container
-with-report
Create processes execution html report
-with-singularity
Enable process execution in a Singularity container
-with-timeline
Create processes execution timeline file
-with-tower
Monitor workflow execution with Seqera Tower service
-with-trace
Create processes execution tracing file
-with-weblog
Send workflow status messages via HTTP to target URL
-without-conda
Disable the use of Conda environments
-without-docker
Disable process execution with Docker
Default: false
-without-podman
Disable process execution in a Podman container
-w, -work-dir
Directory where intermediate result files are stored

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$

@rpetit3
Copy link
Member

rpetit3 commented Feb 9, 2023

Are there any .nextflow.log files created?

@llk578496
Copy link
Author

Yes.

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ cat .nextflow.log
Feb-09 09:54:54.492 [main] DEBUG nextflow.cli.Launcher - $> nextflow run -h
(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$

@rpetit3
Copy link
Member

rpetit3 commented Feb 9, 2023

Let's try this one again so we can get a fresh .nextflow.log

nextflow run /home/gilmansiu3/miniconda3/envs/bactopia//share/bactopia-2.2.x/main.nf \
    -w /mnt/data2/Food/REAL/work/ \
    --wf shigeifinder \
    --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 \
    -profile docker \
    --cleanup_workdir

@llk578496
Copy link
Author

llk578496 commented Feb 9, 2023

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ nextflow run /home/gilmansiu3/miniconda3/envs/bactopia//share/bactopia-2.2.x/main.nf
-w /mnt/data2/Food/REAL/work/
--wf shigeifinder
--bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673
-profile docker
--cleanup_workdir
^C(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ cat .nextflow.log
Feb-09 10:22:52.736 [main] DEBUG nextflow.cli.Launcher - $> nextflow run /home/gilmansiu3/miniconda3/envs/bactopia//share/bactopia-2.2.x/main.nf -w /mnt/data2/Food/REAL/work/ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir
(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$

@rpetit3
Copy link
Member

rpetit3 commented Feb 9, 2023

Thta's weird, nothing is happening.

Doesthis give you anything?

bactopia -profile test,docker

@llk578496
Copy link
Author

It stucks also.

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ bactopia -profile test,docker
2023-02-09 13:27:30:root:INFO - Checking if environment pre-builds are needed, use --verbose to see full details.

@rpetit3
Copy link
Member

rpetit3 commented Feb 9, 2023

Ok, let's try this:

nextflow run nf-core/rnaseq -profile test,docker --outdir test

Trying another pipeline to see if its a nextflow thing. I'm going to try replicating on my end with the versions of Java and Nextflow that you have.

Another option might be to create a new bactopia environment

@llk578496
Copy link
Author

llk578496 commented Feb 9, 2023

Seems it works.

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/test$ nextflow run nf-core/rnaseq -profile test,docker --outdir test
N E X T F L O W ~ version 22.10.4
Pulling nf-core/rnaseq ...
downloaded from https://github.com/nf-core/rnaseq.git
Launching https://github.com/nf-core/rnaseq [adoring_shannon] DSL2 - revision: 6e1e448f53 [master]


                                    ,--./,-.
    ___     __   __   __   ___     /,-._.--~'

|\ | |__ __ / / \ |__) |__ } { | \| | \__, \__/ | \ |___ \-.,--, .,._,'
nf-core/rnaseq v3.10.1-g6e1e448

Core Nextflow options
revision : master
runName : adoring_shannon
containerEngine : docker
launchDir : /mnt/data2/test
workDir : /mnt/data2/test/work
projectDir : /home/gilmansiu3/.nextflow/assets/nf-core/rnaseq
userName : gilmansiu3
profile : test,docker
configFiles : /home/gilmansiu3/.nextflow/assets/nf-core/rnaseq/nextflow.config

Input/output options
input : https://raw.githubusercontent.com/nf-core/test-datasets/rnaseq/samplesheet/v3.10/samplesheet_test.csv
outdir : test

UMI options
umitools_bc_pattern : NNNN

Read filtering options
bbsplit_fasta_list : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/bbsplit_fasta_list.txt
skip_bbsplit : false

Reference genome options
fasta : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genome.fasta
gtf : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genes.gtf.gz
gff : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genes.gff.gz
transcript_fasta : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/transcriptome.fasta
additional_fasta : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/gfp.fa.gz
hisat2_index : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/hisat2.tar.gz
rsem_index : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/rsem.tar.gz
salmon_index : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/salmon.tar.gz

Alignment options
pseudo_aligner : salmon

Institutional config options
config_profile_name : Test profile
config_profile_description: Minimal test dataset to check pipeline function

Max job request options
max_cpus : 2
max_memory : 6.GB
max_time : 6.h

!! Only displaying parameters that differ from the pipeline defaults !!

If you use nf-core/rnaseq for your analysis please cite:


WARN: =============================================================================
Both '--gtf' and '--gff' parameters have been provided.
Using GTF file as priority.

WARN: =============================================================================
'--transcript_fasta' parameter has been provided.
Make sure transcript names in this file match those in the GFF/GTF file.

Please see:
nf-core/rnaseq#753

executor > local (197)
[df/caf740] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:GUNZIP_GTF (genes.gtf.gz) [100%] 1 of 1 ✔
[08/a8723d] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:GUNZIP_ADDITIONAL_FASTA (gfp.fa.gz) [100%] 1 of 1 ✔
[e2/45826f] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:CAT_ADDITIONAL_FASTA (gfp.fa) [100%] 1 of 1 ✔
[21/2c761a] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:GTF2BED (genome_gfp.gtf) [100%] 1 of 1 ✔
[79/ba8ce4] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:CUSTOM_GETCHROMSIZES (genome_gfp.fasta) [100%] 1 of 1 ✔
[72/f8b804] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:BBMAP_BBSPLIT [100%] 1 of 1 ✔
[d0/41ee40] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:STAR_GENOMEGENERATE (genome_gfp.fasta) [100%] 1 of 1 ✔
[e6/c0f37e] process > NFCORE_RNASEQ:RNASEQ:PREPARE_GENOME:UNTAR_SALMON_INDEX (salmon.tar.gz) [100%] 1 of 1 ✔
[74/17ae9a] process > NFCORE_RNASEQ:RNASEQ:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet_test.csv) [100%] 1 of 1 ✔
[4d/455556] process > NFCORE_RNASEQ:RNASEQ:CAT_FASTQ (WT_REP1) [100%] 2 of 2 ✔
[19/20f348] process > NFCORE_RNASEQ:RNASEQ:FASTQ_SUBSAMPLE_FQ_SALMON:FQ_SUBSAMPLE (WT_REP1) [100%] 2 of 2 ✔
[18/8eb35e] process > NFCORE_RNASEQ:RNASEQ:FASTQ_SUBSAMPLE_FQ_SALMON:SALMON_QUANT (RAP1_UNINDUCED_REP1) [100%] 2 of 2 ✔
[a7/9b2706] process > NFCORE_RNASEQ:RNASEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:FASTQC (WT_REP1) [100%] 5 of 5 ✔
[b1/de8dfe] process > NFCORE_RNASEQ:RNASEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (WT_REP1) [100%] 5 of 5 ✔
[- ] process > NFCORE_RNASEQ:RNASEQ:MULTIQC_TSV_FAIL_TRIMMED -
[c1/6cb30c] process > NFCORE_RNASEQ:RNASEQ:BBMAP_BBSPLIT (5) [100%] 5 of 5 ✔
[9f/0e7c2b] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:STAR_ALIGN (WT_REP1) [100%] 5 of 5 ✔
[73/5b8dfa] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:SAMTOOLS_SORT (WT_REP1) [100%] 5 of 5 ✔
[25/915489] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:SAMTOOLS_INDEX (WT_REP1) [100%] 5 of 5 ✔
[54/9c2739] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_STATS (WT_REP2) [100%] 5 of 5 ✔
[7e/f0cd5a] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_FLAGSTAT (RAP1_UNINDUCED_REP2) [100%] 5 of 5 ✔
[b5/04cb6e] process > NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:BAM_SORT_STATS_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_IDXSTATS (WT_REP2) [100%] 5 of 5 ✔
[1b/76260e] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_QUANT (WT_REP1) [100%] 5 of 5 ✔
[93/8e7c68] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_TX2GENE (genome_gfp.gtf) [100%] 1 of 1 ✔
[cd/fc0dcd] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_TXIMPORT [100%] 1 of 1 ✔
[ae/146548] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_SE_GENE (salmon_tx2gene.tsv) [100%] 1 of 1 ✔
[61/81d1b6] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_SE_GENE_LENGTH_SCALED (salmon_tx2gene.tsv) [100%] 1 of 1 ✔
[79/8b4497] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_SE_GENE_SCALED (salmon_tx2gene.tsv) [100%] 1 of 1 ✔
[14/8e25e0] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_STAR_SALMON:SALMON_SE_TRANSCRIPT (salmon_tx2gene.tsv) [100%] 1 of 1 ✔
[65/bcce42] process > NFCORE_RNASEQ:RNASEQ:DESEQ2_QC_STAR_SALMON [100%] 1 of 1 ✔
[- ] process > NFCORE_RNASEQ:RNASEQ:MULTIQC_TSV_FAIL_MAPPED -
[a9/a2f7f8] process > NFCORE_RNASEQ:RNASEQ:BAM_MARKDUPLICATES_PICARD:PICARD_MARKDUPLICATES (WT_REP1) [100%] 5 of 5 ✔
[98/108e70] process > NFCORE_RNASEQ:RNASEQ:BAM_MARKDUPLICATES_PICARD:SAMTOOLS_INDEX (WT_REP1) [100%] 5 of 5 ✔
[67/9905ec] process > NFCORE_RNASEQ:RNASEQ:BAM_MARKDUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_STATS (WT_REP2) [100%] 5 of 5 ✔
[60/9c6a15] process > NFCORE_RNASEQ:RNASEQ:BAM_MARKDUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_FLAGSTAT (WT_REP2) [100%] 5 of 5 ✔
[62/7e9487] process > NFCORE_RNASEQ:RNASEQ:BAM_MARKDUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_IDXSTATS (RAP1_UNINDUCED_REP2) [100%] 5 of 5 ✔
[2a/12dd12] process > NFCORE_RNASEQ:RNASEQ:STRINGTIE_STRINGTIE (WT_REP1) [100%] 5 of 5 ✔
[ca/c609f1] process > NFCORE_RNASEQ:RNASEQ:SUBREAD_FEATURECOUNTS (WT_REP1) [100%] 5 of 5 ✔
[61/fc4187] process > NFCORE_RNASEQ:RNASEQ:MULTIQC_CUSTOM_BIOTYPE (WT_REP1) [100%] 5 of 5 ✔
[bb/6986f7] process > NFCORE_RNASEQ:RNASEQ:BEDTOOLS_GENOMECOV (WT_REP1) [100%] 5 of 5 ✔
[7f/3484e6] process > NFCORE_RNASEQ:RNASEQ:BEDGRAPH_BEDCLIP_BEDGRAPHTOBIGWIG_FORWARD:UCSC_BEDCLIP (WT_REP1) [100%] 5 of 5 ✔
[4a/41d448] process > NFCORE_RNASEQ:RNASEQ:BEDGRAPH_BEDCLIP_BEDGRAPHTOBIGWIG_FORWARD:UCSC_BEDGRAPHTOBIGWIG (WT_REP1) [100%] 5 of 5 ✔
[26/3a6ed7] process > NFCORE_RNASEQ:RNASEQ:BEDGRAPH_BEDCLIP_BEDGRAPHTOBIGWIG_REVERSE:UCSC_BEDCLIP (WT_REP1) [100%] 5 of 5 ✔
[d5/6d98f0] process > NFCORE_RNASEQ:RNASEQ:BEDGRAPH_BEDCLIP_BEDGRAPHTOBIGWIG_REVERSE:UCSC_BEDGRAPHTOBIGWIG (WT_REP1) [100%] 5 of 5 ✔
[6b/4290bd] process > NFCORE_RNASEQ:RNASEQ:QUALIMAP_RNASEQ (WT_REP1) [100%] 5 of 5 ✔
[60/56a1c9] process > NFCORE_RNASEQ:RNASEQ:DUPRADAR (WT_REP1) [100%] 5 of 5 ✔
[31/0d1406] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_BAMSTAT (WT_REP1) [100%] 5 of 5 ✔
[90/d946f8] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_INNERDISTANCE (WT_REP1) [100%] 5 of 5 ✔
[51/dfd48f] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_INFEREXPERIMENT (WT_REP1) [100%] 5 of 5 ✔
[3b/f01315] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_JUNCTIONANNOTATION (WT_REP1) [100%] 5 of 5 ✔
[5d/9f3214] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_JUNCTIONSATURATION (WT_REP1) [100%] 5 of 5 ✔
[bc/74564d] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_READDISTRIBUTION (WT_REP1) [100%] 5 of 5 ✔
[46/8704c8] process > NFCORE_RNASEQ:RNASEQ:BAM_RSEQC:RSEQC_READDUPLICATION (WT_REP1) [100%] 5 of 5 ✔
[a6/708bc0] process > NFCORE_RNASEQ:RNASEQ:MULTIQC_TSV_STRAND_CHECK [100%] 1 of 1 ✔
[7b/a5a075] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_SALMON:SALMON_QUANT (WT_REP1) [100%] 5 of 5 ✔
[04/7bd02c] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_SALMON:SALMON_TX2GENE (genome_gfp.gtf) [100%] 1 of 1 ✔
[cc/b01ce1] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_SALMON:SALMON_TXIMPORT [100%] 1 of 1 ✔
[19/0ce761] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_SALMON:SALMON_SE_GENE (salmon_tx2gene.tsv) [100%] 1 of 1 ✔
[1d/8811a0] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_SALMON:SALMON_SE_GENE_LENGTH_SCALED (salmon_tx2gene.tsv) [100%] 1 of 1 ✔
[c6/0d0b2b] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_SALMON:SALMON_SE_GENE_SCALED (salmon_tx2gene.tsv) [100%] 1 of 1 ✔
[c5/630dff] process > NFCORE_RNASEQ:RNASEQ:QUANTIFY_SALMON:SALMON_SE_TRANSCRIPT (salmon_tx2gene.tsv) [100%] 1 of 1 ✔
[06/afd565] process > NFCORE_RNASEQ:RNASEQ:DESEQ2_QC_SALMON [100%] 1 of 1 ✔
[52/789585] process > NFCORE_RNASEQ:RNASEQ:CUSTOM_DUMPSOFTWAREVERSIONS (1) [100%] 1 of 1 ✔
[77/fe48d5] process > NFCORE_RNASEQ:RNASEQ:MULTIQC (1) [100%] 1 of 1 ✔
-[nf-core/rnaseq] 5/5 samples passed STAR 5% mapped threshold:
95.96%: RAP1_UNINDUCED_REP1
95.85%: RAP1_UNINDUCED_REP2
90.44%: RAP1_IAA_30M_REP1
89.54%: WT_REP2
88.99%: WT_REP1

-[nf-core/rnaseq] Pipeline completed successfully-
Completed at: 09-Feb-2023 22:52:42
Duration : 5m 10s
CPU hours : 0.4
Succeeded : 197

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/test$

@rpetit3
Copy link
Member

rpetit3 commented Feb 9, 2023

Good to know. I'm going to try your versions of Nextflow/Java today to see if I can replicate.

In the meantime, since it was working previously. Maybe try building a new Bactopia environment?

@llk578496
Copy link
Author

llk578496 commented Feb 9, 2023

Interestingly, I just created a new directory /mnt/data2/Food/REAL/test/test-01 and tried to run bactopia -profile test,docker & bactopia --wf shigeifinder --bactopia /mnt/data2/Food/REAL/test/test-01/ -profile docker --cleanup_workdir, and both completed successfully.

But when I go back to the original directory /mnt/data2/Food/REAL/ and run bactopia -profile test,docker, it hangs again.

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ mkdir -p test/test-01
(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ cd test/test-01/
(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL/test/test-01$ bactopia -profile test,docker
2023-02-09 23:35:07:root:INFO - Checking if environment pre-builds are needed, use --verbose to see full details.
N E X T F L O W ~ version 22.10.4
Launching /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/main.nf [cranky_magritte] DSL2 - revision: 145bb11899



| |__ __ _ | | ___ _ __ () __ _
| '
\ / |/ __| __/ _ \| '_ \| |/ _ |
| |
) | (| | (__| || () | |) | | (| |
|.__/ _,|___|__/| ./||_,|
|
|
bactopia v2.2.0
Bactopia is a flexible pipeline for complete analysis of bacterial genomes.

Core Nextflow options
runName : cranky_magritte
containerEngine : docker
container : quay.io/bactopia/bactopia:2.2.0
launchDir : /mnt/data2/Food/REAL/test/test-01
workDir : /mnt/data2/Food/REAL/test/test-01/work
projectDir : /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x
userName : gilmansiu3
profile : test,docker
configFiles : /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/nextflow.config

Required Parameters
R1 : https://github.com/bactopia/bactopia-tests/raw/main/data/species/portiera/illumina/SRR2838702_R1.fastq.gz
R2 : https://github.com/bactopia/bactopia-tests/raw/main/data/species/portiera/illumina/SRR2838702_R2.fastq.gz
sample : SRR2838702

Dataset Parameters
genome_size : 358242

Max Job Request Parameters
max_cpus : 2
max_memory : 6

Nextflow Profile Parameters
condadir : /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/conda/envs
registry : quay
singularity_cache: /home/gilmansiu3/.bactopia/singularity

!! Only displaying parameters that differ from the pipeline defaults !!

If you use bactopia for your analysis please cite:


--datasets not given, skipping the following processes (analyses):
antimicrobial resistance
ariba_analysis
blast (genes, proteins, primers)
call_variants
mapping_query
minmer_query
sequence_type
Will use 358242 bp for genome size

If something looks wrong, now's your chance to back out (CTRL+C 3 times).
Sleeping for 5 seconds...

executor > local (7)
[55/14eca5] process > BACTOPIA:GATHER_SAMPLES (SRR2838702) [100%] 1 of 1 ✔
[8c/1a5c6a] process > BACTOPIA:QC_READS (SRR2838702) [100%] 1 of 1 ✔
[8a/41c744] process > BACTOPIA:ASSEMBLE_GENOME (SRR2838702) [100%] 1 of 1 ✔
[d2/14bbf1] process > BACTOPIA:ASSEMBLY_QC (SRR2838702) [100%] 1 of 1 ✔
[35/910c3e] process > BACTOPIA:ANNOTATE_GENOME (SRR2838702) [100%] 1 of 1 ✔
[8d/a3d1c1] process > BACTOPIA:MINMER_SKETCH (SRR2838702) [100%] 1 of 1 ✔
[- ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE -
[- ] process > BACTOPIA:MINMER_QUERY -
[- ] process > BACTOPIA:BLAST -
[- ] process > BACTOPIA:CALL_VARIANTS -
[- ] process > BACTOPIA:MAPPING_QUERY -
[- ] process > BACTOPIA:SEQUENCE_TYPE -
[25/1fbe14] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS (1) [100%] 1 of 1 ✔

Bactopia Execution Summary
---------------------------
Bactopia Version : 2.2.0
Nextflow Version : 22.10.4
Command Line     : nextflow run /home/gilmansiu3/miniconda3/envs/bactopia//share/bactopia-2.2.x/main.nf -w /mnt/data2/Food/REAL/test/test-01/work/ -profile test,docker
Resumed          : false
Completed At     : 2023-02-09T23:37:14.352640220+08:00
Duration         : 2m 4s
Success          : true
Exit Code        : 0
Error Report     : -
Launch Dir       : /mnt/data2/Food/REAL/test/test-01

Completed at: 09-Feb-2023 23:37:14
Duration : 2m 4s
CPU hours : 0.1
Succeeded : 7

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL/test/test-01$ bactopia --wf shigeifinder --bactopia /mnt/data2/Food/REAL/test/test-01 -profile docker --cleanup_workdir
2023-02-10 00:23:27:root:INFO - Checking if environment pre-builds are needed, use --verbose to see full details.
N E X T F L O W ~ version 22.10.4
Launching /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/main.nf [distraught_visvesvaraya] DSL2 - revision: 145bb11899



| |__ __ _ | | ___ _ __ () __ _ | | ___ ___ | |___
| '_ \ / |/ __| __/ _ \| '_ \| |/ _ | | __/ _ \ / _ | / __|
| |
) | (| | (__| || () | |) | | (| | | || () | () | __ \
|.__/ _,|___|__/| ./||_,| __/ _/||_/
|
|
bactopia tools shigeifinder v2.2.0
Shigella and EIEC serotyping from assemblies

Core Nextflow options
runName : distraught_visvesvaraya
containerEngine : docker
container : quay.io/bactopia/bactopia:2.2.0
launchDir : /mnt/data2/Food/REAL/test/test-01
workDir : /mnt/data2/Food/REAL/test/test-01/work
projectDir : /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x
userName : gilmansiu3
profile : docker
configFiles : /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/nextflow.config

Required Parameters
bactopia : /mnt/data2/Food/REAL/test/test-01

Optional Parameters
outdir : /mnt/data2/Food/REAL/test/test-01
run_name : shigeifinder

Nextflow Configuration Parameters
cleanup_workdir : true

Nextflow Profile Parameters
condadir : /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/conda/envs
registry : quay
singularity_cache: /home/gilmansiu3/.bactopia/singularity

Helpful Parameters
wf : shigeifinder

!! Only displaying parameters that differ from the pipeline defaults !!

If you use bactopia for your analysis please cite:


Each task will use 4 CPUs out of the available 20 CPUs. At most
5 task(s) will be run at a time, this can affect the efficiency
of Bactopia. You can use the '-qs' parameter to alter the number of
tasks to run at a time (e.g. '-qs 2', means only 2 tasks or a maximum
of 8 CPUs will be used at once)

logs does not appear to be a Bactopia sample, skipping...
bactopia does not appear to be a Bactopia sample, skipping...
Found 1 samples to process

If this looks wrong, now's your chance to back out (CTRL+C 3 times).
Sleeping for 5 seconds...

executor > local (3)
executor > local (3)
[72/c17f57] process > BACTOPIATOOLS:SHIGEIFINDER:SHIGEIFINDER_MODULE (SRR2838702) [100%] 1 of 1 ✔
[a4/693ce2] process > BACTOPIATOOLS:SHIGEIFINDER:CSVTK_CONCAT (shigeifinder) [100%] 1 of 1 ✔
[a5/6d7352] process > BACTOPIATOOLS:CUSTOM_DUMPSOFTWAREVERSIONS (1) [100%] 1 of 1 ✔

Bactopia Tools: `shigeifinder Execution Summary
---------------------------
Bactopia Version : 2.2.0
Nextflow Version : 22.10.4
Command Line     : nextflow run /home/gilmansiu3/miniconda3/envs/bactopia//share/bactopia-2.2.x/main.nf -w /mnt/data2/Food/REAL/test/test-01/work/ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/test/test-01 -profile docker --cleanup_workdir
Resumed          : false
Completed At     : 2023-02-10T00:23:39.803066841+08:00
Duration         : 9.7s
Success          : true
Exit Code        : 0
Error Report     : -
Launch Dir       : /mnt/data2/Food/REAL/test/test-01

Bactopia finished successfully! Found '--cleanup_workdir' removing '/mnt/data2/Food/REAL/test/test-01/work/'
(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL/test/test-01$

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL/test/test-01$ cd ../..
(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ bactopia -profile test,docker
2023-02-10 00:25:50:root:INFO - Checking if environment pre-builds are needed, use --verbose to see full details.

@rpetit3
Copy link
Member

rpetit3 commented Feb 9, 2023

Whoa!!! Super interesting, are there any files in that directory that stand out, maybe something ending in "*.config"?

I still think its a Bactopia issue since the rnaseq test run workked

haha either way, glad its working for you!

@llk578496
Copy link
Author

I think I found the real reason now!
It is all about the .nextflow directory generated which sits in the directory '/mnt/data2/Food/REAL/', which the structure looks like this:

(tree) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ tree -L 1 .nextflow
.nextflow
├── cache
├── history
├── history.lock
└── plr

I think it is because I always launch bactopia in the same directory - '/mnt/data2/Food/REAL', 21200 directories accmulated in the cache, and the last modified time for both cache & history is last Sunday when bactopia hangs.

After I move the .nextflow directory to another directory - 'BACKUP' and run the bactopia --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir, it completed successfully.

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ mv .nextflow BACKUP/
(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ bactopia --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir
2023-02-10 01:31:09:root:INFO - Checking if environment pre-builds are needed, use --verbose to see full details.
N E X T F L O W ~ version 22.10.4
Launching /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/main.nf [spontaneous_banach] DSL2 - revision: 145bb11899



| |__ __ _ | | ___ _ __ () __ _ | | ___ ___ | |___
| '_ \ / |/ __| __/ _ \| '_ \| |/ _ | | __/ _ \ / _ | / __|
| |
) | (| | (__| || () | |) | | (| | | || () | () | __ \
|.__/ _,|___|__/| ./||_,| __/ _/||_/
|
|
bactopia tools shigeifinder v2.2.0
Shigella and EIEC serotyping from assemblies

Core Nextflow options
runName : spontaneous_banach
containerEngine : docker
container : quay.io/bactopia/bactopia:2.2.0
launchDir : /mnt/data2/Food/REAL
workDir : /mnt/data2/Food/REAL/work
projectDir : /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x
userName : gilmansiu3
profile : docker
configFiles : /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/nextflow.config

Required Parameters
bactopia : /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673

Optional Parameters
outdir : /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673
run_name : shigeifinder

Nextflow Configuration Parameters
cleanup_workdir : true

Nextflow Profile Parameters
condadir : /home/gilmansiu3/miniconda3/envs/bactopia/share/bactopia-2.2.x/conda/envs
registry : quay
singularity_cache: /home/gilmansiu3/.bactopia/singularity

Helpful Parameters
wf : shigeifinder

!! Only displaying parameters that differ from the pipeline defaults !!

If you use bactopia for your analysis please cite:


Each task will use 4 CPUs out of the available 20 CPUs. At most
5 task(s) will be run at a time, this can affect the efficiency
of Bactopia. You can use the '-qs' parameter to alter the number of
tasks to run at a time (e.g. '-qs 2', means only 2 tasks or a maximum
of 8 CPUs will be used at once)

logs does not appear to be a Bactopia sample, skipping...
bactopia does not appear to be a Bactopia sample, skipping...
ClonalComplex does not appear to be a Bactopia sample, skipping...
Found 1 samples to process

If this looks wrong, now's your chance to back out (CTRL+C 3 times).
Sleeping for 5 seconds...

executor > local (3)
[1e/dfa3a7] process > BACTOPIATOOLS:SHIGEIFINDER:SHIGEIFINDER_MODULE (PPS... [100%] 1 of 1 ✔
executor > local (3)
[1e/dfa3a7] process > BACTOPIATOOLS:SHIGEIFINDER:SHIGEIFINDER_MODULE (PPS... [100%] 1 of 1 ✔
[33/3bd97c] process > BACTOPIATOOLS:SHIGEIFINDER:CSVTK_CONCAT (shigeifinder) [100%] 1 of 1 ✔
[88/43a4b9] process > BACTOPIATOOLS:CUSTOM_DUMPSOFTWAREVERSIONS (1) [100%] 1 of 1 ✔

Bactopia Tools: `shigeifinder Execution Summary
---------------------------
Bactopia Version : 2.2.0
Nextflow Version : 22.10.4
Command Line     : nextflow run /home/gilmansiu3/miniconda3/envs/bactopia//share/bactopia-2.2.x/main.nf -w /mnt/data2/Food/REAL/work/ --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir
Resumed          : false
Completed At     : 2023-02-10T01:31:22.365594338+08:00
Duration         : 10.4s
Success          : true
Exit Code        : 0
Error Report     : -
Launch Dir       : /mnt/data2/Food/REAL

Bactopia finished successfully! Found '--cleanup_workdir' removing '/mnt/data2/Food/REAL/work/'
(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$

After that, I put the .nextflow back to the directory - '/mnt/data2/Food/REAL', and run the same command, it hangs again.

(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ mv BACKUP/.nextflow/ .
(bactopia) gilmansiu3@gilmansiu3-Z490-AORUS-ULTRA:/mnt/data2/Food/REAL$ bactopia --wf shigeifinder --bactopia /mnt/data2/Food/REAL/04_bactopia/04.1_tmp/PPS-Case-00009-22MB252673 -profile docker --cleanup_workdir
2023-02-10 01:37:44:root:INFO - Checking if environment pre-builds are needed, use --verbose to see full details.

@rpetit3
Copy link
Member

rpetit3 commented Feb 9, 2023

@llk578496 amazing detective work! This is very nice to know!

@llk578496
Copy link
Author

Thank you so much for your help! I am going to close this issue now.

@llk578496 llk578496 changed the title [bug] Bactopia --wf shigeifinder stucks [Solved] Bactopia stucks or hangs [Because .nextflow directory accumulates] Feb 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants