snakemake: error: argument --configfile: expected one argument

[anilchauhanhp9]

Hi,
i downloaded the dadasnake from github by git clone https://github.com/a-h-b/dadasnake.git command
then i installed snakemake in conda env as described in step 4 of the installation guide
after that i tried step 6 of the guide to Initialize conda environments using command /home/anil/dadasnake/submit_scripts/dadasnake_withReport.sh -i /home/anil/dadasnake/config/config.init.yaml and i got the following error
usage: snakemake [-h] [--profile PROFILE] [--snakefile FILE] [--gui [PORT]]
[--cores [N]] [--local-cores N]
[--resources [NAME=INT [NAME=INT ...]]]
[--config [KEY=VALUE [KEY=VALUE ...]]] [--configfile FILE]
[--list] [--list-target-rules] [--directory DIR] [--dryrun]
[--printshellcmds] [--debug-dag] [--dag]
[--force-use-threads] [--rulegraph] [--d3dag] [--summary]
[--detailed-summary] [--archive FILE] [--touch]
[--keep-going] [--force] [--forceall]
[--forcerun [TARGET [TARGET ...]]]
[--prioritize TARGET [TARGET ...]]
[--until TARGET [TARGET ...]]
[--omit-from TARGET [TARGET ...]] [--allow-ambiguity]
[--cluster CMD | --cluster-sync CMD | --drmaa [ARGS]]
[--drmaa-log-dir DIR] [--cluster-config FILE]
[--immediate-submit] [--jobscript SCRIPT] [--jobname NAME]
[--cluster-status CLUSTER_STATUS] [--kubernetes [NAMESPACE]]
[--kubernetes-env ENVVAR [ENVVAR ...]]
[--container-image IMAGE] [--reason] [--stats FILE]
[--nocolor] [--quiet] [--nolock] [--unlock]
[--cleanup-metadata FILE [FILE ...]] [--rerun-incomplete]
[--ignore-incomplete] [--list-version-changes]
[--list-code-changes] [--list-input-changes]
[--list-params-changes] [--latency-wait SECONDS]
[--wait-for-files [FILE [FILE ...]]] [--benchmark-repeats N]
[--notemp] [--keep-remote] [--keep-target-files]
[--keep-shadow]
[--allowed-rules ALLOWED_RULES [ALLOWED_RULES ...]]
[--max-jobs-per-second MAX_JOBS_PER_SECOND]
[--max-status-checks-per-second MAX_STATUS_CHECKS_PER_SECOND]
[--restart-times RESTART_TIMES] [--attempt ATTEMPT]
[--timestamp] [--greediness GREEDINESS] [--no-hooks]
[--print-compilation]
[--overwrite-shellcmd OVERWRITE_SHELLCMD] [--verbose]
[--debug] [--runtime-profile FILE] [--mode {0,1,2}]
[--bash-completion] [--use-conda] [--conda-prefix DIR]
[--create-envs-only] [--list-conda-envs] [--use-singularity]
[--singularity-prefix DIR] [--singularity-args ARGS]
[--wrapper-prefix WRAPPER_PREFIX]
[--default-remote-provider {S3,GS,FTP,SFTP,S3Mocked,gfal,gridftp}]
[--default-remote-prefix DEFAULT_REMOTE_PREFIX]
[--no-shared-fs] [--version]
[target [target ...]]
snakemake: error: argument --configfile: expected one argument
please help me with the issue, did i ran a wrong command or i forgot something during installation
Thank you

[a-h-b]

Hi -
thanks for getting in touch. I think you've indeed missed something when installing or just got the command for the initialisation wrong. If I got you right, you tried
/home/anil/dadasnake/submit_scripts/dadasnake_withReport.sh -i /home/anil/dadasnake/config/config.init.yaml
when indeed, you should call the dadasnake script:
/home/anil/dadasnake/dadasnake -i /home/anil/dadasnake/config/config.init.yaml `
This script isn't there when you clone the pipeline from github. You copy it from either of the auxiliary_files/dadasnake_* files, as described in step 3.
I hope this helps. Please let me know, if there are more questions.
-A

[anilchauhanhp9]

Thank you for your quick response.
I realized that i skipped some of the steps of installation as i was unable to understand some of the terms as i am a wet lab student and very recently i started learning all this dry lab work.

as suggested i ran the above command and got a new issue
/home/anil/dadasnake/dadasnake -i /home/anil/dadasnake/config/config.init.yaml
Initializing conda environments.
module: command not found
Traceback (most recent call last):
File "/home/anil/dadasnake/conda/snakemake_env/bin/snakemake", line 10, in
sys.exit(main())
File "/home/anil/dadasnake/conda/snakemake_env/lib/python3.9/site-packages/snakemake/init.py", line 2592, in main
success = snakemake(
File "/home/anil/dadasnake/conda/snakemake_env/lib/python3.9/site-packages/snakemake/init.py", line 504, in snakemake
overwrite_config.update(load_configfile(f))
File "/home/anil/dadasnake/conda/snakemake_env/lib/python3.9/site-packages/snakemake/io.py", line 1596, in load_configfile
raise WorkflowError(
snakemake.exceptions.WorkflowError: Config file must be given as JSON or YAML with keys at top level.
Please help

[a-h-b]

Hi -
glad we could straighten that one out.
yes, for your new error, you need to change the VARIABLE_CONFIG file, particularly in the line that currently reads:
LOADING_MODULES module load Anaconda3/5.3.0
it should just be
LOADING_MODULES
because it seems your system does not rely on modules.
Let me know, if there are other errors.
Best wishes -
A

[anilchauhanhp9]

sorry for the constant trouble
i edited the VARIABLE_CONFIG file as suggested but still got this error
/home/anil/dadasnake/dadasnake -i /home/anil/dadasnake/config/config.init.yaml
Initializing conda environments.
Traceback (most recent call last):
File "/home/anil/dadasnake/conda/snakemake_env/bin/snakemake", line 10, in
sys.exit(main())
File "/home/anil/dadasnake/conda/snakemake_env/lib/python3.9/site-packages/snakemake/init.py", line 2592, in main
success = snakemake(
File "/home/anil/dadasnake/conda/snakemake_env/lib/python3.9/site-packages/snakemake/init.py", line 504, in snakemake
overwrite_config.update(load_configfile(f))
File "/home/anil/dadasnake/conda/snakemake_env/lib/python3.9/site-packages/snakemake/io.py", line 1596, in load_configfile
raise WorkflowError(
snakemake.exceptions.WorkflowError: Config file must be given as JSON or YAML with keys at top level

[a-h-b]

Three tests, please:
a) how did you install snakemake?
b) can you please post the output of
/home/anil/dadasnake/dadasnake -h
c) I guess you did not change anything in /home/anil/dadasnake/config/config.init.yaml ?
Thanks -
A

[anilchauhanhp9]

a) i installed snakemake using step 4 of the guide as
conda install -c conda-forge mamba
mamba create --prefix /home/anil/dadasnake/conda/snakemake_env
conda activate /home/anil/dadasnake/conda/snakemake_env
mamba install -c conda-forge -c bioconda snakemake
conda deactivate

b)/home/anil/dadasnake/dadasnake -h
Usage: /home/anil/dadasnake/dadasnake [-u|d|c|l|i] [-r] [-t number] [-n name] /absolute_path/to/config_file
-n , only works with -c
-r is set, a report is generated (it's recommended to run -c or -l with -r)
-d if set, a dryrun is performed
-c if set, the whole thing is submitted to the cluster
-i if set, only the conda environments will be installed, if they don't exist
-u if set, the working directory will be unlocked (only necessary for crash/kill recovery)
-l if set, the main snakemake thread and indivdual rules are run in the current terminal session
-t <max_threads> maximum number of cpus to use for all rules at a time. Defaults to 50 for -c, and to 1 for -l. No effect on -r, -d or -u only.
c) yes i did change some in the config.init.yaml file
outputdir dadasnake_initialized
taxonomy
mothur
do true
blast
do true
hand_off
biom true
phyloseq true

[a-h-b]

do you mean to say you deleted the ":"s ?

[a-h-b]

and the spaces at the start of the lines?

[anilchauhanhp9]

No i only removed : and ":" not the spaces at the start of the line

[a-h-b]

okay, that explains the error. Please don't remove the : - they're part of the yaml format.
-A

[anilchauhanhp9]

ok as suggested i added the : in the file
still got this
/home/anil/dadasnake/dadasnake -i /home/anil/dadasnake/config/config.init.yaml
Initializing conda environments.

Final resource settings:
maxCores: 1
Sample table was not found. Please enter the absolute path and file name in the config file.
FileNotFoundError in line 113 of /home/anil/dadasnake/workflow/rules/get_config.smk:
[Errno 2] No such file or directory: '/home/anil/testdata/samples.small.tsv'
File "/home/anil/dadasnake/Snakefile", line 8, in
File "/home/anil/dadasnake/workflow/rules/get_config.smk", line 113, in
File "/home/anil/dadasnake/conda/snakemake_env/lib/python3.9/site-packages/pandas/io/parsers.py", line 689, in read_table
File "/home/anil/dadasnake/conda/snakemake_env/lib/python3.9/site-packages/pandas/io/parsers.py", line 462, in _read
File "/home/anil/dadasnake/conda/snakemake_env/lib/python3.9/site-packages/pandas/io/parsers.py", line 819, in init
File "/home/anil/dadasnake/conda/snakemake_env/lib/python3.9/site-packages/pandas/io/parsers.py", line 1050, in _make_engine
File "/home/anil/dadasnake/conda/snakemake_env/lib/python3.9/site-packages/pandas/io/parsers.py", line 1867, in init
File "/home/anil/dadasnake/conda/snakemake_env/lib/python3.9/site-packages/pandas/io/parsers.py", line 1362, in _open_handles
File "/home/anil/dadasnake/conda/snakemake_env/lib/python3.9/site-packages/pandas/io/common.py", line 642, in get_handle

[anilchauhanhp9]

also i am not running dadasnake in the cluster but on a ubuntu desktop so which dadasnake file should i copy in the step 3 from auxiliary_files.
i copied the one with the process running snakemake to the cluster

[a-h-b]

Regarding your second question: it shouldn't make a difference in that case. For the proper runs, you'd be using the -l setting then.
I am still trying to figure out why the workflow is looking in the wrong directory. You didn't change the dadasnake file, did you? nor the config/config.default.yaml?
-A

[anilchauhanhp9]

i guess you are asking me about this file /home/anil/dadasnake/dadasnake if that's the case i didn"t change anything in this file nor in the config/config.default.yaml file

[a-h-b]

ok

[a-h-b]

oh, I am sorry, I did not describe this well in the README. Can you please do
cd /home/anil/dadasnake/
and then run the initialisation from there?
i.e.
./dadasnake -i config/config.init.yaml

thanks -
A

[anilchauhanhp9]

ya now the command is running untill it runs i have some questions

1. can i process my 16s nanopore reads using the pipeline
2. in the database downloading steps you suggested to download database for mothur, DADA2 implementation database from McLaren, database for DECIPHER, database for fungal traits
   do i need to download all these databases....?

[a-h-b]

1. ha, well, we've set up a workflow that should be able to work on ChopSeq-treated data from Nanopore sequencing - this is from a consensus sequencing method (https://academic.oup.com/gigascience/article/7/12/giy140/5202451). It shouldn't work native Nanopore reads, though: Can DADA2 pipeline process MinION whole 16S single read fastq files? benjjneb/dada2#718
2. no, you just need the database that suits your needs (e.g. just DECIPHER, if your dataset is quite small; or just the SILVA data base for mothur) assuming you want to only run 16S data

[anilchauhanhp9]

ok, so u meant that to analyse the nanopore reads i have to firstly process my reads using chopseq and then run the dadasnake right...?

[a-h-b]

if you've sequenced them with the method described in that paper or the INCseq method. If you've just sequenced amplicons without circular consensus, please use the Nanopore (what's in my pot?) workflow for processing.

[anilchauhanhp9]

okay
The command ran smoothly
Thank you for your support

[anilchauhanhp9]

if you've sequenced them with the method described in that paper or the INCseq method. If you've just sequenced amplicons without circular consensus, please use the Nanopore (what's in my pot?) workflow for processing.

okay thank you for the information

[a-h-b]

you're welcome. Good luck with your analysis.
-A

[anilchauhanhp9]

Thanks