Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incompatibility between Snakemake (v7.0.0 or later) and Tibanna? #1475

Closed
dvav opened this issue Mar 11, 2022 · 3 comments · Fixed by #1491
Closed

Incompatibility between Snakemake (v7.0.0 or later) and Tibanna? #1475

dvav opened this issue Mar 11, 2022 · 3 comments · Fixed by #1491
Assignees
Labels
bug Something isn't working

Comments

@dvav
Copy link

dvav commented Mar 11, 2022

Snakemake version: 7.1.1

Describe the bug

I 've been using Snakemake with Tibanna for some time now without any problem. However, it seems that I cannot make Snakemake v7.0.0 or higher work with Tibanna (see traceback below). It works fine, though, with Snakemake v6.15.5. Any idea what's wrong? Maybe it has something to do with Snakemake v7.0.0 requiring at least Python v3.7?

Logs

Traceback (most recent call last):
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/utils.py", line 430, in format
    return fmt.format(_pattern, *args, **variables)
  File "/usr/lib/python3.9/string.py", line 161, in format
    return self.vformat(format_string, args, kwargs)
  File "/usr/lib/python3.9/string.py", line 165, in vformat
    result, _ = self._vformat(format_string, args, kwargs, used_args, 2)
  File "/usr/lib/python3.9/string.py", line 205, in _vformat
    obj, arg_used = self.get_field(field_name, args, kwargs)
  File "/usr/lib/python3.9/string.py", line 270, in get_field
    obj = self.get_value(first, args, kwargs)
  File "/usr/lib/python3.9/string.py", line 227, in get_value
    return kwargs[key]
KeyError: 'job_specific_args'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/__init__.py", line 714, in snakemake
    success = workflow.execute(
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/workflow.py", line 1097, in execute
    success = self.scheduler.schedule()
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/scheduler.py", line 540, in schedule
    self.run(runjobs)
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/scheduler.py", line 588, in run
    executor.run_jobs(
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 153, in run_jobs
    self.run(
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 2360, in run
    tibanna_input = self.make_tibanna_input(job)
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 2344, in make_tibanna_input
    self.add_command(job, tibanna_args, tibanna_config)
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 2221, in add_command
    command = self.format_job_pattern(
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 375, in format_job_pattern
    cmd = format(
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/utils.py", line 432, in format
    if str(ex).strip("'") in variables["wildcards"].keys():
KeyError: 'wildcards'

Minimal example

Snakefile (under workflow/):

ID = ['SAMPLE1', 'SAMPLE2']

rule all:
    input:
        tag = expand('results/TEST/{id}/{id}.txt', id = ID)

rule my_rule:
    output:
        tag = 'results/TEST/{id}/{id}.txt'
    shell: "touch {output.tag}"

Profile config.yaml (under workflow/config/aws/):

tibanna: true
tibanna-sfn: tibanna_unicorn_something
default-remote-prefix: mybucket/sudir/
default-resources:
  - disk_mb=8000
tibanna-config:
  - instance_type=t2.micro
  - log_bucket=mylogbucket
keep-going: true

Run as: snakemake --jobs 2 --profile wokflow/config/aws/ -n (works fine)
Run as: snakemake --jobs 2 --profile wokflow/config/aws/ (see traceback above)

Additional context None

@dvav dvav added the bug Something isn't working label Mar 11, 2022
@SooLee
Copy link
Contributor

SooLee commented Mar 11, 2022

It looks like some new incompatibility issue. I can take a look this weekend.

@hwalinga
Copy link
Contributor

hwalinga commented Mar 14, 2022

Seems the call in the tibanna executor should be format_job and not format_job_pattern.

@dvav
Copy link
Author

dvav commented Mar 16, 2022

OK, many thanks for sorting this out @SooLee , @hwalinga and @johanneskoester. Looking forward to the updated version of the software.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants