Skip to content


Browse files Browse the repository at this point in the history
fix: Repair MREs from #823 (#1203)
* add failing tests 823

* fix mistakes

* black

* Update tests from #823 MREs

- Can't have the b.txt in first text be temp and expect it present, it
will correctly be deleted.
- Add third MRE which is targeting more of a log issue

* Add error message printed if #823 is encountered, also exit 1

The main scheduler loop terminates if there are no more runnable
jobs. This tests if there still are jobs that should be run, but that
we cannot seem to get to. If so, print an error message, print the
files affected, and exit 1. The latter is important so Snakemake can,
as hotfix, be rerun in a loop until it's actually completed all

* Fix checkpoint+temp leads to incomplete run issue

The aggressive early deletion of temp files must ignore
checkpoints. Other rules depend on a checkpoint only via the first
output file of the checkpoint (flagged string) as a stub. All other
outputs marked as temp would be deleted (and have been before this
patch). Since those files are missing, the dependent rules will never
be runnable and the workflow ends before all targets have been built.

* Don't break untiljobs

* fix typo

Co-authored-by: Maarten-vd-Sande <>
Co-authored-by: Johannes Köster <>
Co-authored-by: Johannes Köster <>
  • Loading branch information
4 people committed Feb 21, 2022
1 parent bd65057 commit b007979
Show file tree
Hide file tree
Showing 4 changed files with 32 additions and 1 deletion.
19 changes: 18 additions & 1 deletion snakemake/
Expand Up @@ -43,6 +43,11 @@ def cumsum(iterable, zero=[0]):
"Exiting because a job execution failed. " "Look above for error message"

"BUG: Out of jobs ready to be started, but not all files built yet."
" Please check for more information."

class DummyRateLimiter(ContextDecorator):
def __enter__(self):
Expand Down Expand Up @@ -470,11 +475,23 @@ def schedule(self):
return False

# normal shutdown because all jobs have been finished
# all runnable jobs have finished, normal shutdown
if not needrun and (not running or self.workflow.immediate_submit):
if errors:
# we still have unfinished jobs. this is not good. direct
# user to github issue
if self.remaining_jobs:
"Remaining jobs:\n"
+ "\n".join(
" - " + str(job) + ": " + ", ".join(job.output)
for job in self.remaining_jobs
return False
return not errors

# continue if no new job needs to be executed
Expand Down
9 changes: 9 additions & 0 deletions tests/test_issue823_3/Snakefile
@@ -0,0 +1,9 @@
rule all:

checkpoint a:
"touch {output}"
Empty file.
5 changes: 5 additions & 0 deletions tests/
Expand Up @@ -887,6 +887,11 @@ def test_issue823_2():

def test_issue823_3():

def test_pathlib():
Expand Down

0 comments on commit b007979

Please sign in to comment.