Skip to content
Permalink
Browse files
fix: Repair MREs from #823 (#1203)
* add failing tests 823

* fix mistakes

* black

* Update tests from #823 MREs

- Can't have the b.txt in first text be temp and expect it present, it
will correctly be deleted.
- Add third MRE which is targeting more of a log issue

* Add error message printed if #823 is encountered, also exit 1

The main scheduler loop terminates if there are no more runnable
jobs. This tests if there still are jobs that should be run, but that
we cannot seem to get to. If so, print an error message, print the
files affected, and exit 1. The latter is important so Snakemake can,
as hotfix, be rerun in a loop until it's actually completed all
tasks.

* Fix checkpoint+temp leads to incomplete run issue

The aggressive early deletion of temp files must ignore
checkpoints. Other rules depend on a checkpoint only via the first
output file of the checkpoint (flagged string) as a stub. All other
outputs marked as temp would be deleted (and have been before this
patch). Since those files are missing, the dependent rules will never
be runnable and the workflow ends before all targets have been built.

* Don't break untiljobs

* fix typo

Co-authored-by: Maarten-vd-Sande <maartenvandersande@hotmail.com>
Co-authored-by: Johannes Köster <johannes.koester@uni-due.de>
Co-authored-by: Johannes Köster <johannes.koester@tu-dortmund.de>
  • Loading branch information
4 people committed Feb 21, 2022
1 parent bd65057 commit b0079791718a390d1f920df15a405cf633314312
Show file tree
Hide file tree
Showing 4 changed files with 32 additions and 1 deletion.
@@ -43,6 +43,11 @@ def cumsum(iterable, zero=[0]):
"Exiting because a job execution failed. " "Look above for error message"
)

_ERROR_MSG_ISSUE_823 = (
"BUG: Out of jobs ready to be started, but not all files built yet."
" Please check https://github.com/snakemake/snakemake/issues/823 for more information."
)


class DummyRateLimiter(ContextDecorator):
def __enter__(self):
@@ -470,11 +475,23 @@ def schedule(self):
return False
continue

# normal shutdown because all jobs have been finished
# all runnable jobs have finished, normal shutdown
if not needrun and (not running or self.workflow.immediate_submit):
self._executor.shutdown()
if errors:
logger.error(_ERROR_MSG_FINAL)
# we still have unfinished jobs. this is not good. direct
# user to github issue
if self.remaining_jobs:
logger.error(_ERROR_MSG_ISSUE_823)
logger.error(
"Remaining jobs:\n"
+ "\n".join(
" - " + str(job) + ": " + ", ".join(job.output)
for job in self.remaining_jobs
)
)
return False
return not errors

# continue if no new job needs to be executed
@@ -0,0 +1,9 @@
rule all:
input:
"a.txt"

checkpoint a:
output:
"a.txt"
shell:
"touch {output}"
Empty file.
@@ -887,6 +887,11 @@ def test_issue823_2():
run(dpath("test_issue823_2"))


@skip_on_windows
def test_issue823_3():
run(dpath("test_issue823_3"))


@skip_on_windows
def test_pathlib():
run(dpath("test_pathlib"))

0 comments on commit b007979

Please sign in to comment.