Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Renku complains about outputs of un-executed workflow file steps #3261

Closed
m-alisafaee opened this issue Jan 13, 2023 · 0 comments · Fixed by #3263
Closed

Renku complains about outputs of un-executed workflow file steps #3261

m-alisafaee opened this issue Jan 13, 2023 · 0 comments · Fixed by #3263
Labels

Comments

@m-alisafaee
Copy link
Contributor

When executing a subset of step from a workflow file, Renku will complain the outputs of the un-executed steps cannot be found.

To reproduce:

  1. Create a project
  2. Unzip and copy the attached workflow file to the project: workflow-file.yml.zip
  3. renku run workflow-file.yml head tail
Executing step 'workflow-file.head': 'head -n10 Dockerfile > intermediate' ...
Executing step 'workflow-file.tail': 'tail -n5 intermediate > results/output.csv' ...
Error: Git command failed: Cmd('git') failed due to: exit code(1)
  cmdline: git commit --message=renku run workflow-file.yml head tail

renku-transaction: d37de79845544734b296883470fbc9a1 --no-verify -- workflow-file.yml Dockerfile intermediate results/output.csv results/output.csv results/output.csv.wc /private/tmp/workflow-file-bug/.renku
  stderr: 'error: pathspec 'results/output.csv.wc' did not match any file(s) known to git'

Traceback (most recent call last):
  File "/Users/Mohammad/Documents/Workspace.noindex/sdsc/code/renku-python/renku/infrastructure/repository.py", line 1780, in _run_git_command
    return getattr(repository.git, command)(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/git/cmd.py", line 639, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/git/cmd.py", line 1184, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/git/cmd.py", line 984, in execute
    raise GitCommandError(redacted_command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(1)
  cmdline: git commit --message=renku run workflow-file.yml head tail

renku-transaction: d37de79845544734b296883470fbc9a1 --no-verify -- workflow-file.yml Dockerfile intermediate results/output.csv results/output.csv results/output.csv.wc /private/tmp/workflow-file-bug/.renku
  stderr: 'error: pathspec 'results/output.csv.wc' did not match any file(s) known to git'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/Mohammad/Documents/Workspace.noindex/sdsc/code/renku-python/renku/ui/cli/exception_handler.py", line 92, in main
    return super().main(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/Mohammad/Documents/Workspace.noindex/sdsc/code/renku-python/renku/ui/cli/run.py", line 612, in run
    run_workflow_file_command(no_commit=no_commit, commit_only=commit_only)
  File "/Users/Mohammad/Documents/Workspace.noindex/sdsc/code/renku-python/renku/command/command_builder/command.py", line 266, in execute
    hook(self, context, result, *args, **kwargs)
  File "/Users/Mohammad/Documents/Workspace.noindex/sdsc/code/renku-python/renku/command/command_builder/command.py", line 202, in _post_hook
    raise result.error
  File "/Users/Mohammad/Documents/Workspace.noindex/sdsc/code/renku-python/renku/command/command_builder/repo.py", line 96, in _post_hook
    finalize_commit(
  File "/Users/Mohammad/Documents/Workspace.noindex/sdsc/code/renku-python/renku/core/util/git.py", line 1127, in finalize_commit
    repository.commit(commit_message + transaction_id, committer=committer, no_verify=True, paths=paths)
  File "/Users/Mohammad/Documents/Workspace.noindex/sdsc/code/renku-python/renku/infrastructure/repository.py", line 263, in commit
    self.run_git_command(
  File "/Users/Mohammad/Documents/Workspace.noindex/sdsc/code/renku-python/renku/infrastructure/repository.py", line 406, in run_git_command
    return _run_git_command(self._repository, command, *args, **kwargs)
  File "/Users/Mohammad/Documents/Workspace.noindex/sdsc/code/renku-python/renku/infrastructure/repository.py", line 1782, in _run_git_command
    raise errors.GitCommandError(
renku.core.errors.GitCommandError: Git command failed: Cmd('git') failed due to: exit code(1)
  cmdline: git commit --message=renku run workflow-file.yml head tail

renku-transaction: d37de79845544734b296883470fbc9a1 --no-verify -- workflow-file.yml Dockerfile intermediate results/output.csv results/output.csv results/output.csv.wc /private/tmp/workflow-file-bug/.renku
  stderr: 'error: pathspec 'results/output.csv.wc' did not match any file(s) known to git'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
1 participant