You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have a data pipeline DAG composed of a series of dependent transformations. When one particular tasks fails, I would expect a memoized resubmit to restart at the failed task and to respect dependencies.
What actually happens is that all subsequent tasks are executed at the same time, without respecting their dependencies. This happens whether failFast is set to true or false. This happens using both the UI and the CLI.
Here is the original workflow view:
Now when I run argo resubmit --memoized resubmit-bug-dag, the resubmit workflow ends up in the following state:
Thus I cannot use the resubmit memoized feature for the DAG as it breaks my pipelines. For the moment, I think I will try to implement step-level memoization as a workaround, though it is not optimal as I now have to set an arbitrary maxAge for future pipelines to properly work.
Thank you for your help!
Version
v3.5.5
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
agilgur5
changed the title
Failed Workflow resubmit does not respect DAG dependencies
Failed Workflow memoized resubmit does not respect DAG dependencies
Apr 15, 2024
Pre-requisites
:latest
What happened/what did you expect to happen?
I have a data pipeline DAG composed of a series of dependent transformations. When one particular tasks fails, I would expect a memoized resubmit to restart at the failed task and to respect dependencies.
What actually happens is that all subsequent tasks are executed at the same time, without respecting their dependencies. This happens whether
failFast
is set to true or false. This happens using both the UI and the CLI.Here is the original workflow view:
Now when I run
argo resubmit --memoized resubmit-bug-dag
, the resubmit workflow ends up in the following state:Thus I cannot use the resubmit
memoized
feature for the DAG as it breaks my pipelines. For the moment, I think I will try to implement step-level memoization as a workaround, though it is not optimal as I now have to set an arbitrarymaxAge
for future pipelines to properly work.Thank you for your help!
Version
v3.5.5
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
Logs from the workflow controller
Logs from in your workflow's wait container
The text was updated successfully, but these errors were encountered: