Repository navigation
Replies: 2 comments
|
Dagster can run application-defined validity checks and select a dependency subgraph for repair. It does not infer the meaning of semantic validity from the DAG alone. For an op job, you can reexecute a selected step and its downstream dependencies from a previously successful run. For example, the selection in I checked this locally on Dagster 1.13.26 with a five-op graph, an in-process executor and a temporary instance. After a successful run, selecting For asset-based workflows, asset checks can express the validity tests. I also checked a separate local asset example where a failed ERROR-severity check with The complementary part of your proposal would therefore be turning application-specific evidence into an explicit invalidation and repair decision. I'd evaluate it on a case where the earlier run succeeded, a later validator rejects one result, and only the affected dependency branch is selected for repair. Make the upstream artifact versions and validation results part of that decision. The parent-run input reuse above was tested for an op job. In a separate asset repair, after I rematerialized the source with a new value, rerunning only its two descendants read that new source value through the filesystem IO manager. It did not retrieve the source version from the earlier run. Preserve or explicitly select the intended artifact versions if that distinction matters. For the repair itself, external writes need idempotency or an explicit compensation strategy. A correct dependency selection alone cannot roll back a notification, payment or other external effect from the original execution. |
|
Thanks for your detailed explanation. I've now completed two controlled integration experiments using Dagster 1.13.26. In the first, a five-step Dagster workflow completed successfully before a later application-level validator rejected an intermediate output. Consistency identified the affected dependency boundary and passed the selection to Dagster's native re-execution mechanism. Across 10 trials, both Dagster's informed manual replay and Consistency's selection executed three steps, compared with five for a full replay. All corrected outputs matched. I then extended the experiment to use actual Dagster asset-check events and materialization-version records, with an automated Dagster-only approach as the baseline. That comparison also passed 10/10 trials. Both strategies rematerialized three assets and produced identical outputs. One additional test was particularly useful: when an upstream source was rematerialized after the failed check, Consistency refused to authorize recovery using the outdated parent-run evidence. I'm not claiming an execution-performance improvement over Dagster's existing capabilities. The question I'm exploring is whether a reusable, evidence-aware recovery decision layer provides practical value across orchestration systems, particularly around version consistency and deciding when recovery should be refused. I'd welcome your perspective on whether that boundary represents a useful integration opportunity. |
Uh oh!
There was an error while loading. Please reload this page.
Hello Dagster team,
I'm Oluseyi, founder of Consistency, a Python recovery layer for multi-step software and AI-agent workflows.
We've built and tested dependency-aware selective recovery, which uses application-defined validity checks to identify trustworthy completed work and regenerate only affected stages.
I'm interested in understanding how Dagster handles a case where a previously completed step or asset is subsequently determined to be semantically invalid, particularly when some downstream results depend on it while other branches remain valid.
Does Dagster already provide a native mechanism for identifying and selectively recovering that affected dependency boundary?
I'd welcome a technical discussion to determine whether Consistency offers any complementary capability.
Our project: https://github.com/Tarchy01/Consistency
Thanks!
All reactions