Skip to content

Feature: Dependent / sequential task chains #11

@deepjoy

Description

@deepjoy

Summary

Allow tasks to declare dependencies on other tasks, so a task only starts after its dependencies have completed successfully.

Motivation

Some workflows have strict ordering constraints that go beyond parent-child relationships. For example, in an S3 sync engine:

  • "Delete the old version" must only happen after "upload the new version" is confirmed
  • "Update sync state in DB" must only happen after the transfer completes and checksum is verified
  • A rename operation (copy + delete) requires the copy to succeed before the delete is submitted

Without dependency support, consumers must implement their own sequencing logic outside the scheduler, losing the benefits of TaskMill's persistence and retry handling.

Proposed Behavior

  • TaskSubmission accepts depends_on: Vec<TaskId> — the task remains in a blocked state until all dependencies are in completed status
  • If a dependency fails (after exhausting retries), dependents are either:
    • Auto-cancelled (default)
    • Moved to a dependency_failed state for manual intervention (configurable)
  • Dependency relationships are persisted in SQLite and survive restarts
  • Circular dependencies are detected at submission time and rejected
  • SchedulerSnapshot exposes the dependency graph for debugging

Example

let upload = scheduler.submit(
    TaskSubmission::new("upload-file")
        .payload_json(&upload_plan)?
).await?;

// Only delete after upload succeeds
scheduler.submit(
    TaskSubmission::new("delete-old-version")
        .depends_on(vec![upload.task_id()])
        .payload_json(&delete_plan)?
).await?;

Design Considerations

  • Dependencies are distinct from parent-child: dependencies are between peers, parent-child is hierarchical
  • Priority inheritance: a blocked task's priority should propagate to its dependencies to avoid priority inversion
  • The scheduler's dispatch loop should skip blocked tasks efficiently (don't re-evaluate on every tick)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions