Airflow now has two mechanisms for handling long-running or multi-step tasks:
- Deferrable operators / Triggerer — task suspends itself and hands polling off to the triggerer; worker slot is freed
- ResumableMixin / sync mode (AIP-103) — task saves progress to the task state store, exits, gets rescheduled, and resumes from the checkpoint
These solve overlapping but distinct problems, and there is currently no documentation explaining when to reach for one vs the other.
We should add that
Airflow now has two mechanisms for handling long-running or multi-step tasks:
These solve overlapping but distinct problems, and there is currently no documentation explaining when to reach for one vs the other.
We should add that