Skip to content

v0.8.0 — Phase 7: Advanced Workflows

Choose a tag to compare

@Abolfazlrwm Abolfazlrwm released this 22 Oct 10:02
· 94 commits to main since this release

Phase 7 adds the workflow features that make FlowForge useful beyond a linear pipeline. Everything is proven on real PostgreSQL + RabbitMQ in CI.

Features

Conditional branching & fan-out/fan-in

Tasks may declare a when condition over the run's task results and a join policy — all (default) or any (one surviving path is enough). A false condition SKIPs the task, and the skip propagates to its dependants so an untaken branch cannot stall the run. The SKIPPED state is a new task-run state.

A sandboxed expression evaluator

when conditions are evaluated by a hand-written tokenizer + recursive-descent parser and evaluator (core/domain/expressions.py) — no eval, no attribute access, no calls, no indexing. A broken condition is a loud definition error, never a silent skip. Injection tests included.

Cron scheduling with no double-fire

ScheduledTrigger starts a run on a cron schedule. The scheduler fires each due trigger with SELECT FOR UPDATE SKIP LOCKED, so N scheduler instances cannot double-fire, and advances next_run_at before triggering so a crash is a missing run, not a duplicate. Proven by a 16-thread race on real PostgreSQL.

Cooperative cancellation

cancel_task_run / request_cancellation cancel a run and every unresolved task, including RUNNING siblings, and bump the fencing token so a handler still running has its late result discarded. A cancel-vs-completion race on real PostgreSQL proves exactly one of the two wins.

Execution history / event log

TaskRunEvent is an append-only record of every task-run transition (claim, finish, cancel, requeue, reclaim, promote, retry, skip). GET /api/v1/runs/<id>/events/ returns a run's events oldest first.

Manual re-run of a failed task

POST /api/v1/task-runs/<id>/requeue/ resets a FAILED task to a fresh PENDING attempt and reverts a FAILED run (never a SUCCEEDED/CANCELLED one). Two operators racing the same task re-run it exactly once.

Saga-style compensation

A task may declare compensate_for: <key>. It is held out of the forward DAG and runs only when the run fails, to undo the tasks that had succeeded — strictly one at a time in reverse dependency order, holding the run RUNNING until the rollback settles before it finishes FAILED.

New API

  • GET /api/v1/runs/<id>/events/ — a run's task-run event log.
  • POST /api/v1/task-runs/<id>/requeue/ — re-run a failed task.

Fixes

  • advance_workflow_run now walks topological_order() so skip/promotion propagation completes in one scheduler pass on any backend (it previously depended on database row order, which differs between SQLite and PostgreSQL).
  • The cancel-vs-completion test destructured the race helper's result tuple correctly.

Verification

All five CI checks green: lint, test, schema, docs, docker. The unit suite (581 tests) and the PostgreSQL/RabbitMQ integration suite — including the cron no-double-fire and cancel-vs-completion races — run and pass, not skip.
https://github.com/Abolfazlrwm/flowforge/actions/runs/39759366032

Full changelog: https://github.com/Abolfazlrwm/flowforge/blob/v0.8.0/CHANGELOG.md