Run workflows as managed jobs with progress and cancellation

[CHXZAVARKA]

Problem

The workflow tool runs in the foreground and returns only after its worker finishes. While it is running, the parent agent cannot do other work. Cancellation also loses the partial result, and there is no supported way to inspect the active phases or continue from a completed phase.

This is a rough edge for the kind of fan-out work the tool is designed for. A ten-minute research or review workflow should not make the parent session look stuck.

Proposal

Let a workflow run as a managed job through the existing jobs subsystem.

An optional background mode could return a job id immediately. The existing job tools could then expose status, recent logs, the active phase, completed branches, failures, and the final result. Killing the job should cancel the worker and every child agent, then settle the job once.

Foreground behavior should remain unchanged for callers that want one blocking result.

How I would implement it

1. Add a workflow job producer that owns the worker and its child agents.
2. Emit structured progress records for phase start, phase completion, agent start, agent completion, log output, and the final result.
3. Store bounded recent output in the job snapshot, with the full durable trace in the session log.
4. Route cancellation through the jobs lifecycle so owner teardown and explicit job_kill use the same cleanup path.
5. Persist completed phase() outputs as checkpoints. Arbitrary JavaScript stacks cannot be resumed safely, so the first resumable contract should restart only from an explicit phase boundary.

The existing jobs subsystem already owns most of the lifecycle rules. The workflow package would become a producer instead of inventing another background-task API.

Acceptance criteria

• Background workflow start returns a job id without blocking the parent turn.
• job_read and job_wait expose the current phase, branch outcomes, and bounded logs.
• job_kill cancels the worker and all live children without duplicate completion notices.
• Session or service teardown waits for cleanup and leaves no orphan children.
• The final background result matches the current foreground result shape.
• Foreground workflow behavior remains compatible.
• A workflow written with explicit phase() boundaries can restart from the last completed checkpoint after interruption.

The current behavior is described in the workflow tool README.

[Electricitysheep]

workflow 前台阻塞确实限制编排——managed jobs（后台运行 + 进度 + 取消）能让父 agent 并行处理，是工作流系统化的关键一步。

第 9 章的工作流章节讨论了这个方向（确定性编排 vs 并行）：https://github.com/Electricitysheep/dsh-handbook/blob/main/docs/09-mcp-subagent-workflow.md