ADR 0036: run the agent loop as Temporal workflows, behind AGENT_ENGINE

[DavidNic11]

Proposes running the agent loop as Temporal workflows, selected by AGENT_ENGINE=langgraph|temporal.

This PR is the ADR only — docs, no code. It's deliberately first and deliberately standalone so the design can be accepted or rejected before anyone reads ~17k lines of Go. If the answer is "we don't want a second agent loop", this is the only PR that needed to exist.

The argument, briefly

ADR 0002 chose LangGraph because the flow is "stateful, branching, resumable, waiting on asynchronous Job completion mid-turn" — which is accurate, and is also a description of a durable execution engine's product.

Look at what we've had to build since to keep that flow correct while the loop itself isn't durable:

• Tool results resolve through in-memory pending-promise maps; a restart between launch and callback loses the turn.
• ADR 0006's invocation records live in a process Map — that ADR documents the restart/scale-out loss itself.
• ADR 0012/0017's session store is explicitly best-effort.
• ADR 0034 found every linked credential in the cluster gone, because the store behind two "durable" interfaces was a Redis with --save "" on an emptyDir.
• ADR 0033 is the sharpest one. An agent turn's work lives in a Job pod; the turn's wait lives in an orchestrator pod. release.yml deploys on every push, and that incident recorded eleven rollouts in fourteen hours. Core NATS has no durability, so a reply published while no orchestrator is subscribed is discarded. The fix — hold the concluding message and re-offer every 10s until acked — is careful and correct and entirely about compensating for a wait that can't survive a deploy. Its own closing line says durability retires it.

Each is a good local decision. Together they're four state stores and a hold protocol standing in for one property.

What this does not touch

With AGENT_ENGINE=langgraph — the default — every one of those ADRs stays exactly as it is. Nothing is retracted.

agent-orchestrator keeps the OpenAI facade, /invoke, identity and RBAC resolution, the authorization pre-flight (0030/0031), the credential store (0034), both launchers, and session pages. The flag chooses only whether a turn runs buildAgentGraph() or is forwarded to the engine. AgentGraphLike is already the Server's dependency, so it's a second implementation of an existing interface rather than a refactor.

That split is the whole reviewability argument: the claim is about the loop, and every layer above it is where the last six weeks of work landed.

Three things I'd like you to push back on if you disagree

1. Pod agents keep their NATS channel. The engine's own ADR dropped NATS. That's right for agents written for it and wrong for claude-code-swe-agent, which is in production — 0026's tunnel, 0028's tool calls and 0033's hold all live on that channel. So a bridged style drives an unmodified AgentRun over the existing protocol: same image, same CR, same subjects, with a workflow holding the durable half. Nothing in production gets rewritten as a precondition.

2. Credentials must not enter workflow state. Doing only what the LangGraph path does would be weaker here, not equal — anything a workflow holds is written to Temporal event history, durably and in the clear. So the engine's pre-flight writes values into a Secret and returns only its name.

3. Enabling it is two steps, so the engine can be deployed, watched and rolled back before a single turn depends on it.

Also worth knowing

Temporal is already running on the target platform, so the subchart takes an address and bundles no server. That was the strongest argument against this whole thing and it doesn't apply.

The ADR enumerates the behavioural differences individually rather than smoothing them over — including one that looks like a small bug on the current path: the ADR 0032 §5 identity gate covers the planner's runTool but dispatchResolvedTool has no equivalent, so a Tool meant to act as a specific human runs with whatever static token its template carries. Worth fixing here regardless of this proposal.

Sequence, if you want it

#	What	Behaviour change
1	this PR — ADR only	No
2	Import the engine (git subtree, history intact), CI + release wiring	No
3	The AGENT_ENGINE switch, default langgraph	No
4	Subchart, enabled: false	No
5	Flip the default	Yes

PR 5 is not written yet, on purpose: its precondition is the e2e/ suite run whole under AGENT_ENGINE=temporal, and writing it before that evidence exists would assert a result nobody has. Two expectations there, judged differently — everything must pass unchanged, except resilience and rollout-recovery, which encode losses that stop occurring and so need re-baselining. That re-baselining is the evidence, not a workaround.

Happy to close this and stop here if the answer is no.

🤖 Generated with Claude Code

https://claude.ai/code/session_018Uj1SJ41DJJ8woZM7fd3DQ

[imaustink]

🚢