Skip to content

Workflow durability: LangGraph checkpointer support - #8

Merged
sagi5060 merged 2 commits into
devfrom
feat/3-workflow-durability
Jul 26, 2026
Merged

Workflow durability: LangGraph checkpointer support#8
sagi5060 merged 2 commits into
devfrom
feat/3-workflow-durability

Conversation

@sagi5060

Copy link
Copy Markdown
Collaborator

Closes #3

What

BaseWorkflow gains a declarative durable: ClassVar[bool] = False opt-in. When True:

  • build() compiles the graph with a checkpointer resolved from a new CheckpointSettings group (AGENTDECK_CHECKPOINT_* env / checkpoint: YAML section, following the existing LayeredSettings pattern exactly): backend (sqlite | postgres | memory, default sqlite) and url (sqlite file path or postgres DSN).
  • App.run_workflow(...) and BaseWorkflow.run(...) take thread_id: str | None = None, threaded into LangGraph's config={"configurable": {"thread_id": ...}} so a run can resume state from a checkpoint.
  • durable=True with no thread_id raises a clear ValueError instead of silently running unscoped.
  • durable=False (the default) compiles and runs byte-for-byte as before — verified by the existing tests/test_app.py suite passing unchanged.

agentdeck.runtime.checkpointer.resolve_checkpointer() is the one place that turns settings into a saver:

  • memory uses langgraph.checkpoint.memory.MemorySaver, which ships with core langgraph — no extra needed.
  • sqlite / postgres live in a new optional [durability] extra (langgraph-checkpoint-sqlite, langgraph-checkpoint-postgres), imported lazily. If durable=True picks one of these backends and the extra isn't installed, it raises a clear ImportError with an install hint instead of a bare ModuleNotFoundError — mirroring how runtime/observability.py degrades for the optional Langfuse extra.

Why

Durable, resumable workflow instances are the core requirement of the Middle PRD (§11: durable waits, resume on restart, optimistic concurrency). Non-durable workflows (the common case today) are completely unaffected.

Notable decisions / deviations

  • Core dep vs. extra: langgraph-checkpoint-sqlite pulls in aiosqlite and sqlite-vec (a LangGraph vector-store dependency unrelated to checkpointing) as transitive requirements. Given "new dependencies need a reason the stdlib or an existing dep can't cover" and the unrelated sqlite-vec pull-in, both sqlite and postgres landed in the optional [durability] extra rather than core deps.
  • Async saver, not the sync one: LangGraph's plain SqliteSaver raises NotImplementedError on every async method (aget_tuple, etc.), and the workflow runner always calls graph.ainvoke. The sqlite backend therefore uses AsyncSqliteSaver (aiosqlite-backed), whose one-shot connection handshake is async; checkpointer._run_sync bridges that from BaseWorkflow.build() (sync), handling both the "no loop running yet" and "already inside the caller's event loop" cases.
  • Known caveat, documented in code: AsyncSqliteSaver holds an asyncio.Lock that binds to whichever event loop first acquires it, and BaseWorkflow._compiled caches the compiled graph (checkpointer included) for the class's lifetime. A script that calls asyncio.run() more than once against the same durable workflow class in one process will hit "Lock ... bound to a different event loop" on the second call. This is a non-issue for the intended shape (one long-lived loop per process — a server, or a single top-level asyncio.run), and is called out as future work in checkpointer._sqlite_saver's docstring rather than solved here, to keep this change minimal.
  • Connection lifecycle stays self-contained: per issue App lifecycle: async open/close + DI seam #1 (App lifecycle) being implemented in parallel, this PR does not touch App's lifecycle, aclose(), or serve.py. The checkpointer connection is a module-level cache in runtime/checkpointer.py for the process lifetime — wiring it into App.aclose() for graceful shutdown is explicit follow-up work once App lifecycle: async open/close + DI seam #1 lands.

Test plan

New tests/test_workflow_durability.py:

  • durable=False ignores thread_id and never persists (existing behavior unchanged)
  • durable=True with no thread_id raises ValueError
  • memory backend: same thread_id accumulates state across invokes in one event loop; a different thread_id starts fresh
  • sqlite backend (skipped if [durability] isn't installed): same-loop sequential invokes accumulate, mirroring one long-lived server process
  • sqlite backend, genuine cross-process restart: two separate subprocess invocations against the same sqlite file and thread_id — the second resumes from the first's checkpoint. This is the issue's actual acceptance test ("interrupted mid-graph resumes ... after process restart").
  • Unknown backend value raises a clear ValueError
  • Full gate green with only .[dev,serve] installed (no [durability] extra) — make check passes, sqlite/postgres tests skip gracefully via pytest.importorskip

🤖 Generated with Claude Code

@sagi5060

Copy link
Copy Markdown
Collaborator Author

Review follow-up pushed to this branch: the postgres arm constructed the sync PostgresSaver, which can't back graph.ainvoke (the same NotImplementedError trap the sqlite arm dodged with AsyncSqliteSaver). Switched to AsyncPostgresSaver + added a stub-based wiring test that fails on the sync/async mixup without needing a postgres server. Also: _run_sync now re-raises bootstrap exceptions instead of masking them with IndexError, and ty's unused-ignore-comment is off (extra-dependent ignores are env-relative). Gate green with and without the [durability] extra.

sagi5060 and others added 2 commits July 26, 2026 23:15
BaseWorkflow.durable (default False) compiles with a checkpointer resolved
from a new CheckpointSettings group (AGENTDECK_CHECKPOINT_*, backend:
sqlite|postgres|memory). App.run_workflow / BaseWorkflow.run take
thread_id, threaded into LangGraph's configurable.thread_id so a run can
resume; durable=True with no thread_id raises. sqlite/postgres ship in a
new optional [durability] extra, lazy-imported with a clear error if
missing. durable=False is unchanged behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- AsyncPostgresSaver (sync PostgresSaver raises NotImplementedError on async
  methods — same trap the sqlite arm dodged), entered via _run_sync
- _run_sync re-raises bootstrap-thread exceptions instead of IndexError
- stub-based wiring test that fails on the sync/async mixup, no server needed
- ty: unused-ignore-comment off — extra-dependent ignores are env-relative

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@sagi5060
sagi5060 force-pushed the feat/3-workflow-durability branch from ba7ed5c to ebc3b27 Compare July 26, 2026 20:16
@sagi5060
sagi5060 merged commit ff5f30e into dev Jul 26, 2026
1 check passed
@sagi5060
sagi5060 deleted the feat/3-workflow-durability branch July 26, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Workflow durability: LangGraph checkpointer support

1 participant