CONTINUUM: Verifiable semantic recovery for long-running AI agents. Semantic checkpoints (not conversation dumps), an idempotent action ledger that refuses duplicate side effects, and a hash-chained tamper-evident event log, all exposed as a deny-by-default MCP server. Framework-agnostic, Python 3.11+.
Why · Quick Start · How it works · Features · Security Extension · Empirical Verification · MCP Integration · Framework Integration · Core Concepts · Architecture · API and CLI · Roadmap · What CONTINUUM Is Not · Related work · Status and limitations · Contributing · License
Modern AI agents run long tasks (hundreds of LLM calls, tool invocations, file and database writes). When they crash, the usual response is to replay everything from scratch, which duplicates work, duplicates side effects, wastes tokens, and loses decisions.
CONTINUUM asks a narrower, harder question: can an agent resume from a compact semantic representation of its task state while independently verifying that state is still valid in the current environment? It is not a generic agent framework, a memory system, or a workflow engine. Its differentiator is three-part:
- Semantic checkpoints: a compact, versioned representation of what the agent needs to continue, not a conversation dump.
- Independent environment revalidation: every checkpoint component is verified against the current environment before resume, with staleness propagating through the dependency graph.
- Provenance-aware state: every fact traces to its origin, so agent-reported progress is never self-certifying.
Not published to PyPI yet. Install from a clone:
uv venv
uv pip install -e ".[dev]" # library, CLI, and test tooling
uv pip install -e ".[mcp]" # adds the MCP server (optional)Two entrypoints are installed: continuum (CLI) and continuum-mcp (MCP server). The core library and CLI use only the standard library; the mcp extra is required solely for the server.
Minimal example, record and recover:
from continuum import EventType, Run, SQLiteStorage, project
store = SQLiteStorage("agent.db")
store.create_run(Run(run_id="run_4821", goal="Analyze 10,000 documents"))
store.append_event("run_4821", EventType.RUN_STARTED, {"goal": "Analyze 10,000 documents", "total": 10_000})
for i, doc in enumerate(documents):
analyze(doc)
store.append_event("run_4821", EventType.WORK_COMPLETED, {"doc": i})
# After a crash, a new process picks up exactly where it stopped:
state = project("run_4821", store.read_events("run_4821"))
print(state.progress.completed) # already done, not repeated
print(store.verify_events("run_4821").ok) # True, chain intact after the crashRun the proof yourself. These scripts are the primary evidence, verified end to end rather than described:
python examples/crash_recovery_agent.py # real process kill, real side effect
python examples/context_compaction.py # transcript lost, checkpoint survives
python examples/model_switch.py # Model A dies, Model B resumes safely
python scripts/mcp_smoke.py # real subprocess, real JSON-RPC trafficThe e2e-autonomy-test/ kit scripts a real invoice-batch task, a hard-kill mid-run, and a fresh resume session, then scores the outbox, ledger, and event chain out of band. Run 1 scored 7/7 mechanics against a real Claude Code session, and the autonomy half was observed (an agent used the tools unprompted, refused to re-send verified invoices, and surfaced the request_human verdict). Full walkthrough and the open questions are in references/quickstart.md and references/e2e.md.
CONTINUUM separates LLM context (temporary) from durable task state (permanent). Instead of saving conversation history, it constructs a semantic checkpoint, the minimum verified information required to continue.
The detailed explanation, the projection model, and the recovery context are in references/architecture.md.
| Capability | What it gives you |
|---|---|
| Semantic checkpoints | Compact, versioned, inspectable state, not a transcript dump |
| Idempotent action ledger | Refuses duplicate external side effects; surfaces uncertain ones for reconciliation |
| Environment revalidation | Every checkpoint component verified against the current world before resume |
| Provenance-aware state | Agent-reported progress is marked REQUIRES_REVIEW, never self-certifying |
| Recovery engine | Seven recovery modes with a deterministic, sealed next-action contract |
| Deny-by-default MCP server | Nine tools, read-only/mutating split, caller allowlist |
| Framework adapters | Generic Python, OpenAI Agents SDK, LangGraph, and LangChain integrations |
| Secure planning loop | Two-signal observation verification escalates high-risk branches to REQUIRES_REVIEW |
| Periodic revalidation | Environment re-checked on a schedule, catching mid-run drift within one cycle |
| Tamper-evident log | Hash-chained event log (32 event types) with integrity verification |
CONTINUUM adds two additive security extensions on top of the existing recovery and checkpoint substrate. They do not change resume, replay, or the existing crash-time revalidation path.
- Secure Planning Loop: observations (for example a perception of a UI
element) carry provenance and are verified by two independent signals
(
verified/unverified/contested). A plan branch gated on an observation is escalated toREQUIRES_REVIEWwhen it is high risk and the observation is not fully verified, or when an environment observation is contested. Verification decisions and branch resolutions are appended to the ledger asPERCEPTION_OBSERVEDandBRANCH_RESOLVEDevents. - Periodic Revalidation: reuses the recovery engine on a step interval (default 25) and on app switch, so mid-run environment drift is caught within one cycle instead of only at the next crash.
See docs/PROBLEM.md for the problem statement and honest scope, docs/RESULTS.md for results, and STATUS.md for the implementation status.
CONTINUUM is verified not just with mock unit tests, but against real LLM agents, live protocol boundaries, and hard process crashes.
- Claude Code (Opus 4.8) End-to-End Autonomy: Driven across multi-session invoice-processing batches with mid-run
SIGKILLhard process terminations. Resumed sessions cleanly queriedcontinuum_resume, routed side effects through the two-phase intercept/complete ledger, and scored 7/7 on mechanics. The agent refused to duplicate verified outbox writes and respected therequest_humansafety verdict. - Drift-Hardened Deduplication: Live agent testing revealed real-world prompt drift across sessions (argument field renames such as
targetvsoutbox_file, and relative vs absolute paths). This prompted the implementation of canonical path normalization and token-based fallback deduplication inActionLedger.claim(). - Gemini CLI and Kilo Code: Both third-party clients connected over stdio JSON-RPC and invoked tools against the live SQLite store, validating multi-agent co-existence and authorization isolation.
- Stdio Protocol Compliance: Verified with
@modelcontextprotocol/inspectorin--climode driving real subprocess JSON-RPC 2.0 lifecycles across process deaths. - Deny-by-Default Security: Mutating tools require explicit allowlisting (
CONTINUUM_MCP_MUTATING_CLIENTS), while read-only tools (validate,resume,list_actions) remain ungated. - Anti-Self-Certification: External agent claims written via MCP are signed with
Origin.EXTERNAL_AGENTprovenance and degraded toREQUIRES_REVIEW(safe: false), preventing an agent from validating its own unverified work.
- WAL Sidecar Auto-Recovery: Hard-killing a server process (
kill -9) can leave SQLite in an inconsistent state with orphaned-waland-shmsidecars. The MCP server startup incorporates single-retry self-healing that clears stale sidecars and reopens cleanly.
- 1038 tests passing, 9 skipped on Python 3.11, 3.12, and 3.13 (including unit,
hypothesisproperty-based, concurrency, and adversarial tests). - CONTINUUM-Bench:
continuum benchmarkexecutes in-process recovery benchmarks across five scenarios (process_crash,dataset_change,unknown_side_effect,partial_completion,early_crash), proving 0 duplicate work, 0 duplicate side effects, and automatic detection of stale environment dependencies.
The MCP server was audited end to end over the live stdio protocol, with every
tool result checked against the SQLite store rather than taken at its word. The
self-certification gate, the two-phase ledger, crash-mid-action reconciliation,
tamper-evidence, and deny-by-default authorization all held. Three defects were
found and fixed: environment drift was detected but did not invalidate state,
list_actions under-reported an interrupted row, and the WAL sidecar recovery
could delete committed transactions. Method, per-claim results, and reproduction
steps are in test.md.
CONTINUUM ships an MCP server so an agent can record progress, checkpoint, and route external side effects through the ledger without embedding the library:
uv pip install -e ".[mcp]"
CONTINUUM_MCP_MUTATING_CLIENTS=your-client-name continuum-mcpTen tools over stdio. Three are read-only (continuum_validate, continuum_resume, continuum_list_actions); seven mutate. Side effects are two-phase (claim, perform, complete), and mutating tools deny by default behind an allowlist. Agent-reported state is recorded with Origin.EXTERNAL_AGENT provenance and marked REQUIRES_REVIEW. Verification details, including crash recovery at startup and the end to end Claude Code test, are in references/mcp.md. The authentication limitation is covered in references/architecture.md (MCP server and Security sections), and the MCP narrative is in references/quickstart.md.
CONTINUUM plugs into agent frameworks without becoming one. Nine adapters ship in src/continuum/adapters/ (one in-process facade plus eight integrations), all optional installs so the core stays standard-library-only:
| Adapter | Class | Notes |
|---|---|---|
| Generic Python agent | GenericAgentAdapter |
In-process facade; writes trusted (Origin.DETERMINISTIC) state. |
| Filesystem sandbox | FilesystemSandboxAdapter |
Local directory sandbox, no external service, default for docs and CI. |
| Python in-process | PythonInProcAdapter |
Runs Python in a temp workdir, records via ledger. |
| Container | ContainerAdapter |
Docker backed, guarded skip when docker is absent. |
| Browser | BrowserAdapter |
Playwright backed, guarded skip when not installed. |
| Kubernetes | KubernetesAdapter |
kubectl backed, guarded skip when not configured. |
| OpenAI Agents SDK | OpenAIAgentAdapter |
Experimental. Hooks ToolContext / RunHooks; optional openai-agents. |
| LangGraph | LangGraphAgentAdapter |
Experimental. Wraps a StateGraph; optional langgraph. |
| LangChain | LangChainAgentAdapter |
Experimental. Drops checkpoint_node into an LCEL Runnable pipeline and the create_agent tool-calling loop; optional langchain. |
Each adapter records progress and side effects through the ledger and routes external effects through the two-phase intercept/complete protocol. The framework adapters are newer than the generic facade, but each now has end-to-end integration tests (tests/test_integration_langgraph.py, tests/test_integration_langchain.py, and tests/test_integration_langchain_agent.py for a real create_agent tool-calling loop) covering checkpoint durability, exactly-once side effects, and crash-after-checkpoint resume. All three framework adapters (LangChain, LangGraph, OpenAI Agents SDK) have now
been driven against a live OpenRouter model (examples/langchain_real_llm.py,
examples/langgraph_real_llm.py, examples/openai_real_llm.py; recorded in
STATUS.md), where the runs surfaced and then closed an LLM argument-drift dedup gap
via an explicit idempotency key and two OpenAI-adapter schema/context bugs. Each
adapter also has a examples/*_real_llm_crash.py harness that proves the
hard-crash contract: a mid-side-effect os._exit(137) leaves the side effect
uncertain and blocks resume until a human reconciles it. examples/multitool_real_llm.py
is a richer live demo where one prompt orchestrates lookup, notify, and ticket tools
through the LangGraph adapter, showing exactly-once survives the model's argument
drift. Treat them as experimental until their adapter-specific tests cover the full
crash and resume matrix. Full usage, with runnable examples for every adapter, is in
references/adapters.md.
All three framework adapters were driven against a live gpt-4o-mini through
OpenRouter (key from OPENROUTER_API_KEY, never written to disk). Each was proven
two ways: a soft resume (exactly-once side effect across a second clean invocation)
and a hard crash (os._exit(137) mid-side-effect, then a fresh process asserts the
run is blocked as uncertain). A richer examples/multitool_real_llm.py demo has one
prompt orchestrate lookup_order + notify_customer + create_ticket through the
LangGraph adapter.
| Adapter | Soft resume (exactly-once) | Hard crash (resume blocked) |
|---|---|---|
| LangChain | PASS, 1 side effect, resume safe |
PASS, request_human, 1 uncertain |
| OpenAI SDK | PASS, 1 side effect, request_human* |
PASS, request_human, 1 uncertain |
| LangGraph | PASS, 1 side effect, resume safe |
PASS, request_human, 1 uncertain |
* The OpenAI adapter yields request_human even on a clean soft resume because it
records Origin.EXTERNAL_AGENT: an agent must not self-certify its own unverified
work. That is expected and safe. LangChain and LangGraph use Origin.DETERMINISTIC
and resume cleanly.
Two OpenAI-adapter bugs that only surface with a real model were found and fixed:
the tool JSON schema was emitted with no type key (OpenRouter rejected it), and the
context parameter was dropped from the inspectable signature, which bypassed
interception and let the side effect fire twice. The live runs also confirmed the
idempotency lesson: a stable business key (for example ticket:O-9) is required,
because a key derived from the model's rendered arguments does not dedupe the model's
argument drift and produced a duplicate ticket. Full run logs are in STATUS.md.
State reported over MCP, or through the OpenAI adapter, is recorded with Origin.EXTERNAL_AGENT provenance, which the validator marks REQUIRES_REVIEW. That is intentional: an agent must not validate its own unverified work. The consequence is that such runs resolve to request_human on continuum resume until a human has eyeballed them.
Runs started through the LangGraph or LangChain adapter use Origin.DETERMINISTIC provenance (the adapter is the orchestrator starting the run on CONTINUUM's behalf), so a consistent run resumes (RESUME) without a human in the loop.
To clear that review and resume, confirm the run as the operator:
continuum confirm <run_id> # records REVIEW_CONFIRMED, then re-assesses
continuum resume <run_id> # now reports RESUMEOver MCP the equivalent is the continuum_confirm tool followed by continuum_resume. Confirmation is a one-time, human-attested event; it is the escape hatch for the self-certification safety so an externally-driven run is never permanently stuck.
The deep reference for each concept lives in references/concepts.md.
- Semantic Checkpoints - a compact, versioned representation of what the agent needs to continue.
- State Validation - every component independently verified; staleness propagates through the dependency graph.
- Idempotent Action Ledger - external side effects tracked and de-duplicated; uncertain outcomes raise instead of silently retrying.
- Recovery Modes -
RESUME,REPAIR_AND_RESUME,ROLLBACK,WAIT,REQUEST_HUMAN,ABORT(plusREPLAN). - Recovery Contract - a deterministic, integrity-sealed, gated next action.
The system is built on immutable Pydantic v2 models with a cryptographic hash chain. State is projected from an append-only event log by a pure fold, not stored and mutated. The full reference, including the data model, event log, projection, extraction, versioning, durable storage, checkpointing, recovery context, state validation, action ledger, recovery engine, and security model, is in references/architecture.md. A complete system diagram and enumerated reference (tools, recovery modes, policies, reconcilers) is in references/architecture-diagram.md.
Key guarantees: append-only events, atomic sequence allocation, durability on append_event return, write races fail loudly, and corruption is refused rather than returned.
CONTINUUM is one library (src/continuum, about 80 Python files) plus a large test suite (68 files, 1038 tests). The modules are layered and all append to and replay one hash-chained event log:
| Module | LOC | Role |
|---|---|---|
events.py |
391 | Append-only, hash-chained event log and verify() |
state/ |
1,637 | Projection (semantic.py), validation (validator.py), extraction |
storage/ |
1,690 | SQLiteStorage (v2 schema), postgres.py, migrations.py |
actions/ |
1,183 | Idempotent action ledger, reconciliation, claim/complete |
checkpoint/ |
924 | Policy-driven checkpoints |
recovery/ |
699 | Engine (max-severity wins), planner, sealed contract |
adapters/ |
1,596 | Generic, LangChain, LangGraph, OpenAI Agents SDK |
mcp/ |
1,334 | Ten stdio tools plus authz.py (token auth, allowlist) |
serve/ |
739 | Language-agnostic newline-JSON sidecar mirroring MCP |
cli/ |
1,218 | argparse commands, exit codes as verdict |
benchmark/ |
440 | CONTINUUM-Bench scenario harness |
environment/ |
514 | Snapshots and diffs |
security/ |
608 | Provenance, trust gate, revalidation (in progress) |
interchange/ |
312 | B4 portable recovery-state JSON envelope |
concurrency/ |
255 | B2.2 lease and distributed-lock coordinator |
plugins/ |
174 | Registry and capability seams |
models.py, observability.py, __init__.py |
~1,100 | Shared models, metrics, public surface |
Three entry points, all from main: the continuum CLI, the continuum-mcp server, and the continuum serve sidecar. storage/, state/, adapters/, mcp/, cli/, actions/, and checkpoint/ hold roughly 72% of the core and are the mature, heavily-tested layers. security/, storage/postgres.py, migrations.py, and concurrency/ are committed but newer: the Postgres backend is unverified against a live server in this environment (its tests skip without CONTINUUM_TEST_POSTGRES_DSN / psycopg). interchange/ is done and tested. The full data model, projection, and recovery reference is in references/architecture.md.
Python surface (EventType, Run, SQLiteStorage, diff_states, project) and the adapter API are documented with runnable examples in references/api.md. The CLI is the same surface in shell form:
continuum runs # list runs
continuum inspect <run_id> # semantic state
continuum validate <run_id> --env dataset=v4 # validate, read-only
continuum resume <run_id> --env dataset=v4 # recovery decision + contract
continuum checkpoint <run_id> # force a checkpoint, mutates
continuum actions <run_id> # external side effects
continuum show-contract <run_id> # the machine-readable contractEvery command accepts --json, and the read-only commands never write, so they are safe against a live database while an agent is mid-run. Exit codes are a safety contract (only a verified-safe run exits 0). The full command list, exit-code table, and state-diff output are in references/cli.md.
| Phase | Component | Status |
|---|---|---|
| 1-11 | Data models, semantic state, persistence, checkpointing, validation, action ledger, recovery engine, CLI, crash-recovery examples, environment snapshots/diffs, framework adapters | Complete |
| 12 | Benchmark suite (CONTINUUM-Bench) | Complete (minimal harness) |
| 13 | Cloud API (FastAPI + PostgreSQL) | Planned |
| 14 | Dashboard | Planned |
Beyond the original plan: the MCP server, MCP authorization layer, provenance and anti-self-certification, community files, schema versioning, and a bounded recovery context are shipped. The design for CONTINUUM-Bench is in references/bench.md. See STATUS.md for the verified-vs-believed breakdown and open correctness bugs.
| Not this | This instead |
|---|---|
| An LLM | A reliability layer for agents that use LLMs |
| An agent framework | A recovery layer that plugs into any framework |
| A vector database | Structured semantic state, not embeddings |
| A RAG system | Verified checkpoints, not retrieval-augmented memory |
| A workflow engine | A recovery layer, not an orchestrator |
The core abstraction: semantic state + environment validation + action reconciliation = safe recovery.
CONTINUUM sits at the overlap of durable execution, idempotent side-effect tracking, and crash recovery for LLM agents. The surrounding literature is mostly engineering writing, with a few recent preprints that examine the same failure modes directly.
- Idempotency keys. The standard "do not do it twice" mechanism for external systems. See Stripe's idempotent requests and the AWS Lambda Powertools idempotency utility.
- Transaction outbox pattern. Write intent and effect record in one durable step, then dispatch, so a crash cannot lose an in-flight side effect (Chris Richardson's write-up).
- Saga pattern and compensating actions. A sequence of local steps where each has a semantic undo, so a failure can be repaired without an ACID rollback. Relevant to CONTINUUM's
COMPENSATEDaction state and dependency-safe repair (saga pattern). - Durable execution engines. Temporal, Restate, and DBOS persist a journal of completed steps and replay it for exactly-once semantics across crashes and redeploys.
- Anthropic, Building Effective Agents (2024). Workflow and orchestration patterns that frame agents as stateful processes worth making durable (research post).
Recent preprints that measure or model the same reliability gaps CONTINUUM targets (all arXiv links verified live):
- Khan, Resume Means Resume: A Machine-Checked Conformance Contract for Checkpoint, Interrupt, and Resume Semantics in Workflow Persistence Layers, arXiv:2608.03836 (2026). Proves a reference resume contract in TLA+ and measures that widely deployed frameworks re-execute durably recorded work after a real SIGKILL and cannot resume after a mid-node crash, the exact defects CONTINUUM's ledger and recovery gate exist to prevent.
- Chang, Geng, and Chang, Mnemosyne: Agentic Transaction Processing for Validating and Repairing AI-generated Workflows, arXiv:2607.00269 (2026). Treats generated actions as untrusted proposals admitted only against a declared constraint set, with an append-only transition log and dependency-safe compensation. Close to CONTINUUM's deny-by-default admission and provenance model.
- Liu, Zhao, Shang, and Shen, Dive into Claude Code: The Design Space of Today's and Future AI Agent Systems, arXiv:2604.14228 (2026). Finds that most agent code is operational infrastructure (context management, permission systems, append-oriented session storage) rather than model logic, the layer CONTINUUM lives in.
- Tavori, Bremler-Barr, Levy, and Lavi, RetryGuard: Preventing Self-Inflicted Retry Storms in Cloud Microservices Applications, arXiv:2511.23278 (2025). Shows default retry patterns amplify cost and load under failure, motivating global retry budgets rather than per-call loops.
- Tested: 1038 tests passing, 9 skipped (a few skip without optional services such as Postgres; see STATUS.md). The MCP surface has also been audited adversarially over the live protocol; see test.md.
- Not on PyPI. Install from a clone (see Quick Start).
- MCP caller authentication is optional. When
CONTINUUM_MCP_TOKENis set, the server refuses every mutating tool unless the caller presents that shared secret in theinitializehandshake's_meta.authToken. Without it, authorization is by declared identity only (the historical default, preserved for local single-user use). Tracked as #1. - Unbuilt components: Cloud API (Phase 13) and Dashboard (Phase 14).
- Framework adapters are experimental. The OpenAI Agents SDK and LangGraph adapters are newer than the generic facade and do not yet carry the same crash-and-resume verification coverage. Prefer
GenericAgentAdapterfor production recovery until their adapter-specific tests cover the full recovery matrix. - Agent/MCP runs need an explicit confirm before auto-resume. Because externally-reported state is
REQUIRES_REVIEW,continuum resumereturnsrequest_humanuntil a human runscontinuum confirm <run_id>(or the MCPcontinuum_confirmtool). This is by design, not a bug; see Framework Integration. - e2e autonomy test series (issue #6): Three full Claude Code runs scored 7/7 mechanics with unprompted recovery behavior observed. Defensive token-based fallback and path normalization bridge argument drift. Further test iterations across diverse prompt styles remain open.
For a full account of what is verified, believed, and neither, see STATUS.md. The current set of open correctness bugs (a 2026-08-12 code audit) is tracked there.
Contributions are welcome. This project is open source under Apache 2.0 and deliberately built to be extended: by researchers validating the recovery semantics, by engineers porting the ledger or MCP server to other frameworks or languages, and by anyone turning the planned roadmap into reality. A good place to start is the good first issue label on the issue tracker, or the open correctness bugs listed in STATUS.md.
Open an issue before submitting large PRs. See CONTRIBUTING.md for the full contribution guide, including the Code of Conduct.
Apache 2.0 - see LICENSE.
Deep reference material:
- references/concepts.md - semantic checkpoints, validation, ledger, recovery modes, contract
- references/architecture.md - data model, event log, projection, storage, checkpointing, recovery engine, security, project structure
- references/api.md - Python and adapter API
- references/adapters.md - framework adapter usage (Generic, OpenAI, LangGraph, LangChain) with runnable examples
- references/cli.md - full CLI command list, exit codes, state diff
- references/quickstart.md - install, examples, the proof scripts
- references/e2e.md - end to end autonomy test walkthrough
- references/mcp.md - MCP server status, verification, open questions
- references/bench.md - CONTINUUM-Bench design





