Repository navigation
Replies: 3 comments
|
Branch is now public: 298 files, +7,707/−1,235, based on |
0 replies
|
Contract point 3 leaves the unknown-outcome receipt for the model to resolve, and in practice the model either re-runs the call or asks a user who can't see the external system either. If an unsafe tool could declare a probe keyed on the durable intent id, passed to the external side as its idempotency key, recovery could settle most unknowns to done or not-done before the model ever sees the receipt. |
0 replies
|
I maintain Call4Agents, a small Python library that keeps the same intent-before-effect record. I tested contract point 3 and renezander030's question with real process kills. The setup is one non-idempotent call to an outside system (a file with one line per real effect). The agent is killed with
Two things that may help:
The test is Python, but the crash windows are the same as in the TypeScript module. Script (standard library + call4agents 0.6.0)"""Test for deepseek-ai/deepseek-harness discussion 11762 (pi-durable: intent before policy and body,
one settlement, unknown-outcome receipt for replay:'unsafe' tools).
One non-idempotent tool call against an outside "API" (a file; one line per real effect). The agent process is
killed with os._exit at one of three points, then a new process recovers:
P after the intent is durable, during the policy step (body not started)
B body started, before the outside call
C after the outside call, before the settlement is written
Setups:
intent-only the proposal's shape: durable intent, in-memory body start; recovery turns an unanswered
intent into an unknown-outcome receipt (nothing re-runs)
+body-mark same, plus a durable "body started" record before the body; an intent without it settles as
"not run" and the call runs once on recovery
+key published call4agents 0.6.0, idempotent_for: the intent's key goes to the outside API, which
drops a repeat with the same key; recovery re-sends with the same key
Counts real effects in the outside system after recovery, N runs per case.
PYTHONPATH=<path to published call4agents 0.6.0> python dsh_11762_intent.py
"""
import json
import os
import subprocess
import sys
import tempfile
from pathlib import Path
def api_post(work: Path, key: str, body: str) -> str:
"""The outside system: applies the effect once per key (an empty key is never deduplicated)."""
p = work / "effects.txt"
lines = p.read_text().splitlines() if p.exists() else []
if key and any(l.split(" ", 1)[0] == key for l in lines):
return "duplicate ignored"
with open(p, "a") as f:
f.write(f"{key or '-'} {body}\n")
f.flush()
os.fsync(f.fileno())
return "applied"
def effects(work: Path) -> int:
p = work / "effects.txt"
return len(p.read_text().splitlines()) if p.exists() else 0
def log(work: Path, rec: dict) -> None:
with open(work / "session.jsonl", "a") as f:
f.write(json.dumps(rec) + "\n")
f.flush()
os.fsync(f.fileno())
def records(work: Path):
p = work / "session.jsonl"
return [json.loads(l) for l in p.read_text().splitlines()] if p.exists() else []
def agent(work: Path, setup: str, kill: str) -> None:
if setup in ("intent-only", "+body-mark"):
log(work, {"t": "tool/call", "id": "call-1"}) # intent, before policy and body
if kill == "P":
os._exit(17) # e.g. a policy/approval step
if setup == "+body-mark":
log(work, {"t": "tool/body-started", "id": "call-1"})
if kill == "B":
os._exit(17)
api_post(work, "", "send report") # no key reaches the outside system
if kill == "C":
os._exit(17)
log(work, {"t": "tool/result", "id": "call-1", "ok": True})
return
import call4agents as c4a
c4a.init(str(work / "ledger.sqlite"), secret=b"0123456789abcdef-test")
if kill == "P":
os._exit(17) # c4a writes nothing before the call itself
def send(body: str) -> str:
if kill == "B":
os._exit(17)
r = api_post(work, c4a.idempotency_key(), body)
if kill == "C":
os._exit(17)
return r
tool = c4a.effect(send, name="send", idempotent_for=86400.0)
with c4a.run("turn-1"):
tool(body="send report")
def recover(work: Path, setup: str) -> str:
if setup in ("intent-only", "+body-mark"):
recs = records(work)
called = any(r["t"] == "tool/call" for r in recs)
settled = any(r["t"] == "tool/result" for r in recs)
started = any(r["t"] == "tool/body-started" for r in recs)
if not called:
api_post(work, "", "send report")
return "never recorded: ran"
if settled:
return "settled"
if setup == "+body-mark" and not started:
api_post(work, "", "send report")
log(work, {"t": "tool/result", "id": "call-1", "ok": True})
return "not started: ran once"
return "unknown-outcome receipt"
import call4agents as c4a
c4a.init(str(work / "ledger.sqlite"), secret=b"0123456789abcdef-test")
def send(body: str) -> str:
return api_post(work, c4a.idempotency_key(), body)
tool = c4a.effect(send, name="send", idempotent_for=86400.0)
try:
with c4a.run("turn-1"):
return "settled: " + tool(body="send report")
except c4a.EffectInDoubt:
return "in doubt"
def one(setup: str, kill: str) -> dict:
with tempfile.TemporaryDirectory(ignore_cleanup_errors=True) as d:
work = Path(d)
first = subprocess.run([sys.executable, __file__, "agent", d, setup, kill]).returncode
rec = subprocess.run([sys.executable, __file__, "recover", d, setup], capture_output=True, text=True)
return {"first_exit": first, "recovery": rec.stdout.strip() or rec.stderr[-200:], "effects": effects(work)}
if __name__ == "__main__":
if sys.argv[1:2] == ["agent"]:
agent(Path(sys.argv[2]), sys.argv[3], sys.argv[4])
sys.exit(0)
if sys.argv[1:2] == ["recover"]:
print(recover(Path(sys.argv[2]), sys.argv[3]))
sys.exit(0)
n = int(os.environ.get("N", "10"))
out = {}
for setup in ("intent-only", "+body-mark", "+key"):
for kill in "PBC":
runs = [one(setup, kill) for _ in range(n)]
out[f"{setup} | kill {kill}"] = {k: sorted({json.dumps(r[k]) for r in runs}) for k in runs[0]}
import call4agents
print(json.dumps({"call4agents": call4agents.__version__, "python": sys.version.split()[0],
"runs_per_case": n, "cases": out}, indent=1)) |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
pi-durable: crash-consistent tool execution for DeepSeek Harness
Patch: https://gist.github.com/wzsxb233/02c73d5c145b0edb4a520d1e76abcb53 (2,284,417 B, sha256
aea9850b…c7eef)Baseline:
0.2.1-rc.2(687f282e), applies cleanly withgit apply --checkMotivation
A long-running agent turn can be interrupted at any point: process kill, OOM, laptop sleep, container eviction. Today the session log records the model-visible facts, but a tool call that was dispatched and never settled leaves the next owner guessing. The two failure modes we kept hitting:
tool/call, and the model has no way to know whether the side effect happened.run_codeprogram that spawned nested tool calls is re-run wholesale after a crash, repeating side effects that already succeeded.Both are reliability problems, not feature gaps. They show up exactly where Harness is being positioned: long-horizon, multi-tool, real-environment work.
Changes
298 files, 14 of them new. The core is 4 new source modules —
packages/core/agent-loop/src/durable-task.tspackages/core/tools/src/ptc-journal.tspackages/core/tools/src/search.tspackages/core/session/src/task-graph.ts— plus three specs, 15 files in
packages/core/tools, 9 inagent-loop, 9 inbundle/headless, 8 inllm/llm-pi-ai, 7 incore/session, 5 insession-checkpoint-policy, 29 docs, and 191 expected-snapshot updates (session format v4).Seven published packages are touched. Everything below is additive to the existing session/checkpoint model — no second journal, no parallel persistence path.
dsh-agent-loopsrc/durable-task.ts: each top-level tool call is a Session-backed durable task. Intent is committed beforeprepare()or the tool body; exactly one model-facing receipt is written after settlement. Resume resolves the current scoped replay policy inside the initialization maintenance barrier, before closing the interrupted step/turn. Cancellation preserves unanswered intents for the next owner.dsh-toolsreplay: 'safe' | 'unsafe'on a tool definition — carried into the durable task and the recovery guidance; (b)callers: ['model'] | ['tools']capability split, with wrong-class calls rejected asUNKNOWN_TOOL; (c)exec.executeTool()nested child calls through the same validation/policy/cancellation pipeline, drained before the parent settles; (d)src/ptc-journal.ts— pure fold of PTC start/settle records that rejects orphaned, duplicate and divergent receipts before replay; canonical binding values enter durable replay receipts; (e)src/search.ts+ctx.tools.searchTools()— deterministic BM25 ranking for deferred/codemode tool discovery.dsh-session-checkpoint-policydsh-sessionwatch/closed) and recovery-result plumbing used by the above.dsh-headlessdsh-llm-pi-ai@earendil-works/pi-ai^1.1.0→^1.2.0.dsh-tool-cordisOne deletion worth calling out
dsh-llm-pi-aino longer shipspatches/@earendil-works__pi-ai@1.1.0.patch. pi-ai 1.2 owns provider-side partial JSON parsing, so the vendored O(n²) re-parse fix is retired upstream rather than maintained here. Net: one fewer vendor patch for you to carry.Behavior contract (what a reviewer should check)
tool/call(or the durable intent record) is flushed beforeprepare()and before the body. A crash can therefore never produce an unrecorded side effect.replay: 'safe'tools are re-run on resume. Unsafe/undeclared tools get an unknown-outcome receipt that tells the model to verify external state or ask — the conservative default is unchanged.Testing
Recorded on macOS arm64 / Node 22 from the prepared release tree. The raw logs ship with the bundle and every command below is re-runnable on your side; we are not asking you to trust a summary.
pnpm testfull suite —full-final.log; patch applies cleanly on687f282e(git apply --check)packages/session/session-checkpoint-policy/tests/crash-recovery.e2e.tsand the new durable-task cases) —recovery-tests*.logjournal-checkpoints.logsnapshots-qualified.loglint-*.log,hygiene.logrelease-pack-final.log,built-runtime-smoke-final.log,live-web-smoke.jsonclosure-audit.json)Proof (excerpts)
Rebase status (measured 2026-11-01)
We tried to rebase onto
masterbefore posting this, and we are reporting the result instead of hiding it.masterdrift from0.2.1-rc.2git rebase origin/master.ts)durable-task.ts,ptc-journal.ts,search.ts,task-graph.ts)Three master-side changes make this a port, not a rebase:
packages/core/tools/src/{ptc,py-types,ts-types,types}.tsmoved topackages/core/tool-executor-ptc/. The PTC journal slice must move with it.agent-loop/src/tool-calls.tswas removed — its logic is now insideagent.ts; the durable-task integration point has to be re-anchored there.^1.3.0on master, with its ownpatches/@earendil-works__pi-ai@1.3.0.patch— our "1.1.0 → 1.2.0 + delete the vendor patch" slice is superseded and should be dropped from any master-targeted version.So this branch targets the
0.2.1-rc.2tag on purpose: it is a reproducible release point, and cherry-picking onto master is your call. If you would rather review a master-based version, say so and we will do the port (drop the pi-ai slice, move PTC intotool-executor-ptc, re-anchordurable-taskinagent.ts, regenerate the 191 snapshots).Compatibility & risk
replay. No silent re-execution.callers,replay), one new context method (searchTools), one nested execution helper (exec.executeTool). All opt-in.What we're asking for
We know CONTRIBUTING says external PRs aren't accepted right now. We're not asking you to merge a 2.3 MB patch blind. Three options, cheapest first:
durable-task.ts+ replay policy, (b) PTC journal, (c) nestedexecuteTool+ caller classes, (d)searchToolsBM25, (e) pi-ai 1.2 bump. Each slice is independently reviewable and independently revertable.We're also happy to sign whatever contribution terms you need, and to be listed as the origin of the design if that helps.
中文摘要（给中文读者）
这个补丁解决的是长任务被打断后的正确性：工具调用在真正执行前先把"意图"写进会话日志，执行完只写一条回执；崩溃后恢复时，只有声明了
replay: 'safe'的工具才会重跑，其余给模型一条"结果未知，请先核实外部状态"的回执。嵌套调用（PTC 里的子调用）也走同一套检查点，因此"没有持久记录就不会产生副作用"在任意深度都成立。PTC 的日志可以按源码指纹+调用顺序+参数+规范化值确定性重挂，孤儿/重复/分叉的记录会被拒绝而不是重放。另外顺手把 pi-ai 升到 1.2 并删掉了我们自己的 vendor patch。改动 7 个包、2.28 MB patch、717 个 manifest 的闭包审计零缺失。全部是增量改动，不引入第二套日志，不改默认行为。
我们要的不是"求合并"，而是三选一：指定通道 / 按切片评审 / 直接交出设计由你们内部实现。
All reactions