Refactor/v0.1.0 runtime rewrite - #2
Merged
Merged
Conversation
…hard + DB) 2026-05-30 設計議論のまとめ。 6 ファイル (overview / value-and-streaming / runtime-architecture / storage-schema-and-api / implementation-plan / v0.2-streaming)、 decision log D1-D36。 observable streaming は v0.2 送り、 v0.1.0 は complete blob のみ。 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ec [Phase A TS] - Value: string/file/secret carry BytesRep (inline | ref); file is always ref. helpers mkString/mkSecret/inlineText/tryInlineString. - value-codec: $ref envelope (module,id,as,hash,size), as-is (no promotion), decode routing adds $ref. valueToRaw/From stay sync. - value-secret-codec / snapshot: rep-aware, VALUE_KIND_TAGS += file. - equality: string=content (hash/text), file=identity (module,id). - agentLiteral.snapshot deferred to Phase E. Tests still reference old value shape (green after Phase A完了). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
opaque byte sequence、 string と disjoint、 identity-typed (file==file は参照同一性)、 literal なし。 lexer KeywordFile → parser → SemanticTypeFile → ConstraintGenerator / NormalizedType (fileLayer) / Solver shapeKind / Render / Schema ($ref envelope schema)。 Exhaustive は catch-all で処理 (wildcard 必須)。 680 compiler tests green。 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the content-addressed byte-sequence store backing string/file/secret refs. Three layers per docs/2026-05-30-storage-schema-and-api.md §2: - value_refs ephemeral CORE/FFI values (reachability GC) - api_files persistent API-owned files (user-managed) - value_blobs+chunks project-wide content-addressed bytes (refcount dedup) `ValueStore` interface lives in katari-runtime (lower layer, so engine modules can consume it); katari-api-server provides the Postgres + memory impls and wires it onto the Storage facade (participates in withTransaction). - runtime: hashBytes/hashText (Blake3 via @noble/hashes, pure-TS), ValueStore interface + types, exported from index - produce: putComplete + open/pushChunk/close (host-buffered, re-chunked at close, no per-chunk DB write — D32); MAX_PRODUCE_BYTES cap - consume: getState / fetch / fetchRange (ref→hash→blob; module=api → file) - files: create/get/list/delete + persistRef (ephemeral→persistent promote sharing the blob); GC primitives sweepInstance / sweepUnreachable - dedup via value_blobs.ref_count; blob swept at 0 Schema additions are additive (existing tables untouched); the project-scoping + engine_shards migration lands with the Phase E consumer rewrite. Data plane HTTP routes follow next. ValueStore unit tests: 12 passing (produce/consume/dedup/range/file CRUD/ persist/sweep/tx rollback). Full TS workspace builds green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Katari Protocol data plane per docs/2026-05-30-storage-schema-and-api.md §4.2.
Mounted at /project/:projectId/value:
GET /:module/ref/:id full bytes
GET /:module/ref/:id?range=N-M partial bytes (also honours `Range:`)
GET /:module/ref/:id/state metadata (state / hash / size / contentType)
Read-only by design — production is module-internal (FFI sidecar / in-process),
not exposed here. module ∈ {core, ffi, api}; api ids resolve through api_files.
errored refs → 409, unknown → 404, ranges → 206 + Content-Range. Path carries
no snapshot (a value's owner is a module, not a code version — D24). Auth rides
the app-wide bearer for now; per-module sidecar tokens land in Phase C.
Integration tests: 7 passing (fetch / range query / Range header / state /
api file / 404 / 409).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e C] The handler-facing `katari.value` namespace. A byte-sequence arg arrives inline (bare JSON string) or as a `$ref` envelope; this client erases the difference so a handler writes `await katari.value.text(args.history)` whether history is a 3-word inline string or a 200 KB ref pulled over the data plane (docs §4.2 / Phase C deliverable #2 consume half). fetch(v) → Uint8Array (inline encode | data-plane GET) text(v) → string (UTF-8 decode) fetchRange(v,o,l) → Uint8Array (?range=o-end for refs; local slice inline) - $ref → GET {KATARI_PROTOCOL_URL}/project/{id}/value/{module}/ref/{id} with Bearer KATARI_PROTOCOL_TOKEN; config read lazily so inline-only handlers (and import-time tooling) need no protocol env - non-byte-sequence RawValue / missing env / non-2xx all throw clearly - injectable env + FetchLike for hermetic tests 9 unit tests (inline fast-path / $ref URL + auth / range / errors / guards). Produce side (put/open/persist) + sidecar env wiring + FfiModule + the 23-blob-echo e2e sample follow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The FFI/CORE produce surface (docs §4.3) — separate from the read-only data
plane (production is each module's own business, not the generic protocol):
POST /project/:p/value/:owner/produce bytes → ephemeral ref
POST /project/:p/value/:owner/ref/:id/persist ephemeral → persistent api file
owner ∈ {core, ffi}. Produce reads X-Katari-Semantic-Kind + Content-Type +
optional X-Katari-Owner-Instance, host-buffers the body, and writes via
ValueStore.putComplete (re-chunked at the store). persist promotes through
persistRef (shared blob). Returns a {module,id,hash,size,contentType?} shape
the produce client wraps into a $ref. Establishes the real contract the
katari-port produce client builds against next (mirrors how the consume
client followed the existing data plane).
Integration tests: 5 passing (produce→data-plane round-trip / semantic-kind +
content-type / persist + shared blob / 404 / non-producer owner 400). Full
api-server suite green (35).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the `katari.value` surface against the now-real produce-route
contract. A handler produces and returns a ref:
put(bytes, {as, contentType}) → $ref (POST .../{owner}/produce)
open({as}) → pushChunk* → close → $ref (host-buffered, one blob at close)
persist(ref, {displayName}) → $ref api (POST .../{owner}/ref/{id}/persist)
- owner from KATARI_SIDECAR_OWNER (produce only); persist targets the source
ref's own module (core/ffi) and yields a module=api file ref (as="file")
- produce response {module,id,hash,size,contentType?} is wrapped into a
`$ref` RawValue; abort discards the host buffer with no store call
- persist rejects non-ephemeral / non-ref inputs
6 more unit tests (produce URL+headers+body / SIDECAR_OWNER required / stream
concat / abort no-op / persist promote / guards) — 15 total in katari-port.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the metadata-only byte-sequence semantics ref-aware, no fetch:
- == / != on string|secret compare content hashes (inline reps hash their
UTF-8 text, refs carry their hash). Resolves the Phase-A "mixed inline/ref
requires materialize" throw — it never needed bytes, only the hash.
- match against a string literal compares the literal's hash to the
subject's, so a ref subject matches the same as an inline one.
- file == file stays identity (module, id) — already correct.
Centralised in engine/value.ts: bytesHash / bytesContentEqual / bytesEqualsText
(shared by prim equality + pattern match). These are pure metadata ops.
The fetch-requiring half of Phase D (materializeBytes + concat / format /
to_string / from_string on ref operands) needs the engine's async quantum,
which the runtime-architecture builds as part of the Phase E actor host — and
no refs reach the engine until snapshot promotion / FFI produce are wired. So
that half sequences with E; inlineText still throws loudly on a ref until then.
6 unit tests (inline==ref / ref==ref by hash / file identity / match literal
vs ref + inline). Full workspace builds green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The design doc the implementation-plan mandates before touching Phase E.
Turns runtime-architecture's durable-object concept (D12-D23) into a
concrete implementation plan against the measured current code.
Key findings driving the design:
- Module.feed/persist/load are ALREADY async → async-ifying the engine
is localised to drive/step/applyEvent + CoreModule.feed awaiting them
- State is flat → the shard split is a restructure of flat State into
per-agent EngineShard + project-local ProjectIndex (2-level routing:
id→ShardId in the index, id→ThreadId in the shard)
- Orchestrator.tick (cold per-request) → warm ProjectActor (serial loop)
Specifies: flat→shard/index field mapping, event→shard routing table
(incl. the escalate-routing porting risk, flagged test-first), async
quantum + materialize injection (where D-async lands), CoreModule
projectId+multi-snapshot rewrite, ProjectActor host, agentLiteral.snapshot,
crash recovery, 8 invariants, file impact list, open questions.
Staged rollout to de-risk: E0 async engine (D-async merges here, behaviour-
preserving) → E1 shard storage → E2 actor host → E3 multi-snapshot + sidecar
env (Phase C tail merges here). Both the Phase C and Phase D tails converge
on E, as noted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the engine deterministic-but-async so content-transform prims can fetch ref bytes (runtime-architecture §5). Confined to the create-path — the only place a prim is evaluated: executePrim → primOps.create → dispatchCreate → onCreate → step → drive → applyEvent (all async; the other 5 thread-ops + internal events stay sync, since they spawn work via the queue rather than evaluating prims) - StepCtx.materialize(rep): inline → resolved immediately, ref → injected RefFetcher (ValueStore-backed at the host). The fetch is a pure function of the content hash → the state transition stays deterministic. - concat awaits materialize for ref operands; inline operands stay a direct text join (no fetch — the common path stays cheap). secret taint preserved. - applyEvent(state, event, fetchRef?) — fetchRef defaults to throw-on-ref; CoreModule passes none yet (CORE state holds no refs until persist-time promotion lands in E1), so the ref path is dormant-but-wired. "Always async" over a sync-fast-path: Katari is an orchestration language (agents take a few steps then await LLM/IO), so per-prim microtask overhead is negligible vs the actual work — simplicity wins. Behaviour preserved: integration.test.ts green after adding awaits (delegate→ add→ack, missing-entry escalate, terminate cascade). New async-prim.test.ts (5) covers inline/ref/mixed concat + taint. Full build + api-server e2e (35) green. format/to_string/json keep inlineText (throw on ref) until refs reach them; same pattern applies when they do. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Solves the motivating problem — a large inline string (e.g. an accumulated AI conversation) was copied into the CORE checkpoint on every persist. Now CORE state persist promotes any inline string over a byte threshold (4 KiB default) to an owner=core value-store ref; the checkpoint carries only the small handle. This also makes E0's materialize non-dormant: refs now actually flow into the engine and content-transform prims fetch them. - snapshot.ts: promoteCheckpoint(checkpoint, promote, threshold) — an async structural walk that replaces large `string` inline reps with refs (secrets stay inline; already-ref strings pass through unchanged → stable id across persists; nested strings in arrays/records/tagged are promoted too) - CoreModule: projectId + valueStore injected; persist promotes before encrypt (disjoint: promotion=strings, encryption=secrets); feed threads a ValueStore-backed fetchRef into applyEvent so reloaded refs materialize - orchestrator-adapter: ResolvedSnapshot carries projectId; CoreModule wired with snapshot.projectId + tx.values (tx-scoped, so a rolled-back persist's blob writes roll back too) Observationally transparent (E-design invariant #8): == / match compare by hash (no fetch), concat materializes — so existing runs are unaffected. promotion test (3): large promotes / small + secret stay inline / nested / ref round-trip (== by hash + concat by materialize). Full build + api-server e2e (35) green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-async] With persist promotion live, a large string can now be a ref anywhere in CORE state — so every prim that READS string content (not just concat) must materialize it instead of throwing on `inlineText`. Without this, promoting a >4 KiB string and then to_string/json/from_string-ing it would break. - to_string / json.stringify: materializeValueDeep — recursively replace string refs with inline before valueToRaw / jsonTaggedToRaw, so the serialized JSON carries real content, not a `$ref` envelope (handles refs nested in arrays / records / tagged values) - from_string / json.parse: materialize the ref text arg before JSON.parse - get_field / record.get / set / remove / has: materialize the (rare) ref key - format stays a value passthrough (no text read → already ref-safe) This completes the D-async content-transform prim set begun with concat in E0. ref-prims test (6): to_string ref + nested array / from_string ref JSON / record.get + has ref keys / inline still works. Full build + api-server e2e (35) green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 4 test files deferred during the Phase A value-model rewrite, brought to
green so the runtime suite is a complete regression net again:
- value-codec / value-secret-codec / snapshot-secret: {kind:"string"|"secret",
value} → mkString / mkSecret (rep-based); redact assertion reads via inlineText
- prim-ops: executePrim is now async + takes a materializer — awaited with an
inline-only stub `M` (throws if a fetch is unexpectedly attempted); throw
cases use rejects.toThrow; applyEvent awaited
Full runtime suite green: 10 files, 72 tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The storage foundation for per-agent sharding (docs/2026-05-30-phase-e-actor- host.md §2/§4). A shard IS a State scoped to one agent instance and persists as an EncryptedEngineCheckpoint (the existing checkpoint codec, reused verbatim — no separate shard serialization) keyed by (projectId, shardId). - runtime engine/shard.ts: ProjectIndex (delegationId/escalationId → shardId routing table), ShardStatus, ShardStore + ProjectIndexStore interfaces. Documents the verified routing: an inbound escalate/ack returns to the shard that issued the delegate/escalate, so all 6 events route by index lookup. - api-server: PgShardStore / PgProjectIndexStore over engine_shards / project_index (Phase B schema), InMemory versions, wired onto the Storage facade (participate in withTransaction). shard-store test (6): checkpoint round-trip / listActive status filter / delete / project scoping / index round-trip / facade tx rollback. Build + api-server suite (41) green. The CoreModule rewrite to use these (split flat State → per-shard, index-based routing, on-demand load) follows next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Split the flat per-snapshot State (which held ALL agents of a snapshot in one State) into per-agent-instance shards + a project-local routing index. A shard IS a State scoped to one agent; the engine's applyEvent runs unchanged on it. This is the memory/IO win — an event loads only the touched shard, not the whole snapshot's state. CoreModule now: - routes each event to its shard via the index (delegate → new shard; every ack / terminate / escalate → the issuing shard, an O(1) index lookup — the escalate path was verified against the engine's pendingDelegateOut routing) - loads shard bodies on demand in feed() (the tx-scoped ShardStore / ProjectIndexStore are injected at construction, since feed gets no tx), caches them for the tick, persists dirty shards (with per-shard promotion) and deletes completed ones (threadCount 0 → no replay → no retention) - reconciles the project index from each shard's local maps after applyEvent CoreTx collapses to empty (stores are held); the orchestrator's checkpoints plumbing and CoreCheckpointStore are removed. ResolvedSnapshot.projectId feeds the per-project shard scope. Validated end to end: api-server e2e (41) drives multi-agent CORE→CORE delegation through per-shard states + index routing; integration.test asserts the index fully purges once both agent shards complete. Full build + all suites green (runtime 72 / api-server 41 / port 15). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lock in the host / bus / Module responsibility split from the design discussion (§14): - Module = independent entity: implements the katari-protocol (feed) AND owns its domain features as methods (ApiModule.startRun / uploadFile / ...). It opens its OWN tx (1 quantum = 1 tx) and self-serializes. Warm (resident), holds root storage. - bus = pure router (dispatch by event.to). No tx / lock. - host = a thin proxy for each Module: external trigger → Module method call, plus driving the bus. No tx, no serialization. The old "1 request = 1 tx + 1 snapshot lock wrapping the whole drain" is dropped (the coupling §1 criticised). Serialization becomes CORE's own per-project mutex (in-memory); per-shard concurrency (v0.2) is then a CORE-internal mutex-granularity change, not a host change. Module interface simplifies to feed() only (each self-tx); load/persist phases removed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…) [Phase E3]
snapshot is a CORE/FFI-private axis, so it must NOT ride the katari protocol
(bus events). Instead it's carried INSIDE the (bus-opaque) agent def id, on the
ONE identifier that loads versioned code on the receiver: the delegate target.
Rule (confirmed in design): an id carries the snapshot iff it loads
snapshot-dependent code at runtime = a delegate target's agent def id (CORE
agent runs versioned IR; FFI ext picks the per-snapshot sidecar). Everything
else stays a bare qname — interpreted within an already-running shard whose
snapshot is fixed:
- request id (escalate) — dispatched in the handler shard's snapshot
- constructor id / $constructor — a runtime data tag (qname string + fields),
no snapshot lookup, like number/string
- primitive.throw (escalate) — protocol-common, snapshot-free
- closures — run in their captured scope, snapshot-independent
And NOT delegations to ENV / API — those modules are snapshot-independent.
- agent-def-id: CoreAgentDefId.qname / FfiAgentDefId carry optional snapshot,
encoded `qname@<snapshot>` (`@` ∉ qname/UUID/closure; compiled schema /
get_metadata stays bare). CORE and FFI qname encodings are identical.
- CoreModule: stamps the issuing shard's currentSnapshot on outbound delegates
to CORE/FFI only (ENV/API left bare; closures inherit); decodes the inbound
delegate's snapshot → the new shard's currentSnapshot; resolves IR via getIR
(per-snapshot, defaults to the single irModule). Tracks currentSnapshot per
shard; persists it. ShardStore.get returns it on load.
- ApiModule.startRun stamps the run's snapshot on the root CORE delegate.
agent-def-id test (6) + shard-store/get round-trip. Full build + all suites
green (runtime 78 / api-server 41 / port 15).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The snapshot rides inside the agent def id (CORE/FFI-private, opaque to the bus). FFI is the boundary where that id meets a sidecar whose handler registry is keyed by the BARE qname (the sidecar already IS the right snapshot's code), so: - inbound CORE→FFI delegate (`ext.tool@snap`) → strip → `ipcDelegate ext.tool` (store row keeps the bare form so recovery's ipcDelegateRestarted matches) - ext-spawned CORE child (`some.agent`) → stamp → `delegate some.agent@snap` so CORE creates the child shard on the matching IR version throw / escalate stay bare (requests, not delegate targets). The three stamp helpers (agentDefIdSnapshot / stripAgentDefIdSnapshot / stampAgentDefIdSnapshot) move to agent-def-id.ts so CORE + FFI share one place that knows "the snapshot lives inside the id". runtime 81 / api-server 41 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… WIP] Replace the per-snapshot Orchestrator with a warm per-project actor model (user's confirmed Module-self-contained design): - Module interface = feed() only (drop host-driven load/persist) - CoreModule: warm per-project actor — holds shard cache + routing index + IR memo in memory across quanta, opens its OWN tx per feed via a CoreStorage provider (1 quantum = 1 tx), async getIR over snapshots. Snapshot is CORE-private (shard.currentSnapshot), never on the protocol delegations row. - ProjectActor / ProjectActorHost: warm actor registry + per-project serial queue. Thin — no tx, no lock. Bus stays a pure router. - FfiMux: per-project multiplexer over per-snapshot FfiModule lanes (a sidecar is per-snapshot; route delegate by stamped snapshot, others by FFI-private delegation/escalation→snapshot lookup). - modules/storage.ts: CoreStorage / CoreTxStores / EnvStorage providers (runtime→host hand-off so modules open their own tx). - EnvModule / FfiModule: drop load/persist from the Module interface (FfiModule.load() stays as a lane method the mux drives). - Delete runtime orchestrator/ (replaced by actor/). runtime build + 81 tests green. NOTE: api-server still imports the removed Orchestrator — workspace build is RED until the api-server migration (next commit) lands the host wiring + ApiModule warm + schema project-scope. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…otocol [Phase E] Migrate api-server onto the runtime's warm per-project actor. The per-snapshot Orchestrator (tick + advisory lock + AsyncLocalStorage tx threading) is replaced by ApiServerActorHost: one warm ProjectActor per project, each module opening its own tx (1 quantum = 1 tx); the host holds no tx and no lock, serialization is the actor's per-project serial queue. Protocol tables made project-scoped (user's steer: a delegation / escalation is a katari-protocol entity, not a CORE-private one): - delegations / escalations: snapshot_id → project_id. The snapshot a delegation runs is module-private state (CORE engine_shards.current_snapshot, FFI ffi_pending_*.snapshot_id, API runs_audit.snapshot_id), never on the shared protocol row. - recovery's live-snapshot set moves from delegations to ffi_pending_delegations.listLiveSnapshotIds (the sidecars to respawn). Pieces: - ApiModule: warm + projectId + self-tx; startRun resolves (projectId, snapshotId?) inside its own tx and stamps the root delegate's snapshot. - actor-host.ts: builds core/api/env/ffi-mux per project, wires sidecar message routing (snapshot→project→actor→lane) + boot recovery. - StorageDelegationStore projectId-bound; StorageFfiStore carries projectId for its unified-delegations mirror; StorageEnvStore is root-backed (auto- commit per op). - pg + memory storage: delegations/escalations project_id; FFI listLiveSnapshotIds. SnapshotNotFound/NoSnapshotForProject move to snapshot-service. - routes / recovery / bin / test helpers rewired to the host. New ffi-e2e test drives a real CORE→FFI→sidecar→CORE→API round-trip through the actor + FfiMux (validates the snapshot strip/stamp end-to-end). Full TS suite green: runtime 81 / api-server 42 / port 15. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
….throw) ApiModule detected the unhandled-throw escalate by a hand-typed `prim.throw`, but every emitter (engine runner, FFI sidecar error, ENV arg error, engine handle-ask) uses `primitive.throw` — so the match never fired and throws were silently recorded as open escalations while the run hung in `running` forever. - Centralise the id as a single exported `THROW_REQUEST_QNAME = "primitive.throw"` (Katari language vocab — the compiler lowers `throw` to module `primitive` / name `throw`). All emitters + ApiModule reference it, so the strings can't drift again. - CoreModule now acks a `terminate` for a delegation with no live shard (e.g. a missing-entry root that errored before spawning), so the throw → terminate → terminateAck → terminal `error` loop closes instead of leaving the run stuck `cancelling`. - CoreModule evicts a shard from its warm cache if applyEvent throws (in-place mutation would otherwise poison the warm copy across feeds). - New e2e test: running a non-existent agent reaches `error`, not stuck. runtime 81 / api-server 43 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First increment of #5 (content-addressed closures). A closure now has two representations: - `{closureId}` — in-shard (index into state.closures), the in-memory form - `{ref}` — content-addressed: a ref to the serialized {blockId, snapshot, captured env} blob. The canonical wire form a closure takes when it crosses a shard boundary. ref-in-ref foundation: a closure ref's blob references its env's nested refs, so the value graph stays content-addressed and acyclic — the eventual GC (Phase G) is reference-counting, not cross-shard mark-sweep (untenable multi-server). The wire carries only the hash (clean bus). - value.ts: closure union + isLocalClosure narrowing. - value-codec.ts: `$ref` envelope gains `as: "closure"`; round-trips the content-ref form (local id still encodes as `$agent:closure:N`). - in-shard ops (gc / prim.get_metadata / delegate) narrow to the local form. - value-secret-codec EncryptedValue mirrors the union. Design recorded in memory. runtime 82 / api-server 43 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…hase E / #5] Pure functions for freezing a machine-local closure into a content- addressed value-store blob and grafting it back into a receiving shard: - serializeClosure: walk the captured scope chain to root, promote nested local closures to their own refs (finite DAG), drop the single self- reference into selfVar. Blob is Value-form (engine-internal, never crosses FFI), so no wire codec. Refuses a captured secret (plaintext-at- rest gap, documented for Phase G). - materializeClosure: graft scopes with fresh ids (remap parents), re-bind the self-reference to the new closure id, register state.closures — after which the existing closure:N dispatch (resolveDelegateTarget) runs it. Recursion handled without SCC hashing: Katari local agents can only self- reference (siblings cannot forward-reference, per Lowering), so one selfVar suffices. Keeps the pure engine free of snapshot/store concerns — CORE does serialize at the outbound boundary, materialize at inbound. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nbound [Phase E / #5] Closures now cross shard boundaries (in-shard invocation was already broken in the warm-actor model — a closure delegate routes to a new shard whose agentDefId had no snapshot, so getOrLoadShard threw). The fix: - agent-def-id: add a closureRef CoreAgentDefId form (carries the content ref; snapshot rides in the blob, not an @ stamp) + agentDefIdClosureRef. - delegate.ts: a ref-closure target encodes closureRef (was: throw). - CoreModule outbound: serializeOutboundClosures freezes any escaping local closure (delegate target + every arg) to a value-store blob, rewriting the target closure:N to closureRef. Top-level secret args pass; secrets inside a captured scope are refused. - CoreModule inbound (getOrLoadShard): a closureRef delegate fetches + materializes the blob into a fresh shard, then rewrites the target back to closure:N so the existing resolveDelegateTarget dispatch runs the body. - runner: guard — a closureRef must never reach the engine. - value-store: ValueSemanticKind gains closure. e2e test: main makes a local agent capturing base=10, invokes it across shards; the captured env survives the blob round-trip (returns 10). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_metadata on a not-yet-invoked closure ref can't introspect without an async blob fetch (the sync metadata path can't do it); invoking materializes it. Clarify the v0.1.0 limitation in the thrown message. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A spawned sidecar runs user ext code and reaches the value data plane (produce / consume / persist) via katari-port, which reads its coordinates from the process env. Thread those through: - SidecarManager: factory + ensureStarted gain an `env` arg. - ApiServerActorHost: takes SidecarProtocolCoords (baseUrl + token); its FFI lane backend stamps KATARI_PROTOCOL_URL / _TOKEN / _PROJECT_ID (per-project) / _SIDECAR_OWNER=ffi on ensureSidecar. - bin.ts: baseUrl defaults to http://127.0.0.1:$PORT (the sidecar is a local subprocess), token = the api key. PROTOCOL_TOKEN is a distinct env name so the subprocess KATARI_API_KEY filter does not strip it; matches the Authorization: Bearer the data-plane auth middleware expects. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…p [Phase C] An ext (makeBlob) writes bytes to the value store via katari.value.put and returns a file ref; CORE holds the ref; a second ext (readBlob) reads it back via katari.value.text. Exercises the Katari Protocol data plane end to end, using the sidecar env wired in 26dd5b9. Surface verified against the current compiler (file is a primitive type: Lexer.KeywordFile / Parser.parsePrimitiveType / SemanticTypeFile). The on-PATH katari binary predates Phase A's file type, and the runtime e2e needs a combined harness (data plane + a REAL subprocess sidecar, not the MockSidecar the samples-e2e suite uses) — so no samples-e2e entry yet; that harness is the remaining Phase C validation step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…al form [Phase E / #5] Pivot the committed local-form design to the agreed eager-ref model (user direction): a closure VALUE is always a content ref; there is no machine-local closureId value form. - A closure literal spawns a MakeClosureThread (a leaf thread, like a prim): its async create freezes the captured scope chain into a value-store blob via the new ctx.putBlob (symmetric to ctx.materialize) and dones the parent with {kind:closure, ref}. The blob is written BEFORE the ref is returned, so a ref never dangles. Modelled as a thread because persisting is async and in this engine a step that waits IS a thread — the statement loop + done stay sync. - State gains a snapshot field (the shard's version, set by CoreModule on create/load); make-closure stamps it into the blob so (blockId, snapshot) uniquely identifies the body independent of the invoker. - Value union collapses to {kind:closure, ref}; value-codec drops the agent:closure:N encoding (closures are ref-only); value-secret-codec + gc drop the local-closure cases (state.closures is now a one-shot materialize-to-dispatch handoff, swept by GC). - CoreModule provides ctx.putBlob (putComplete owner=core kind=closure); sets state.snapshot; drops serializeOutboundClosures (closures are refs already); materialize re-binds the self var to the closure's own ref. - get_metadata on a closure currently throws (introspection needs an async blob fetch) — addressed next. runtime 86 / api-server 43 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…Phase G slice 3)
- FFI in-flight protection: katari-port's `katari.value.put` stamps
`X-Katari-Owner-Delegation` = the running handler's delegation id (read from
the AsyncLocalStorage delegation context), so an ext-produced ref is owned by
its ext delegation while the handler runs and is re-owned by the parent on
delegateAck — closing the "produced but not yet returned, GC sweeps it" race.
- crash backstop: GcService.sweepAllProjects() drops refs whose owning entity
is gone (not in delegations ∪ runs_audit ∪ escalations), reclaiming releases
lost to a crash. Wired into boot recovery (runs before traffic, so it never
races concurrent production; periodic-while-live is deferred to a
serialized-quantum variant).
- end-to-end proof (gc.integration.test): a real `agent main()` doing
`string_to_file("…")` — when the file does NOT escape the run its blob is
freed on completion (blobs == 0); when the run RETURNS it the blob survives
owned by the run (blobs == 1).
runtime 94 / api-server 60 / port 15 / e2e blob-echo green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…history (Phase G) Honor "escalate also moves ownership up": when the API records an escalation, the escalation entity (persistent until answered/cancelled) takes over any ref the escalating delegation owns in the escalate args. Without this, a file carried up in an escalation is owned by the escalator's shard and gets freed when that shard completes — so viewing the escalation in history after the run ends would dangle. transferOwnership only moves refs OWNED BY the escalator (api_files / higher-owned refs untouched). Exports collectRefs from the runtime. Known v0.1.0 residual: an in-CORE `handle` block that stores a ref captured from an escalate's args into a handle-scope state var outliving the escalator is not yet covered (symmetric CORE-side transfer — uncommon pattern, follow-up). api-server 60 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…model Recast the v0.1.0 runtime onto a first-class Entity model (docs/2026-06-01-entity-model.md), replacing the ad-hoc "ref owner = delegation/run/escalation id" GC and the value_refs/api_files/runs_audit tables. No compat layer — a clean switch. Core ideas: - Two records, distinct owners + nested lifetimes: a `delegations` row is the ISSUER's request edge (created at delegate-emit, deleted at the result ack); an `entities` row is the RECEIVER-managed execution node (minted when the module begins processing, self-deleted on terminal). The receiver mints its entity from the bus event + ambient context ALONE — no cross-server read. - Refs are owned by exactly one Entity (or transiently NULL mid-ascent); a delegation never owns a ref. Ascent is value-driven: a terminating child detaches its escaping refs (owner→NULL) and the parent claims them by id from the result value it receives — no parent lookup, no entity id on the bus. - Single `refs` table (unifies value_refs + api_files); blob refcount is kept by an AFTER DELETE trigger so entity CASCADE frees blobs automatically. - Escalations are raiser-owned; the API records operator-facing ones (pending + answered) in run_escalations_audit, resolving the run from the bus delegation id on its OWN tables. - RunState is running|cancelling|done|error (was succeeded/cancelled); the Haskell CLI follows. Modules touch only their own rows on the hot path: CoreModule + ApiModule never read another module's entity/delegation rows. FFI is entity-ized too (receiver + issuer): each ext call mints an ext entity that owns the refs the sidecar produces and ascends them on terminal; ext-spawned CORE children are issued under it. (ENV stays entity-less — a synchronous, ref-less leaf.) Tests: runtime 94/94, api-server 60/60 (gc.integration / ffi-e2e / end-to-end), e2e samples 31/32 (08-metadata is a pre-existing content-addressed-closure id format issue, unrelated). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`get_metadata` on a closure returns `id = closure:<blob-hash>` — content- addressed since the self-describing-closures change (78176bc, Phase E), where the closure's identity is its content hash (stable across shards), not the old ephemeral machine-local `closure:N`. The sample assertion still pinned the pre-Phase-E `closure:0`, so it had been failing independently. Assert the closure id's SHAPE (`closure:<64-hex>`) rather than the exact hash (an impl detail). e2e samples now 32/32. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lined RefRep)
A closure's agent-def-id was `closureref:<full RefRep JSON>`. Slim it to just the
ref id (`closureref:<id>`): `module` is invariably `core`, and `hash`/`size`
live in the ref store keyed by that id (refs.hash) — CORE fetches the blob by
`(core, id)`, so the inlined hash/size were redundant on the wire + in the
agent_def_id column.
Converge all three on the same string (the value-codec already aimed for this):
- the closure VALUE's wire form (`$agent: closureref:<id>`),
- the delegate target,
- `get_metadata`'s `id` field (was `closure:<content-hash>` — inconsistent;
now `closureref:<ref id>`, the dispatch handle, mirroring a top-level agent's
`id` = its qname).
A wire-decoded / materialized closure reconstructs `{module:core, id, hash:"",
size:0}` — hash/size are vestigial for a closure (dispatch / get_metadata / GC
all key off (module, id)). e2e 08-metadata asserts the `closureref:<uuid>` shape.
Tests: runtime 94/94, api-server 60/60, e2e samples 32/32.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…l-calling)
`call_agent(name, args)` now resolves a `closureref:<ref id>` name — the same
dispatch handle a closure value carries + `get_metadata` returns in its `id`.
It fetches the closure blob (its input schema lives there, not the IR),
validates args against it, and emits `delegate(closureref:<id>)` so CORE
materializes + runs the closure. This closes the AI-tool-calling round-trip for
closures: a model picks a callable by NAME → the host maps name → metadata → id
→ `call_agent(id)` (the uuid stays host-internal, never shown to the model),
exactly as for a top-level agent's qname.
`resolveTarget` is now async (the blob fetch). The dead in-shard `closure:N`
branch is removed — it was an engine-internal id, never a user-facing name (a
closure is a `closureref:<id>`).
Aligns the stale `closure:N` docs/comments (agent-def-id header, Schema.hs's
callable-ref notes, the call_agent stdlib annotation) with the real wire form
`closureref:<id>`. The compiled `$agent` schema is an open `{type: string}` —
there is no closure enumeration. New e2e sample 23-call-closure proves the
round-trip.
Tests: runtime 94/94, api-server 60/60, e2e samples 33/33.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ped handle `PgValueStore` opened its own `this.sql.begin(...)` inside putComplete / createFile / deleteFile / persistRef / reownRefs / sweepDetachedRefs. But postgres.js exposes `.begin` only on the ROOT sql, NOT on a transaction-scoped one (the callback arg of an outer `begin`), so any of these called within a `withTransaction` threw `this.sql.begin is not a function` and 500'd the route. This hit the file-upload route (now wrapped in a tx for the project-root entity) and, latently, EVERY CoreModule ref-produce on Postgres (makePutRef runs inside feed's tx) — the in-memory test backend never exercised the `.begin` path, so it slipped through. Add `inTx(fn)`: open a fresh tx when we hold the root sql (atomicity for direct route calls), otherwise run `fn` directly on `this.sql` (already inside the caller's tx, which provides atomicity). Route the six methods through it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The HTTP layer moved to the Entity model (RunState running|cancelling|
done|error, run-keyed escalations, an entity run-tree). Bring the local
wire mirrors + components in line with the served shapes:
- RunState is 4-state; a user cancel renders as "cancelled" via the
run's cancelReason (passed to RunStatusBadge at every call site).
- EscalationWire is keyed by runId / escalationId (the old delegation /
snapshot ids are gone); the escalation pages now resolve the snapshot
through the run.
- DelegationTree{Node,Graph} -> RunTree{Node,Graph}: the graph renders the
execution-entity forest (node id = entityId, label = module); the stale
back-compat aliases are dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… agent ids Three related cleanups to the value / agent model. 1. Drop `refs.origin`. Lifetime/durability now has a single source of truth — ownership: a durable project file is a `file` ref owned by the project-root entity (id = project id). `listFiles` queries by owner; the `origin` column + index, `RefOrigin`, and `PutInput.origin` are gone. 2. Split agent def ids into two namespaces. The EXTERNAL form (`qualified.name@snapshot`) is what rides the bus and lives inside an agent VALUE; the INTERNAL bare name is the per-snapshot IR-entries / sidecar-registry key. The snapshot is stamped in exactly two places — DelegateThread (CORE/FFI targets) and the API entry — so the module-level fix-up stamp (CoreModule) and the FFI ext-child re-stamp are removed (stampAgentDefIdSnapshot deleted). agentLiteral carries its snapshot (set at the source literal from the shard's snapshot, preserved through the codecs); get_metadata.id is the external form. 3. call_agent accepts ONLY the external id (= get_metadata.id); a bare name is rejected. A statically-known agent is dispatched by its first-class value — call_agent is only for an id that arrived as a string, which always carries its snapshot. Sample 22 updated to feed call_agent the get_metadata.id. runtime 94/94, api-server 60/60, e2e 33/33. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A CORE-produced file (string_to_file) had no content type — the engine's RefPutter couldn't carry one, so the data plane served it as application/octet-stream. Thread an optional contentType through RefPutter / StepCtx.putBlob / CoreModule.makePutRef, and have string_to_file stamp "text/plain; charset=utf-8" (its bytes are a UTF-8 string). Upload and FFI already declare their own content type; closures still omit it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add displayName to RefState (the data-plane `.../ref/:id/state` shape) and have both ValueStore backends return the refs row's display_name. The wire value of a file carries only the ref handle, so a consumer (the admin value viewer) needs this to label / name-download a file ref — for any module (core / ffi / api), not just persisted api uploads. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make Katari's reference shapes first-class across the schema/value/invoke surfaces instead of rendering them as raw objects: - ValueViewer: a file ref shows its display name (fetched from the value state endpoint) + a download (authenticated fetch -> blob, works for any module; filename = display name); an agent value (`qname@snapshot`) links to its agent page at that snapshot; a closure / other ref renders as a ref chip. (Drops the stale $callable handling — the wire uses $agent.) - AgentField: an agent-typed (`$agent`) argument is now a picker over the form's snapshot, building the external id `qualified.name@snapshot` (the front-end half of the two snapshot-stamping sites). Closures aren't selectable. Threaded via a SchemaForm snapshot context. - SchemaViewer: `$agent` / `$ref` schemas show the type name (agent / file / ref) rather than the wire machinery. - FilePicker: an upload prompts for an editable display name (defaulting to the file's name). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…apshot id
Two small additions that the friendlier sidecar value API needs:
- A produced ref can carry a display name: `PutInput.displayName` flows to
the refs row, and the produce route reads an `X-Katari-Display-Name`
header. Lets `katari.makeFile(bytes, { name })` name an ephemeral file.
- The sidecar's env gains `KATARI_SNAPSHOT_ID` (the snapshot it runs).
`katari.makeAgent` needs it to build the external agent id
`qualified.name@snapshot`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…args); slim ValueClient
Give the sidecar SDK a coherent, typed surface so writing an ext module
doesn't mean hand-assembling $ref / $agent envelopes:
- Types: KatariRef<As>, KatariFile, KatariString (string | ref), KatariAgent,
+ guards (isKatariFile / isKatariString / isKatariAgent).
- Construct: katari.makeString (inline, or a ref over a size threshold),
katari.makeFile(bytes, { name?, contentType? }), katari.makeAgent(qname)
(= {$agent: qname@snapshot}, using katari.snapshotId).
- Read: katari.readString / katari.readBytes (inline or ref, transparently).
- katari.persist(value, { name? }).
- katari.delegate now takes a KatariAgent (or a bare id string).
- katari.agent<A>(name, handler) types the handler's args.
The old ValueClient is cut down to an internal data plane (createDataPlane:
fetchBytes / produce / persist) + the guards; the public katari.value
namespace, createValueClient, the fake streaming open(), and fetchRange are
removed. 23-blob-echo migrated to the new API (and demos typed args).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`katari apply` decoded the bundler's UTF-8 stdout with `LC8.pack` (Char8, which truncates each char to its low byte), so any non-ASCII in the bundle — e.g. `@katari-lang/port`'s `// ─── … ───` box-drawing comments (U+2500), or a non-ASCII string literal — became NUL, and Postgres rejected the snapshot with an opaque `unsupported Unicode escape sequence` 500. Decode as UTF-8 instead. (An ext returning Japanese / emoji would have hit this too.) Also add an `assertNoNul` guard to PgSnapshotRepo.insert naming the exact jsonb field + offset + context, so a stray NUL surfaces precisely instead of as a driver error deep in INSERT — that guard is what pinpointed the bug above. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Result card reused the Args card's `JSON.stringify(run.args)`, so Copy JSON
yielded `{}` for a no-arg run. Copy `run.result`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A real KATARI app built to dogfood the language + SDK. examples/* is added to the pnpm workspace and the example installs @katari-lang/port directly (so ext types resolve; external authors install the SDK themselves). The AI slice runs end to end: an `ai_client` (data carrying a `secret` key) provided once via a `get_ai_client` capability, a sidecar-held conversation session, and an `infer` agent over an `ai_infer` ext that calls Gemini. The single self-host example is dropped — `katari init` already scaffolds a self-host docker-compose, so README / the dev compose now point at that. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ler ctx The delegation-bound operations (`delegate`, the `make*` producers) used an AsyncLocalStorage to recover "which delegation am I in" from the global `katari` singleton. That breaks for event-driven exts: a `watch_*` handler registers a listener, then the listener fires LATER off its own async chain (a socket / emitter / timer), outside the ALS `.run()` scope — so the ambient delegation id was null and `katari.delegate(...)` threw "must be called from inside a katari.agent handler", silently (the throw was a swallowed floating rejection). That's why a discord `watch_messages` would start but never deliver. Move every per-delegation op onto the handler `ctx` (which already carried `delegationId`): `ctx.delegate` / `ctx.makeString` / `ctx.makeFile` / `ctx.makeAgent` / `ctx.readString` / `ctx.readBytes` / `ctx.persist` / `ctx.snapshotId`. `ctx` is a plain object, so a handler closes over it and uses these from any later callback — no ambient state to lose. Only `katari.agent` (registration) stays global. `produce` takes `ownerDelegationId` explicitly; ALS is gone. All ext call sites (discord_bot + the e2e samples) updated. Also install a sidecar `unhandledRejection` / `uncaughtException` → stderr net: an error escaping a tracked handler (e.g. a fire-and-forget `void ctx.delegate` in a watch listener) has no delegation to attribute a `throw` to, so it can't become a tree escalation — at least surface it in the logs instead of vanishing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The sidecar bundle is ESM, but bundled CommonJS deps (e.g. discord.js) call
`require("node:events")` at load time. esbuild leaves those as a `__require`
shim that throws "Dynamic require of … is not supported" in ESM, so the sidecar
died on start. Inject `const require = createRequire(import.meta.url)` via the
banner so the shim resolves built-ins / CJS deps at runtime. Affects any ext
with a CommonJS dependency, not just discord.js.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The bus only logged a warn on a dropped event; the delegate / escalate / *_ack
flow between CORE and FFI was invisible. Log each drained event at debug with
{from, to, kind, delegationId?, escalationId?, agentDefId?} so KATARI_LOG_LEVEL=debug
shows the cross-module traffic — which is how you see where an event-driven run
stalls.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bin.ts read the unprefixed LOG_LEVEL, but every .env template documents KATARI_LOG_LEVEL — so the documented knob did nothing (and `pnpm dev`, which reads .env directly, never saw a level). Read only KATARI_LOG_LEVEL and pass it straight through in the compose files / Dockerfile / `katari init` template. (PORT and DATABASE_URL keep their unprefixed names on purpose — PaaS-injected PORT, the de-facto Postgres DATABASE_URL — a log level has no such convention.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the Discord layer on top of the AI slice: `create_discord_client` (bot token as a secret), `watch_messages` (delegates a handler agent per message in a channel; never-returns, disconnects on cancel via the abort signal), and `send_message`. `main` takes the channel id as an arg and serves it. The capabilities (get_ai_client / get_discord_client) flow into the ext-delegated handler automatically. Real Gemini reply posted back to the channel — verified apply → run → live message → reply. Pulls in discord.js; adds a tsconfig so the ext type-checks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Refactor the discord bot to the request/handle model. The ext primitives drop
to internal callable-taking forms (discord_watch / discord_send, client passed
explicitly); the ktr wraps them as capability agents: watch_messages(channel_id)
with on_message, get_discord_client and send_message(channel_id, text) with
get_discord_client. watch_messages raises an on_message(text, channel_id)
request per message; main installs handle { request on_message(...) { infer;
send_message; next } } and provides the capabilities.
Two payoffs: the watch signature no longer enumerates the reaction's effects
(they live in the user's handler), and the session is opened once at the top, so
the bot keeps conversation history across messages (verified end to end).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A git-tracked home for the language / SDK / tooling gaps surfaced while dogfooding, with priority tags. Replaces the scattered notes; the sample-era gaps it would have listed are all resolved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a tool-calling slice to the discord_bot example. run_python is a Katari agent over an e2b_exec ext (runs Python in an e2b sandbox); ask() reads its schema via get_metadata and hands it to infer_with_tools, an ext that runs the tool-call feedback loop: declare the tool to Gemini, on a functionCall dispatch it with ctx.delegate (so the tool is a real agent — it uses the get_e2b_key capability), feed the functionResponse back, repeat until the model answers. A standalone solve() entry runs it from the CLI (no Discord). Verified end to end (model wrote + ran Python, result confirmed via the bus delegate log). The loop is written list-shaped with a single tool, so v1 = pass an array of tools (blocked on language-side list ops). The Gemini call strips JSON-Schema keywords its parameters proto rejects (additionalProperties, …); runtime arg validation still uses the full schema. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_metadata's input schema is draft-07 (right for runtime validation) but not directly consumable by LLM providers — Gemini rejects additionalProperties etc. Note that per-provider schema adaptation needs a home as tool calling matures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
yukikurage
added a commit
that referenced
this pull request
Jul 1, 2026
Address the xhigh review of the http/io/env phase. Runtime + compiler. Shared external-call reactor (review #7 / #9): - Extract the ffi/http callee-call lifecycle into a new `ExternalCallReactor` base (react / complete / afterCommit / persist / load / reset, the running → cancelling → awaitingAnswer state machine, and the caller the reply routes to). FfiReactor and HttpReactor become thin: a per-call payload plus how to dispatch / abort and read / write their ext row. The core/api reactors keep their engines, so this is a call-reactor base, not folded into the root. - The caller reactor is now owned + persisted uniformly by the base (http gains a `caller_reactor` column, migration 0012); HttpReactor no longer hardcodes `core` on recovery. - HttpReactor lifts a result through the shared `jsonToValue` codec (drops the duplicated, coercing `httpResponseValue`). Cancelling recovery, uniform + correct (review #8 / #2 / #11): - Recovery is uniform in the base: a running call re-dispatches, a cancelling call re-aborts, an awaitingAnswer call waits. http no longer reaches a terminateAck via a redispatch error — it aborts, like ffi. - A transport `abort` with no live request now synthesises a `cancelled` (FetchHttpTransport, SnapshotFfiTransport, InProcessFfiTransport), so a cancelling call recovered after a crash is confirmed instead of hanging. Smaller fixes: - http.fetch sends an explicit empty body for body-carrying methods (#4). - env get_all / readPublic build records on a null-prototype map so an env key named `__proto__` is a real field, not a silent drop (#6). - env.set refreshes `updated_at` on overwrite (onConflictDoUpdate does not fire the column's `$onUpdate`). - Compiler rejects an unknown `from "reactor"` name (K3018) instead of a silent runtime fallback to ffi (#1). - `asEffectMetavar` excludes io, so a bare effect metavar is matched exactly (#10). - `signatureValueScheme` sets io through a `withIo` helper, not a hand-written record update that re-derives the lattice join (#12). - `reactivate` loads the reactors' disjoint state concurrently (#13). Restore FetchHttpTransport unit coverage (request building, GET no body, empty body sent, non-2xx = result, at-most-once redispatch, abort → cancelled) and add http cancelling + cancelling-recovery e2e tests. compiler 490, runtime 95 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.