feat(execution-history): move run detail to R2, slim the D1 index

[aryasaatvik]

What

Splits execution-history storage so each store fits its access pattern:

• D1 runs row → a slim index. Drops the bulky, drawer-only fields (full code, resultJson, errorText, logsJson, triggerMetaJson) and removes the toolCalls / interactions collections. Keeps the list/aggregate fields plus two bounded denormalized fields — codePreview and logErrorCount/logWarnCount — so the list renders entirely from D1 with no per-row blob fetch.
• R2 detail object → the bulky payload. One append-only JSON object per run (via the existing plugin blob store, keyed run-detail/<id>): a code-only stub on ExecutionStarted (so a live/paused run shows its code and survives a worker restart), the full detail on ExecutionFinished. Serialized via Effect Schema (fromJsonString), never raw JSON.
• Drawer reads it in one request. get flat-merges the R2 detail onto the slim run; the separate tool-calls endpoint + atom are removed.

Why

The bulky fields are write-once, never mutated, and read only by id in the drawer — the shape object storage is built for. Moving them to R2 keeps the D1 row tiny (~hundreds of bytes), removing the D1 size ceiling as a practical concern, while the slim denormalized fields keep the list a pure D1 read.

Breaking change + cutover

The runs list/detail response shapes change and past runs are cleared (no backfill):

• self-host: a one-off boot data migration deletes the executionHistory rows.
• The slim fields are optional-with-default, so any pre-migration row still decodes/encodes (degrades to an empty preview) rather than 400-ing the list.

Verification

• typecheck: 0 errors · lint: 0/0 · format: clean · tests: 19/19 (the store test round-trips a run's detail through the blob store).

Follow-up

A separate PR adds the first execution-history e2e scenario (run → open drawer → assert code/logs/tool-calls). The optional WAE aggregate-reader remains deferred behind a documented trigger.

[greptile-apps]

Greptile Summary

This PR splits execution-history storage to match each subsystem's access pattern: a slim D1 runs index carrying only list-renderable fields (codePreview, logErrorCount/logWarnCount, timing, counts) and an append-only R2 detail object per run holding the bulky drawer-only payload (full code, result/error/logs, trigger metadata, tool-call and interaction rows). The separate toolCalls/interactions D1 collections and the listToolCalls endpoint/atom are removed; the drawer now reads a single flat-merged response from GET /runs/:id.

• D1 slim row written eagerly on ExecutionStarted and InteractionStarted; logErrorCount/logWarnCount are denormalized from the engine's final logs array at ExecutionFinished, so the list never fetches a blob.
• R2 detail blob written as a code-only stub on ExecutionStarted (drawer durability during live/paused runs and across worker restarts) and overwritten with the full detail on ExecutionFinished; serialized via Effect Schema fromJsonString, never raw JSON.
• Breaking cutover: a self-host boot data migration deletes all executionHistory plugin-storage rows; the changeset is documented and the list degrades gracefully to empty previews for any row that survived.

Confidence Score: 5/5

Safe to merge; the D1/R2 split is well-structured and the typed Effect error channels are used correctly throughout.

The refactor is internally consistent: the slim RunRow schema, the R2 codec, the write/read helpers, and the HTTP response shape all align. The hard-cut data migration is appropriate given the breaking schema change, and the changeset documents the trade-offs clearly. The one write-ordering concern (D1 final state committed ahead of the R2 detail write) is an edge case that only materialises under a transient R2 failure at the exact moment a run finishes; in practice the store degrades gracefully rather than corrupting data.

packages/plugins/execution-history/src/sdk/store.ts — specifically the write ordering in onExecutionFinished and the untested buffer-lost restart path (noted in prior review comments).

Important Files Changed

Filename	Overview
packages/plugins/execution-history/src/sdk/store.ts	Core store rewrite: D1 writes now emit slim rows; R2 detail blobs are written concurrently on ExecutionStarted and overwritten on ExecutionFinished. The buffer-lost path reads the start-stub from R2 to recover code/triggerMeta. Effect error handling is typed throughout. The D1→R2 write ordering in onExecutionFinished means a transient R2 failure can leave a run showing 'completed' in the list but with only stub detail in the drawer.
packages/plugins/execution-history/src/sdk/detail-types.ts	New file: defines RunDetail schema and RunDetailFromJsonString codec for the R2 blob boundary. Clean, minimal, uses Effect Schema correctly.
packages/plugins/execution-history/src/sdk/collections.ts	Removes toolCalls and interactions D1 collections; adds codePreview, logErrorCount, logWarnCount to RunRow as required fields. ToolCallRow/InteractionRow are retained as shared types backing the R2 detail object.
packages/plugins/execution-history/src/api/group.ts	RunDetailResponse updated to include flat-merged R2 fields (code, resultJson, errorText, logsJson, triggerMetaJson); listToolCalls endpoint removed. Matches ExecutionHistoryDetail interface in store.ts.
packages/plugins/execution-history/src/react/detail-drawer.tsx	ToolCallsTab now receives toolCalls as a prop (from the detail atom) instead of its own async atom. DetailContent props extended with code/resultJson/errorText/logsJson/toolCalls. runToolCallsAtom removed throughout.
apps/host-selfhost/src/db/data-migrations.ts	Adds a hard-cut migration that DELETEs all executionHistory plugin-storage rows on boot; no backfill. Correctly scoped to plugin_id = 'executionHistory'.
packages/plugins/execution-history/src/sdk/store.test.ts	Happy-path tests updated: verifies detail fields (code, toolCalls) come from R2 via the get() call. Buffer-lost restart path and interaction-pause tool-call visibility remain untested (noted in prior review comments).

Sequence Diagram

sequenceDiagram
    participant Engine
    participant Store
    participant D1
    participant R2

    Engine->>Store: ExecutionStarted
    par D1 + R2 concurrent
        Store->>D1: "putRun (status=running, codePreview, counts=0)"
        Store->>R2: putDetail (code-only stub)
    end

    Engine->>Store: ToolCallStarted / ToolCallFinished
    Note over Store: buffered in-memory only

    Engine->>Store: InteractionStarted (optional)
    Store->>D1: "putRun (status=waiting_for_interaction)"
    Note over R2: stub unchanged — toolCalls still []

    Engine->>Store: ExecutionFinished
    Store->>D1: putRun (final state: status, timing, logCounts)
    alt buffer present (normal path)
        Store->>R2: putDetail (full: code + result + logs + toolCalls)
    else buffer lost (restart path)
        Store->>R2: readDetail (recover code/triggerMeta from stub)
        Store->>R2: "putDetail (result + logs, toolCalls=[])"
    end

    participant Client
    Client->>Store: get(executionId)
    Store->>D1: runsC.get
    Store->>R2: blobs.get (run-detail/id)
    Store-->>Client: flat-merged ExecutionHistoryDetail

Loading

_{Reviews (3): Last reviewed commit: "refactor(execution-history): require the..." | Re-trigger Greptile}