Skip to content

Move inline specs to the blob table at boot on the libSQL hosts#968

Merged
RhysSullivan merged 1 commit into
mainfrom
spec-blob-boot-migration
Jun 12, 2026
Merged

Move inline specs to the blob table at boot on the libSQL hosts#968
RhysSullivan merged 1 commit into
mainfrom
spec-blob-boot-migration

Conversation

@RhysSullivan

@RhysSullivan RhysSullivan commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

What

Cloud's databases were rewritten by the out-of-band migrate-specs-to-blobs script, but the libSQL hosts (local, selfhost) have no operator — their integration.config rows still inline multi-MB OpenAPI spec text and GraphQL introspection snapshots from before the blob seam existed.

This adds the ledger counterpart: a shared migration body in the SDK (runSqliteConfigBlobMigration) that moves a named inline config field into the blob table under the runtime's exact naming (o:<tenant>/<plugin> namespace, <prefix>/<sha256> key) and rewrites the config to the content-hash pointer, all inside one transaction. The openapi and graphql plugins bind their field names and export ledger entries (2026-06-12-openapi-spec-to-blob, 2026-06-12-graphql-introspection-to-blob), registered in both apps' data-migration registries.

Properties

  • Idempotent: pointer rows plan zero work; blob writes are content-addressed upserts; identical specs across integrations share one blob per tenant.
  • Tolerant of a fresh database (no integration table = nothing to migrate).
  • Stamped once by the data-migration ledger; transactional with rollback on failure.
  • Writes the SQL blob table specifically — correct for the libSQL hosts whose runtime blob backend is the FumaDB store. The D1 host (R2-backed blobs) deliberately does not register it.

Why now

With every host converging on the pointer shape, the legacy inline-read paths in the plugins become removable — the next PR in this stack.

Testing

New ledger-migration tests: pointer rewrite + runtime blob naming round-trip, idempotence, foreign-row safety, cross-integration blob sharing, fresh-DB tolerance, ledger stamping. Full gates green.

Stack

  1. Remove the D1-to-R2 oversized-value offload #967
  2. Move inline specs to the blob table at boot on the libSQL hosts #968 👈 current
  3. Drop the inline-spec read paths; pointer configs are the only shape #969
  4. Project the heavy schema columns off the tool hot paths #970

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 12, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
executor-marketing 7b1a1e3 Commit Preview URL

Branch Preview URL
Jun 12 2026, 05:43 AM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 12, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
executor-cloud 7b1a1e3 Jun 12 2026, 05:43 AM

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Cloudflare preview

Torn down — the PR is closed.

@pkg-pr-new

pkg-pr-new Bot commented Jun 12, 2026

Copy link
Copy Markdown

Open in StackBlitz

@executor-js/cli

npm i https://pkg.pr.new/@executor-js/cli@968

@executor-js/config

npm i https://pkg.pr.new/@executor-js/config@968

@executor-js/execution

npm i https://pkg.pr.new/@executor-js/execution@968

@executor-js/sdk

npm i https://pkg.pr.new/@executor-js/sdk@968

@executor-js/codemode-core

npm i https://pkg.pr.new/@executor-js/codemode-core@968

@executor-js/runtime-quickjs

npm i https://pkg.pr.new/@executor-js/runtime-quickjs@968

@executor-js/plugin-file-secrets

npm i https://pkg.pr.new/@executor-js/plugin-file-secrets@968

@executor-js/plugin-graphql

npm i https://pkg.pr.new/@executor-js/plugin-graphql@968

@executor-js/plugin-keychain

npm i https://pkg.pr.new/@executor-js/plugin-keychain@968

@executor-js/plugin-mcp

npm i https://pkg.pr.new/@executor-js/plugin-mcp@968

@executor-js/plugin-onepassword

npm i https://pkg.pr.new/@executor-js/plugin-onepassword@968

@executor-js/plugin-openapi

npm i https://pkg.pr.new/@executor-js/plugin-openapi@968

executor

npm i https://pkg.pr.new/executor@968

commit: 2536422

@RhysSullivan RhysSullivan changed the base branch from remove-d1-blob-offload to main June 12, 2026 05:41
Cloud's databases were rewritten by the out-of-band
migrate-specs-to-blobs script, but the libSQL hosts (local, selfhost)
have no operator — their integration.config rows still inline multi-MB
OpenAPI spec text and GraphQL introspection snapshots from before the
blob seam existed.

This adds the ledger counterpart: a shared migration body in the SDK
(runSqliteConfigBlobMigration) that moves a named inline config field
into the blob table under the runtime's exact naming
(o:<tenant>/<plugin> namespace, <prefix>/<sha256> key) and rewrites the
config to the content-hash pointer, all inside one transaction. The
openapi and graphql plugins bind their field names and export ledger
entries, registered in both apps' data-migration registries.

Idempotent (pointer rows plan zero work, blob writes are
content-addressed upserts), tolerant of a fresh database with no
integration table, and stamped once by the ledger. With every host now
converging on the pointer shape, the legacy inline-read paths in the
plugins become removable.
@RhysSullivan RhysSullivan force-pushed the spec-blob-boot-migration branch from 2536422 to 7b1a1e3 Compare June 12, 2026 05:41
@RhysSullivan RhysSullivan merged commit 8a74ca0 into main Jun 12, 2026
13 checks passed
@RhysSullivan RhysSullivan deleted the spec-blob-boot-migration branch June 12, 2026 06:02
RhysSullivan added a commit that referenced this pull request Jun 12, 2026
The eval harness's core capability — drive the real OpenCode binary with
the machine's subscription, hermetically, and read structured output — is
useful on its own, so pull it out of the eval-specific harness.

- e2e/src/clients/inference.ts: runInference({ model, prompt, mcp? }) →
  { answerText, events, toolNames, ... }. Hermetic OpenCode home with the
  subscription credential copied in; MCP optional with a pluggable consent
  strategy. No effect on the developer's own OpenCode state.
- e2e/scripts/infer.ts (+ `bun run infer`): the CLI an agent runs while
  developing — `bun e2e/scripts/infer.ts -m opencode/glm-5.1 "..."`.
- evals/harness.ts is now a thin consumer: runTrial = runInference + the
  selfhost cookie-consent strategy; grading vocabulary unchanged.

Also: drop the connect-handoff 'baseUrl resolved' check — running against
current main surfaced that integration-level baseUrl is override-only now
(per-operation baseUrl from the #968-970 storage refactors), so the check
asserted an obsolete field. Documented in EVALS.md. Switch the eval test
import to @effect/vitest per the no-vitest-import rule.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant