Skip to content

🗄️ Own WorkflowRun connections and complete schema v1 - #391

Merged
taras merged 2 commits into
mainfrom
agent/issue-365-2-connection-schema
Aug 8, 2026
Merged

🗄️ Own WorkflowRun connections and complete schema v1#391
taras merged 2 commits into
mainfrom
agent/issue-365-2-connection-schema

Conversation

@taras

@taras taras commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Depends on #389.

Stack layer 2 for #365, replacing the authoritative-connection and complete-schema portion of draft #387. This PR does not close #365.

Why

WorkflowRun handles need one provider-owned SQLite/DOFS connection boundary per canonical run path before retained Workspace mutations can be published atomically. The unreleased version-1 database also needs its final complete structure now, without introducing a migration from the unsupported intermediate pre-release shape.

What changes

Before:

Each handle owned a SQLite connection, and schema version 1 contained only workflow metadata and the filtered journal.

After:

Each canonical run path resolves to one provider-owned entry containing the physical SQLite connection, cooperative queue, pinned DOFS wrapper, WorkspaceFilesystem, and synchronous savepoint allocator. Handles are scope-owned leases. Fresh databases atomically receive the complete frozen version-1 schema, canonical content-addressed empty Workspace root, current-root pointer, and journal root association. Existing databases are recognition-only and strict corruption failures remain unchanged on disk.

Foreign-key failures are interpreted as invalid stop reasons only at the two statements that write stop-reason event references. Other Workspace, root, manifest, blob, and journal-root foreign-key failures remain storage failures and roll back their transactions. An XMD-identified database with user_version = 0 is partial initialization and is rejected as corruption without changing the file.

How it works

provider path registry → authoritative connection + cooperative turn → caller-owned transaction → DOFS initialization savepoint → complete v1 schema + canonical empty root

Review guide

Start with: packages/workflow/src/deno/connections.ts

Then review:

  1. packages/workflow/src/deno/provider.ts for provider lifetime and atomic initialization
  2. packages/workflow/src/deno/schema.ts for the frozen structural manifest and strict recognition
  3. packages/workflow/src/deno/workspace/empty.ts for root-format-1 identity and semantic validation
  4. packages/workflow/tests/workflow-run-storage.test.ts and workflow-run-journal.test.ts

Look carefully at:

  • lease teardown versus provider teardown
  • savepoint ownership inside BEGIN IMMEDIATE
  • byte-for-byte refusal of intermediate or partial version-1 files
  • journal association with the current retained root
  • statement-local stop-reason foreign-key classification

What must stay true

  • WorkflowRunDatabase.transact() remains the only top-level transaction owner; DOFS initialization nests through uniquely named SQLite savepoints.
  • Same-run operations wait cooperatively on one per-entry queue; different run paths remain independent.
  • Existing databases are never repaired or migrated; exact structure and empty-frontier semantics are validated before rows are trusted.
  • Runtime-specific SQLite, DOFS, SQL, and savepoint types remain inside the Deno adapter.

How to verify it

  • deno task setup — passed.
  • deno task lint — passed with 0 errors.
  • deno task check — passed.
  • deno task test — passed: 401 tests, 2,744 steps, 0 failures.
  • deno task check:jsr — passed (Success Dry run complete).
  • Focused workflow storage and journal suites — passed: 12 suites, 79 steps.
  • git diff --check origin/agent/issue-365-1-vendor-dofs...HEAD — passed.
  • deno task verify — passed all nine applicable commands under the repository CI Node 22 runtime; the tracked tree remained unchanged.

Scope

Included

  • provider-owned authoritative connection registry and scope-owned leases
  • cooperative per-run serialization and synchronous savepoint integration
  • pinned DOFS initialization on the authoritative connection
  • complete frozen WorkflowRun schema version 1
  • canonical empty Workspace root, retained reference tables, current-root state, and strict empty-frontier recognition
  • minimal journal root association required by the foreign key
  • architecture and workflow-storage contracts for this layer

Intentionally unchanged

  • no Workspace mutation or durable coordinator API
  • no operation-spanning mutation savepoint or guarded journal routing
  • no snapshot traversal, restoration, history/fork, public <File>, lifecycle, Worker Shell, Git, FUSE, subprocess, workerd, or garbage collection behavior
  • no architecture/specification claims for the later atomic publication layer

New abstractions

  • WorkflowRunConnections owns one authoritative adapter entry per canonical run path.
  • SavepointManager adapts synchronous DOFS transactions to the caller-owned SQLite transaction.
  • Each new abstraction has a concrete provider or transaction consumer.
  • No speculative functionality is included.

Risks and limitations

  • This layer recognizes only the canonical empty Workspace frontier it creates. General roots and Workspace effect publication remain in later stack layers.
  • Unsupported intermediate version-1 databases must be deleted and recreated; they are rejected without mutation.

Scope confirmation

  • Every changed file supports the purpose described above.
  • Unrelated cleanup and formatting changes are excluded.
  • Generated or mechanical changes are clearly identified.
  • The description matches the final diff and test results.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

PR #391: 🗄️ Own WorkflowRun connections and complete schema v1

15 files, +1345 / -287

Scope

🔴 PR has 1632 lines changed. Split into focused PRs.

🟡 1632 lines changed. PRs under 400 receive more thorough review.

Structural

✅ No structural bloat detected.

Slop

✅ Slop indicators look low.

Static Analysis

✅ Oxlint found no issues.

Correctness

No extraneous code patterns detected.

@taras
taras marked this pull request as ready for review August 8, 2026 08:40
Base automatically changed from agent/issue-365-1-vendor-dofs to main August 8, 2026 20:51
@taras
taras force-pushed the agent/issue-365-2-connection-schema branch from fe00594 to 02a7363 Compare August 8, 2026 20:51
@taras

taras commented Aug 8, 2026

Copy link
Copy Markdown
Owner Author

System/software architecture review

Verdict: APPROVE as stack layer 2. I found no architectural blocker in #391.

This is a coherent continuation of #389 and the correct ownership boundary for the rest of #365:

  • A canonical run path now resolves to one provider-owned entry containing the physical SQLite connection, Cloudflare Database wrapper, WorkspaceFilesystem, cooperative queue, and savepoint allocator.
  • A WorkflowRunDatabase remains a scope-owned capability. Closing one lease invalidates only that lease; provider teardown closes the physical connection after child scopes finish.
  • SQLite, Cloudflare, DOFS, filesystem, and savepoint types stay in the Deno adapter. The provider-neutral storage and transaction APIs do not acquire a runtime-specific dependency.
  • Fresh initialization is one caller-owned BEGIN IMMEDIATE: XMD identity, the pinned DOFS-v5 schema, complete XMD schema v1, the canonical empty root/frontier, the run row, and user_version = 1 appear together or not at all.
  • Recognition is intentionally strict. The unsupported metadata-only pre-release v1 shape, partial initialization, altered objects, malformed root state, and foreign-key damage are refused without repair or migration.
  • Journal appends resolve workspace_state.current_root_id on the authoritative connection inside the insertion transaction, so even non-Workspace events establish the event-to-root invariant now.
  • Foreign-key interpretation is correctly statement-local: only the two stop-reason writes translate a missing journal event into a request error. Root, blob, manifest, and journal-root constraint failures remain storage failures.

The three version domains are also kept distinct: XMD schema v1, DOFS schema v5, and Workspace-root format v1. Completing the unreleased schema in place is consistent with the pre-release decision; adding a migration or schema v2 here would create a compatibility contract we do not need.

The canonical-empty validation is appropriately narrow for this stack layer. It prevents #391 from claiming or accepting nonempty state before the root implementation can prove it. The next root layer must replace that narrow recognizer with full semantic validation before any mutation can commit a nonempty root.

Constraints for the dependent layers:

  1. Preserve one authoritative wrapper within the host provider installation. A later process may reopen the file only after lifecycle ownership permits it; do not introduce a second long-lived wrapper as an inspection or mutation convenience.
  2. Keep all DOFS access performed during an XMD-owned top-level transaction inside the sanctioned DOFS/savepoint path. The vendored wrapper's transaction-depth/cache guard cannot observe a raw outer BEGIN; no later root traversal or restoration path may populate provider caches from uncommitted state and then survive an outer rollback.
  3. Generalize the one savepoint allocator to operation-spanning mutation savepoints and add transaction/generation fencing before exposing Workspace effects. Do not add ambient handle-global enlistment.
  4. Replace verifyEmptyWorkspace() only when the implementation can validate a complete canonical root, exact manifest/blob reachability, live-frontier equivalence, and restoration. A structurally valid but semantically inconsistent nonempty database remains corruption.
  5. Keep GC disabled until retained-root deletion is designed. Every event root must continue to protect the manifests and blobs needed to restore it.

The code and specification changes agree on the current boundary: this PR owns and recognizes the canonical empty Workspace, but does not expose mutation, publish nonempty roots, or claim the atomic effect coordinator. That makes #391 independently mergeable without prematurely weakening the later durability proof.

@taras
taras merged commit bc1f312 into main Aug 8, 2026
11 checks passed
@taras
taras deleted the agent/issue-365-2-connection-schema branch August 8, 2026 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Commit Workspace mutations and journal results atomically

1 participant