feat(experimental): Postgres session providers with Hyperdrive support

[mattzcarey]

Summary

Adds Postgres-backed session providers for storing conversation history, context blocks, and searchable knowledge in an external database via Hyperdrive. This enables cross-DO queries, analytics, and shared state without relying on DO SQLite.

What's new

Providers (packages/agents/src/experimental/memory/session/providers/)

• PostgresSessionProvider — tree-structured messages, compaction overlays, message FTS via tsvector
• PostgresContextProvider — writable context block storage (memory, cached prompt)
• PostgresSearchProvider — searchable knowledge base with tsvector + GIN index

Framework improvements (packages/agents/src/experimental/memory/session/)

• Session.create() accepts SessionProvider for external storage (in addition to SqlProvider for DO SQLite)
• Hardened context tools: removed enum constraints on label params, validate inside execute, always return error strings instead of throwing (prevents orphaned tool calls with smaller LLMs)
• Simplified set_context API: removed key param, auto-generates keys from title or content slug
• Fixed prompt lifecycle: freezeSystemPrompt() returns cached, refreshSystemPrompt() force-reloads from providers
• clearMessages() calls refreshSystemPrompt() to invalidate the cached prompt
• appendToBlock() adds newline separator between entries
• Empty writable blocks render in system prompt so the LLM knows they exist
• Clean block tags: [readonly], [searchable], [loadable], [not searchable]
• Search provider get() returns entry count only (no key listing)

Example (experimental/session-planetscale/)

• Full Vite + React chat app with Hyperdrive + pg driver
• System prompt toggle, FULLTEXT search bar, connection indicator, theme toggle
• wrapPgClient helper converts ? placeholders to $1, $2, ... for pg compatibility

Tests (packages/agents/src/tests/experimental/memory/session/postgres-providers.test.ts)

• 37 tests covering: provider CRUD, message round-trip, dynamic-tool part serialization, convertToModelMessages compatibility, prompt lifecycle (freeze/refresh/invalidation/concurrent)

Docs (docs/sessions.md)

• Postgres setup guide: migration SQL, Hyperdrive config, wire-up code
• System prompt lifecycle docs
• Search provider docs (message search + knowledge search)

Migration SQL

Customers run this once — providers never create tables:

CREATE TABLE assistant_messages (
  id TEXT PRIMARY KEY, session_id TEXT NOT NULL DEFAULT '', parent_id TEXT,
  role TEXT NOT NULL, content TEXT NOT NULL, created_at TIMESTAMPTZ DEFAULT NOW(),
  content_tsv TSVECTOR GENERATED ALWAYS AS (to_tsvector('english', content)) STORED
);
CREATE TABLE assistant_compactions (
  id TEXT PRIMARY KEY, session_id TEXT NOT NULL DEFAULT '',
  summary TEXT NOT NULL, from_message_id TEXT NOT NULL, to_message_id TEXT NOT NULL,
  created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE TABLE cf_agents_context_blocks (
  label TEXT PRIMARY KEY, content TEXT NOT NULL, updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE TABLE cf_agents_search_entries (
  label TEXT NOT NULL, key TEXT NOT NULL, content TEXT NOT NULL,
  content_tsv TSVECTOR GENERATED ALWAYS AS (to_tsvector('english', content)) STORED,
  created_at TIMESTAMPTZ DEFAULT NOW(), updated_at TIMESTAMPTZ DEFAULT NOW(),
  PRIMARY KEY (label, key)
);

Test plan

• 37 unit tests passing (providers, round-trip, convertToModelMessages, prompt lifecycle)
• Deploy example and test chat flow end-to-end
• Verify search_context returns ranked results from knowledge base
• Verify clearMessages invalidates cached prompt
• Verify empty memory block renders in system prompt

[changeset-bot]

⚠️ No Changeset found

Latest commit: a4435ab

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[devin-ai-integration]

Devin Review found 1 potential issue.

View 6 additional findings in Devin Review.

[devin-ai-integration]

🔴 PlanetScale searchMessages returns raw JSON blobs instead of extracted text content

The PlanetScaleSessionProvider.searchMessages() selects content from the assistant_messages table, which stores the full JSON.stringify(message) blob (set at planetscale.ts:166-170). This means both the FULLTEXT search and the returned SearchResult.content operate on raw JSON like {"id":"m1","role":"user","parts":[{"type":"text","text":"hello"}]} rather than the actual message text. In contrast, the reference AgentSessionProvider maintains a separate FTS5 table with extracted text parts (packages/agents/src/experimental/memory/session/providers/agent.ts:285-297) and queries that table for search (agent.ts:261-264). This causes two problems: (1) FULLTEXT search matches against JSON structure/keys (e.g., searching for "type" or "parts" would match every message), producing false positives, and (2) SearchResult.content returns an unparseable JSON string where callers expect readable text.

Prompt for agents

In packages/agents/src/experimental/memory/session/providers/planetscale.ts, the searchMessages method (lines 243-257) queries the assistant_messages table directly, which stores full JSON-serialized UIMessage objects. This causes both incorrect search matching (FULLTEXT on JSON blobs) and incorrect output (SearchResult.content is a JSON string instead of readable text).

To fix this properly, mirror the approach used by AgentSessionProvider (packages/agents/src/experimental/memory/session/providers/agent.ts:285-297):

1. Parse the JSON content in the search results to extract actual text parts, similar to how AgentSessionProvider.indexFTS extracts text:
   message.parts.filter(p => p.type === 'text').map(p => p.text).join(' ')

2. For the SearchResult.content field, parse each row's content JSON and extract the text parts instead of returning raw JSON.

3. Ideally, consider creating a separate search table (like the agent provider's assistant_fts) that stores extracted text for proper FULLTEXT indexing. Without this, FULLTEXT search will still match against JSON structure, but at minimum the output content should be parsed text.

At minimum, change lines 251-256 to parse the JSON content and extract text parts before returning.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[mattzcarey]

Superseded by #1297 (clean branch, rebased on main)