Skip to content

chore(db): drop legacy copilot_chats.messages JSONB column#4886

Merged
waleedlatif1 merged 1 commit into
stagingfrom
waleedlatif1/drop-jsonb-tool-column
Jun 4, 2026
Merged

chore(db): drop legacy copilot_chats.messages JSONB column#4886
waleedlatif1 merged 1 commit into
stagingfrom
waleedlatif1/drop-jsonb-tool-column

Conversation

@waleedlatif1
Copy link
Copy Markdown
Collaborator

Summary

  • Drop the dead copilot_chats.messages JSONB column (~5.7GB, ~99% TOAST) now that all reads and writes are fully on the normalized copilot_messages table.
  • Migration 0225_greedy_ares.sql: ALTER TABLE "copilot_chats" DROP COLUMN "messages" + drizzle snapshot/journal.
  • Re-typed CopilotChatDetailRow.messages as an assembled (non-column) field since it no longer comes from the chat row.

Production verification (before dropping)

  • No copilot_chats.messages writes in the last 24h (insights); recently-active chats have empty JSONB while copilot_messages holds the transcript.
  • The only messages-related read queries copilot_messages, not the column.

Deploy sequence (important)

  1. Reconcile any chats where the JSONB still leads copilot_messages (small tail of pre-dual-write chats) before applying this migration — otherwise those messages are lost.
  2. Apply 0225.
  3. pg_repack copilot_chats afterward to actually reclaim the ~5.7GB (DROP COLUMN is metadata-only).

Type of Change

  • Chore / maintenance (DB migration)

Testing

Tested manually: drizzle-kit generate produced a clean single-statement migration; a2a_task.messages untouched. tsc passes (apps/sim + packages/db), 133 copilot/mothership tests pass, biome clean.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Reads and writes are fully cut over to the normalized copilot_messages table
(verified in production: no writes to the column in 24h, recently-active chats
have empty JSONB while copilot_messages holds the transcript). Drop the dead
column via drizzle migration 0225 and re-type CopilotChatDetailRow.messages as
an assembled (non-column) field.

Deploy notes: reconcile any chats where the JSONB still leads copilot_messages
before applying, and pg_repack copilot_chats afterward to reclaim the ~5.7GB
TOAST storage (DROP COLUMN is metadata-only).
@vercel
Copy link
Copy Markdown

vercel Bot commented Jun 4, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment Jun 4, 2026 9:23pm

Request Review

@cursor
Copy link
Copy Markdown

cursor Bot commented Jun 4, 2026

PR Summary

Medium Risk
Irreversible schema change can lose transcript data for chats not reconciled to copilot_messages before migrate; runtime behavior already depends on the normalized table.

Overview
Removes the legacy copilot_chats.messages JSONB column via migration 0225_greedy_ares.sql, now that transcripts live only in copilot_messages.

CopilotChatDetailRow no longer treats messages as a Drizzle column pick; it is typed as an assembled messages: unknown[] field (filled by loadCopilotChatMessages at read time).

Deploy order matters: reconcile any chats where JSONB still leads copilot_messages before applying the migration, then run pg_repack on copilot_chats to reclaim disk after the metadata-only drop.

Reviewed by Cursor Bugbot for commit 0f4d158. Configure here.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Jun 4, 2026

Greptile Summary

This PR completes the migration of copilot chat transcript storage from a legacy messages JSONB column on copilot_chats (≈5.7 GB) to the normalized copilot_messages table that has been the sole write target during the dual-write period.

  • Migration 0225_greedy_ares.sql: single ALTER TABLE \"copilot_chats\" DROP COLUMN \"messages\" statement; Drizzle snapshot and journal updated to match.
  • packages/db/schema.ts: messages field removed from copilotChats table definition; no application code still references the column for reads or writes.
  • lifecycle.ts: CopilotChatDetailRow.messages re-typed as an assembled unknown[] (sourced from loadCopilotChatMessages) rather than a Pick of the now-gone DB column; all call sites already go through loadCopilotChatMessages.

Confidence Score: 5/5

Safe to merge once the pre-deploy reconciliation step from the PR description has been completed — the schema, Drizzle snapshot, journal, and application code are all consistent with the dropped column.

All five changed files are internally consistent: the migration drops exactly the column removed from schema.ts, the Drizzle snapshot matches, and lifecycle.ts correctly assembles the transcript from copilot_messages instead of the now-gone column. No application code writes to or reads from the legacy column, and the two key read paths (getAccessibleCopilotChat, getAccessibleCopilotChatWithMessages) both call loadCopilotChatMessages before returning.

No files require special attention. The operational pre-flight (reconcile script + pg_repack) is documented in the PR and is outside the diff.

Important Files Changed

Filename Overview
packages/db/migrations/0225_greedy_ares.sql Single irreversible DDL statement dropping the legacy messages JSONB column; correct and consistent with schema.ts and the snapshot.
apps/sim/lib/copilot/chat/lifecycle.ts CopilotChatDetailRow.messages re-typed as an assembled unknown[] field; both getAccessibleCopilotChat and getAccessibleCopilotChatWithMessages correctly populate it from loadCopilotChatMessages; new-chat path manually initialises messages: [].
packages/db/schema.ts messages column absent from copilotChats table definition; a2a_task.messages (unrelated) is correctly retained.
packages/db/migrations/meta/0225_snapshot.json Snapshot correctly reflects copilot_chats without the messages column; a2a_task.messages present as expected.
packages/db/migrations/meta/_journal.json Journal correctly appended with entry idx=225 for 0225_greedy_ares.

Sequence Diagram

sequenceDiagram
    participant Client
    participant API as API Route
    participant LC as lifecycle.ts
    participant DB as copilot_chats
    participant CM as copilot_messages

    Note over DB: messages JSONB column dropped (0225)

    Client->>API: GET /copilot/chat/:id
    API->>LC: getAccessibleCopilotChatWithMessages(chatId, userId)
    LC->>DB: SELECT (id, userId, title, ...) — no messages column
    DB-->>LC: chat metadata row
    LC->>CM: loadCopilotChatMessages(chatId)
    CM-->>LC: PersistedMessage[] ordered by seq
    LC-->>API: "{ ...chat, messages: PersistedMessage[] }"
    API-->>Client: chat + assembled transcript

    Client->>API: POST /copilot/chat/update-messages
    API->>DB: UPDATE copilot_chats SET updatedAt, planArtifact, config
    API->>CM: replaceCopilotChatMessages(chatId, messages)
    CM-->>API: OK
    API-->>Client: "{ success: true }"
Loading

Reviews (2): Last reviewed commit: "chore(db): drop legacy copilot_chats.mes..." | Re-trigger Greptile

Comment thread apps/sim/lib/copilot/chat/lifecycle.ts
@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1
Copy link
Copy Markdown
Collaborator Author

@cursor review

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 0f4d158. Configure here.

@waleedlatif1 waleedlatif1 merged commit 5505076 into staging Jun 4, 2026
14 checks passed
@waleedlatif1 waleedlatif1 deleted the waleedlatif1/drop-jsonb-tool-column branch June 4, 2026 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant