Skip to content

fix: Postgres SQL export/import round-trip (#1114)#1126

Merged
datlechin merged 15 commits intomainfrom
fix/1114-postgres-sql-roundtrip
May 8, 2026
Merged

fix: Postgres SQL export/import round-trip (#1114)#1126
datlechin merged 15 commits intomainfrom
fix/1114-postgres-sql-roundtrip

Conversation

@datlechin
Copy link
Copy Markdown
Member

Summary

Closes #1114. Postgres dumps produced by TablePro now round-trip cleanly through TablePro itself.

Three classes of bug were in the same flow:

  • Parser: SQLFileParser treated \' as a MySQL-style escape inside single-quoted strings, but Postgres dumps are written under standard_conforming_strings=on semantics where backslash is literal. A value ending in \ followed by another value containing ; desynced the parser and yielded a truncated INSERT that PG rejected with unterminated quoted string. The parser is now dialect-aware (postgres / mysql / sqlite / generic) with E-string opt-in (E'…'), tagged dollar-quoted strings ($tag$ … $tag$ with $1$ correctly rejected as a parameter), and dialect-gated # comments and DELIMITER handling.

  • Exporter: identity columns (GENERATED ALWAYS AS IDENTITY) and computed columns (GENERATED ALWAYS AS (…) STORED) were emitted into INSERT lists; FK constraints sat inline in CREATE TABLE so child tables could fail when sorted before parents; sequences exported as CREATE SEQUENCE START WITH X never resynced. Export now runs in four phases — DROP (reverse FK-dep + CASCADE), CREATE (forward FK-dep, no FK clauses, with GENERATED … AS IDENTITY preserved via pg_attribute.attidentity), data INSERT (with OVERRIDING SYSTEM VALUE only when at least one identity-always column exists, generated columns filtered out), and ALTER ADD CONSTRAINT FK + per-identity pg_catalog.setval(pg_get_serial_sequence(...), MAX(col)) to advance implicit sequences past the imported max id.

  • Performance and UX: parser was per-unichar CFStringAppendCharacters with an unbounded AsyncThrowingStream (~900 MB / 100 % CPU on a 17 MB file). Now uses run-length substring appends and bufferingPolicy: .bufferingNewest(8), plus a UTF-8 tail buffer for chunks straddling the 64 KB read boundary. PG plugin's foreignKeyDisableStatements now returns SET session_replication_role = replica instead of nil. Tables sidebar refreshes after import — the .refreshData notification is now posted from the success sheet's onDismiss so the main window is key when observeKeyWindowOnly checks.

Plugin ABI bump

PluginColumnInfo gained isIdentity, identityKind, isGenerated (all defaulted, source-compatible). currentPluginKitVersion bumped 9 → 10 across all 21 plugin Info.plists. Built-in plugins ship with the app and rebuild together. User-installable plugins (MongoDB, Oracle, DuckDB, MSSQL, Cassandra, Etcd, CloudflareD1, DynamoDB, BigQuery, LibSQL) need separate plugin-*-v* releases tagged from main after this merges; until they ship, those plugins gracefully reject load with the existing "ABI mismatch" path at PluginManager.swift:442 rather than crashing.

Test plan

  • SQLFileParserTests — 16 cases covering each dialect, dollar quotes, E-strings, $1$ rejection, doubled-quote escape, multi-byte UTF-8 at chunk boundary, large multi-row INSERT, and the verbatim Postgresql export then import fails #1114 repro fixture
  • End-to-end against a local Postgres seeded with FK + identity columns + parser-stress rows: export to .sql.gz, drop tables, re-import, all 1002 rows present including the poison rows with backslashes
  • Verify exported dump shape: 4-phase output, GENERATED ALWAYS AS IDENTITY clauses preserved, 2 pg_catalog.setval calls in phase D
  • Sequence resync proof: post-import INSERT … RETURNING id returns MAX(id) + 1 (no PK collision)
  • CI green
  • Reviewer to spot-check a non-trivial PG schema with composite FKs and views

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

return parser.parseFile(url: fileURL, encoding: encoding)

P1 Badge Pass the selected SQL dialect into the import parser

When importing PostgreSQL SQL files, this path ignores the dialect stored on SqlFileImportSource and calls parseFile with its default .generic dialect. The dialog's statement count now uses the PostgreSQL dialect, but the actual import still treats $$...$$ function bodies as generic SQL and will split on semicolons inside them, so a dump containing dollar-quoted PL/pgSQL can be counted correctly and then fail during import. Use parser.parseFile(url: fileURL, encoding: encoding, dialect: dialect) here.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@datlechin datlechin merged commit ab56cee into main May 8, 2026
2 checks passed
@datlechin datlechin deleted the fix/1114-postgres-sql-roundtrip branch May 8, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Postgresql export then import fails

1 participant