fix(export): complete the portability promise — missing tables, SQL dump, real PKs (#8)

[keithfawcett]

Closes #8 from docs/audit-remaining-work.md. Not money-risky, but it's the one architectural promise the repo makes on its front page.

What wasn't true

CLAUDE.md principle #2 and the README promise every table exports to CSV + JSON + SQL, and a hosted export re-imports into self-host. Three gaps:

1. The exported set omitted PartnerProgram, PartnerCommission and Coupon. A restored instance had the partner roster but no program grants, no record of what any partnership was agreed at, and no coupon → (partner, program) mapping — so coupon-driven conversions couldn't be re-attributed.
2. No SQL dump existed, though the contract names it as one of the three formats.
3. Import hardcoded id as the conflict target, but PartnerCommission is keyed on partnerId — adding it naively would have thrown on every re-import.

Each table now carries its real primary key in EXPORT_TABLES, and that list doubles as the FK-safe import order.

Two more the round-trip test caught (both live bugs)

• A JS array coming out of select * is ambiguous: Program.commissionRule is a jsonb array, Program.categories is a text[]. The driver renders a bare array as a Postgres array literal, so importing any program with compound commission rules failed with invalid input syntax for type json. Both the import and the dump now read the live column types and render each correctly ('[{"type":"percent"}]' vs '{"saas","devtools"}').
• Same ambiguity in the dump: a text[] column was being emitted with ::jsonb.

The SQL dump

GET /export.sql (full bundle) and GET /export/<Table>.sql. Portable by default — rows are written under a psql variable, so one file restores into any instance:

psql "$DATABASE_URL" -v tenant_id=default -f openpartner-export.sql

It opens a transaction, set_config('app.tenant_id', …) so it works on the RLS-scoped app role as well as the privileged one, emits tables in import order, and every statement is ON CONFLICT (pk) DO NOTHING — idempotent and resumable, same guarantee as the JSON path. ?tenantId=<id> bakes a literal instead, for clients that don't run psql meta-commands.

Versioning

schemaVersion 1 → 2. POST /import still accepts v1 bundles — an export sitting on someone's disk must not become worthless because we added tables. SUPPORTED_IMPORT_VERSIONS only ever grows.

Tests

export-roundtrip.test.ts (11): seed one row in every exportable table with the FK chain intact → export → wipe → restore → compare. Once through JSON, once through the SQL dump actually executed against Postgres, and once through the portable dump put through psql's own substitution. Plus idempotency on both paths (the case that used to throw on PartnerCommission), v1 acceptance, PK/order invariants, and escaping of apostrophes, jsonb arrays, text[] and nulls.

Full suite: 274 passing. pnpm typecheck + pnpm lint clean. No migration.

Also

The portal export page listed 9 of the 14 tables and offered no SQL — now complete, with a SQL column and a SQL bundle button. docs/data-portability.md is the format contract: bundle shape, the table list with its keys, both restore procedures, the type-fidelity note, and the checklist for adding a table.

🤖 Generated with Claude Code

[keithfawcett]

Adversarial review pass (Codex, xhigh) — three real defects fixed in 2f4df82

All three were in the generated-artifact path, i.e. a file someone else executes:

1. Command injection via ?tenantId= (critical). The parameter was interpolated verbatim into a -- header comment. psql meta-commands are line-oriented, so ?tenantId=x%0A%5C!%20cmd%0A-- produced a dump containing a line-leading \! cmd, which psql runs as a shell command on the machine doing the restore. The row values were escaped; the comment wasn't. Now validated ([A-Za-z0-9_-]{1,64}, 400 on anything else), comments strip newlines regardless, and buildSqlDump asserts the same rule so no future caller can reintroduce it.

2. The documented restore could never have worked (high). The fallback was \set tenant_id 'default' and the docs said -v tenant_id=default — but default is the slug; tenantId is an FK to Tenant.id = 01J0000000DEFAULTTENANT0000. The first row failed the FK and rolled the restore back. Fixed everywhere (dump fallback, docs, README, portal). My original test masked it by substituting the real id before executing; there's now a test that runs the dump the way psql -f with no -v does.

3. standard_conforming_strings assumption (medium). The escaping assumed it's on (default since 9.1) without saying so; with it off a backslash in exported data breaks out of its literal. The dump now pins it in the preamble.

Plus: normalizeRow revived any ISO-shaped string as a Date, so a partner named 2026-08-09T00:00:00 was corrupted on import. Now that the importer reads column types, the coercion is gated on the column actually being a timestamp.

New tests: no--v restore path · string-mode pin · tenant-id refusal (including the \! payload) · header-comment containment · backslash/quote round-trip through the dump · timestamp-lookalike text.

⚠️ Merge order: this PR must land after #62

Exports rely entirely on RLS for tenant scoping (exportTable is an unfiltered select *). PartnerProgram — newly added to the export set here — has no RLS policy and no app-role grant on main: the rename migration never created them under the new name and packages/db/scripts/ensure-app-role.ts:33 still lists PartnerCampaign. Merging this first would make /export.json either 500 (no grant) or return every tenant's grants (grant, no policy). #62 adds exactly that policy + grant.

Known, not fixed here (follow-ups)

• ON CONFLICT (pk) doesn't cover natural keys — Link(tenantId,linkKey), Identity(clickId,userId), Event.externalEventId — so importing into a non-empty database throws instead of skipping. The round-trip test always wipes first.
• Config, PartnerPostback and BrandAsset are customer-owned data and still aren't exported, against a doc that says "every table".
• exportAll + buildSqlDump build the whole tenant in memory; a multi-million-row tenant needs streaming.
• Sub-millisecond timestamptz precision is lost through both paths (pg returns JS Date).