roachtest: add pg_dump round-trip compatibility test #167543
Draft
rafiss wants to merge 4 commits intocockroachdb:masterfrom
Draft
roachtest: add pg_dump round-trip compatibility test #167543rafiss wants to merge 4 commits intocockroachdb:masterfrom
rafiss wants to merge 4 commits intocockroachdb:masterfrom
Conversation
Contributor
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
|
Your pull request contains more than 1000 changes. It is strongly encouraged to split big PRs into smaller chunks. It looks like your PR touches SQL parser code but doesn't add or edit parser tests. Please make sure you add or edit parser tests if you edit the parser. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
Member
|
This change is |
Add a weekly roachtest that validates pg_dump round-trip compatibility with CockroachDB. The test builds pg_dump from PostgreSQL source (REL_17_4), creates a representative schema in a source database, dumps it with `pg_dump --schema-only` using the `pg_dump_compatibility` session variable, restores into a target database via psql, dumps again, and compares the two outputs. Known deviations are tracked via an expected-diff baseline file in testdata, following the same pattern as the pg_regress roachtest. The current baseline captures one known issue: pg_dump emits a view before the table it depends on, causing the view to be lost during restore. The test covers enum types, tables with various column types, constraints, secondary indexes (including partial and covering), foreign keys, sequences, views, comments, multi-schema objects, and default expressions. Resolves: cockroachdb#167442 Epic: CRDB-28751 Release note: None
Harden the pg_dump roachtest in three ways: 1. **Expanded schema coverage**: Add identity columns, generated/computed columns, composite primary keys, self-referencing foreign keys, multi-level view dependencies (view → view → table), and tables with multiple foreign keys. These exercise pg_dump code paths that the original test didn't cover. 2. **Content verification**: Add `verifyDumpContent` which checks that the source dump contains expected patterns (CREATE TABLE, CREATE VIEW, FOREIGN KEY, COMMENT ON, etc.). This catches cases where pg_dump silently omits objects from both dumps — a gap the round-trip diff comparison cannot detect. Patterns can be marked `knownFailure` to log warnings without failing the test, providing visibility into known compatibility gaps. Currently FK constraints and comments are known-missing. 3. **Restore error tracking**: Add `parseRestoreErrors` to extract ERROR lines from psql stderr and write them to a `restore_errors.txt` artifact. Previously, restore errors were invisible when psql exited 0. Update the expected.diff baseline for the identity column round-trip issue (pg_dump emits nextval() referencing a sequence before the sequence is created, causing restore failure). Epic: CRDB-28751 Release note: None Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
Create 3100 filler tables in a throwaway database before creating the test schema, pushing descriptor IDs into the range of PostgreSQL catalog OIDs (1247–6104). This verifies that the tableoid remapping in pg_dump_compatibility mode doesn't cause confusion when user table descriptor IDs collide with standard PG catalog OIDs like pg_class (1259) or pg_type (1247). Epic: CRDB-28751 Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
|
Detected infrastructure failure (matched: self-hosted runner lost communication with the server). Automatically rerunning failed jobs. (run link) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a weekly roachtest that validates pg_dump round-trip compatibility
with CockroachDB. The test builds pg_dump from PostgreSQL source
(REL_17_4), creates a representative schema in a source database,
dumps it with
pg_dump --schema-onlyusing thepg_dump_compatibilitysession variable, restores into a target database via psql, dumps
again, and compares the two outputs.
Known deviations are tracked via an expected-diff baseline file in
testdata, following the same pattern as the pg_regress roachtest. The
current baseline captures one known issue: pg_dump emits a view
before the table it depends on, causing the view to be lost during
restore.
The test covers enum types, tables with various column types,
constraints, secondary indexes (including partial and covering),
foreign keys, sequences, views, comments, multi-schema objects,
and default expressions.
Resolves: #167442
Epic: CRDB-28751
Release note: None