fix(local-dev): don't abort up on an already-applied sql/updates changeSet - #7076
Open
Yicong-Huang wants to merge 1 commit into
Open
fix(local-dev): don't abort up on an already-applied sql/updates changeSet#7076Yicong-Huang wants to merge 1 commit into
up on an already-applied sql/updates changeSet#7076Yicong-Huang wants to merge 1 commit into
Conversation
…angeSet On a fresh docker volume postgres applies sql/texera_ddl.sql itself — compose mounts sql/ into /docker-entrypoint-initdb.d — and that DDL is kept in sync with sql/updates/*, so the changeSets local-dev replays immediately afterwards re-create objects that are already there. 23-27 are incidentally idempotent and pass; 28.sql's dataset_owner_uid_name_key is not, so `up` died before the sbt build ever started. The `seed` path written for exactly this case was unreachable, because the container entrypoint always wins the race. Treat a psql failure whose every ERROR line is "already exists" as already-applied: record the changeSet and carry on. The check is deliberately narrow — a duplicate-key conflict is a data problem, and a failure with no ERROR line at all is never assumed harmless — so an incomplete schema still stops the build instead of reaching jOOQ codegen. psql's stderr is also kept rather than sent to /dev/null, so a real failure no longer requires re-running the file by hand to find out why. Closes apache#7064
Contributor
Backport auto-label reportThis
|
Contributor
Automated Reviewer SuggestionsBased on the
|
aglinxinyuan
approved these changes
Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
bin/local-dev.sh upagainst a fresh docker volume never reached the sbt build: it died on the lastsql/updateschangeSet.Postgres applies
sql/texera_ddl.sqlitself — compose mountssql/into/docker-entrypoint-initdb.d— and that DDL is kept in sync withsql/updates/*, so the changeSets local-dev replays immediately afterwards re-create objects that are already there. 23–27 are incidentally idempotent and pass;28.sql'sdataset_owner_uid_name_keyis not, andsql/texera_ddl.sql'sUNIQUE (owner_uid, name)ondatasetalready created it under exactly that auto-generated name.infra_ensure_db_schemapicks seed-vs-replay by probing for thefeedbacktable. On a fresh volume the entrypoint has just created it, so the replay path is taken — and theseedbranch written for this very case (record every changeSet as applied without executing it) is unreachable, because the entrypoint always wins the race.The fix is in the replay loop rather than the probe: a psql failure whose every
ERROR:line isalready existsmeans the changeSet's effect is already in the schema, so record it and carry on. That covers the nextsql/updates/N.sqlthat isn't accidentally idempotent too, instead of fixing only28.sql._sql_errors_all_already_existis deliberately narrow — aduplicate keyis a data conflict rather than an applied schema change, and a failure with noERROR:line at all is never assumed harmless — so an incomplete schema still stops the build instead of reaching jOOQ codegen with tables that aren't there.While in the same lines: psql's stderr is kept instead of redirected to
/dev/null. It holds the one line that explains the abort, and the old code discarded it and then told the operator to re-run the file by hand to find out why.Any related issues, documentation, discussions?
Closes #7064
How was this PR tested?
Unit coverage for the detector in both directions, plus two structural guards on the wiring, in the existing
infra-job suite:The pytest half of the same job reports
1 failed, 42 passedon this branch. That failure istest_is_dirty_after_seed_then_edit, which is unrelated to this change — it reproduces identically on a pristinemaincheckout, and this PR touches neithertui.pynor that test. It is #7075, fixed separately.End-to-end on the real stack (Ubuntu 24.04.4, docker 29.1.3), reproducing the failure and then confirming the fix:
Before:
After:
The changeSet is recorded, so it is not retried on the next run:
Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5)