feat: COPY ON CONFLICT (DO NOTHING / DO UPDATE) — AnalyticDB-compatible syntax - #1
Open
coo-moon wants to merge 6 commits into
Open
feat: COPY ON CONFLICT (DO NOTHING / DO UPDATE) — AnalyticDB-compatible syntax#1coo-moon wants to merge 6 commits into
coo-moon wants to merge 6 commits into
Conversation
added 6 commits
August 6, 2026 17:19
Add the DO ON CONFLICT clause to COPY FROM, compatible with Alibaba Cloud
AnalyticDB for PostgreSQL:
COPY table FROM ... DO ON CONFLICT DO NOTHING
When a row conflicts with a unique/exclusion constraint, it is skipped
instead of aborting the whole COPY. DO ON CONFLICT DO UPDATE is parsed
but rejected at startup with a clear 'not supported yet' error.
Implementation notes:
- gram.y: new standalone copy_on_conflict clause following the copy
options (works with both old-style and parenthesized option syntax)
- copy.c: defGetCopyOnConflictChoice() with fail-fast validation
(COPY TO rejected; unknown values rejected; DO UPDATE not supported)
- copyfrom.c: force CIM_SINGLE insertion when ON CONFLICT is specified;
per-tuple ExecCheckIndexConstraints() pre-check before insert; skipped
rows are counted and reported via NOTICE. Leaf partitions get their
speculative index info (BuildSpeculativeIndexInfo) filled in on first
use because COPY's ModifyTable plan node is NULL.
- Target table (non-partitioned) must have at least one unique index;
checked at startup.
Limitations (documented): conflict check is a pre-check (TOCTOU window
under concurrency); partitioned tables enforce per-leaf uniqueness;
DO UPDATE is not implemented yet.
Covers: primary key conflict skip, no-conflict passthrough, startup rejection without unique constraint, COPY TO rejection, DO UPDATE 'not supported yet' error, multi-column unique index, partitioned tables (per-leaf uniqueness), ON_ERROR ignore interaction, parenthesized option syntax, and unknown on_conflict value rejection.
Adds the DO ON CONFLICT clause to the COPY synopsis and a parameter entry describing DO NOTHING semantics, the DO UPDATE not-yet-implemented error, the unique-constraint requirement, per-partition uniqueness for partitioned tables, and the pre-check concurrency caveat.
DO ON CONFLICT DO UPDATE now overwrites the conflicting row with the input row, matching AnalyticDB semantics (no SET targets, full-row overwrite). Reuses the IvorySQL ModifyTable update machinery (ExecUpdatePrologue/Act/Epilogue) instead of a hand-written path: - copyfrom.c: on conflict, switch the ModifyTable operation to CMD_UPDATE, run the update via the three-part ExecUpdate API, count updated rows, and report via NOTICE. es_output_cid is set at COPY start so rows inserted by earlier commands of the transaction are visible to the update. A CMD_UPDATE transition capture state is created when update transition tables exist. - nodeModifyTable.c: make the INSERT ON CONFLICT DO UPDATE cross-partition check NULL-safe (COPY's ModifyTable plan node is NULL). Cross-partition moves remain rejected, matching INSERT ON CONFLICT DO UPDATE semantics. - copy.c: defGetCopyOnConflictChoice() now accepts 'update'. Partitioned tables: PG requires partition key columns in unique constraints, so a conflict implies the same partition key and cross-partition moves are unreachable; the defensive partition check is kept. Concurrent modification of a conflicting row raises an error (fail-loud, no EPQ retry in COPY).
…igned) Rework COPY ON CONFLICT to match the semantics of INSERT ... ON CONFLICT and of Zbyte/Relyt's Phoenix implementation: - After the ExecCheckIndexConstraints pre-check passes, rows are inserted speculatively (table_tuple_insert_speculative + ExecInsertIndexTuples with EIIT_NO_DUPE_ERROR). A concurrent conflicting insert is detected atomically by the index insert, the speculative row is backed out, and the conflict check is re-run - eliminating the check-to-insert (TOCTOU) window of the previous pre-check-only design. - CopyOnConflictUpdate() (ported from Phoenix's copy.c) locks the conflicting tuple with table_tuple_lock before updating, waits out concurrent transactions, and returns 'retry' on TM_Updated/TM_Deleted so the whole conflict check re-runs instead of failing. - Duplicate constrained values within one COPY now raise the standard 'ON CONFLICT DO UPDATE command cannot affect row a second time' error (TM_Invisible + current xid detection), matching INSERT ON CONFLICT. - estate->es_snapshot is set at COPY start (needed by table_tuple_lock). Regression tests: 19 cases, including two new batch-duplicate-key cases (DO UPDATE error, DO NOTHING skip). Concurrent smoke test: two COPYs updating the same row both succeed (the second waits and retries).
- copyfrom.c: move es_output_cid/es_snapshot setup below the declaration block (-Wdeclaration-after-statement) - copyfrom.c: use uint64 for skipped/updated counters (match PRIu64) - copyfrom.c: allocate EvalPlanQualInit/MakeTransitionCaptureState in the per-tuple context (per-row state no longer leaks for the whole COPY duration) - copyfrom.c: CopyOnConflictUpdate() now distinguishes retry (0), updated (1) and BEFORE-trigger-suppressed (2); suppressed updates are no longer counted as updated, and the progress counter uses 'processed' uniformly - copyfrom.c: speculative insert now passes ti_options (FREEZE etc.) and keeps recheckIndexes for the AFTER ROW INSERT triggers - copyfrom.c: startup check accepts exclusion constraints too, matching ExecCheckIndexConstraints() and the docs - doc: copy.sgml now documents unique or exclusion constraint requirement - test: cleanup drops u8/u9 tables and the u7 trigger function
This was referenced Aug 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
COPY ON CONFLICT
Adds Oracle/AnalyticDB-compatible
COPY ... ON CONFLICTsyntax to support idempotent bulk upserts viaCOPY FROM.Syntax
Implementation
COPY ... ON CONFLICT DO NOTHING— skips rows that violate a unique/exclusion constraint (likeINSERT ... ON CONFLICT DO NOTHING)COPY ... ON CONFLICT DO UPDATE— overwrites existing rows on conflict (full-row update)Files changed
gram.y(new syntax),copy.hcopyfrom.c(speculative insert + ON CONFLICT path),nodeModifyTable.ccopy.sgmlcopy_on_conflict.sql+.out,parallel_scheduleRegression tests
Full test suite:
copy_on_conflict— covers DO NOTHING, DO UPDATE, partition tables, exclusion constraints, batch-internal duplicate keys.Commits (squash-able)
feat: support COPY ON CONFLICT DO NOTHINGtest: add regression tests for COPY ON CONFLICT DO NOTHINGdoc: document COPY ON CONFLICT in copy.sgmlfeat: implement COPY ON CONFLICT DO UPDATEfeat: atomic conflict detection via speculative insertionfix: address CodeRabbit review commentsContext
This feature was originally developed for the main IvorySQL repo but is non-Oracle-compatible (inspired by AnalyticDB syntax). Moved to IvorySQL-Labs as an experimental feature per project decision.