oracledb_cdc: handle LOB_TRIM redo SQL separately from SELECT_LOB_LOCATOR#4430
Conversation
d6470ec to
b6945cf
Compare
|
Commits Review LGTM |
b6945cf to
8476c42
Compare
Jeffail
left a comment
There was a problem hiding this comment.
The storage-variant breakdown in the description and the integration tests across SecureFile, BASICFILE, and BASICFILE out-of-row are a meaningful improvement, and the dbms_lob.trim(loc, N) finalisation vs. clear-then-write distinction is now correctly drawn. Approving overall.
A few non-blocking points worth surfacing:
1. Pass 3 fallback (internal/impl/oracledb/logminer/sqlredo/lob.go:189-201) — the code unconditionally selects the most-recent matching event, while the comment describes "exactly one candidate ... otherwise most recent". Largely a documentation cleanup, but there is a latent edge case: if a transaction contains two LOB-only UPDATEs on the same table for different rows and Oracle strips PK columns from both WHERE clauses, both accumulators bypass Pass 1/2 and converge on the same Pass 3 target — one row's LOB silently overwrites the other's. Likely rare, but worth either tightening the predicate or adding a warn-log when multiple accumulators target the same event.
2. LOB_TRIM(N) warning predicate (internal/impl/oracledb/logminer/logminer.go:362-371) — the warning only fires when N > 0 && len(Fragments) == 0. Two adjacent cases also produce a potentially incorrect assembled value but do not warn:
N < M(writes extend past N):Assembleproduces M bytes; the real LOB is N bytes.N > MwithM > 0: a preserved prefix exists beyond what we have.
Assemble is also not currently truncated to N. The current scope catches the most blatant case (nothing to publish at all) and is defensible given SecureFile's typical full-rewrite UPDATE pattern; worth noting in the comment that this is an explicit tradeoff rather than a complete check.
3. Synthetic UPDATE shape (internal/impl/oracledb/logminer/logminer.go:433-442) — the synthesized event publishes only {<pk_columns>, <lob_column>}; other row columns are unavailable from redo. Downstream consumers expecting a full-row UPDATE will see a sparse payload. Is sparse-row UPDATE the intended downstream contract? It may be worth documenting, or carrying over column values from a prior INSERT/UPDATE in the same transaction when one is present.
4. (Minor) Form A in OpLobTrim (logminer.go:340) overwrites state.Accumulators[key] unconditionally, whereas OpSelectLobLocator (line 298) uses if !exists. The comment "establish (or reset)" signals intent, but if Form A can ever fire after fragments have already accumulated for the same key, those fragments would be lost. Worth confirming Oracle does not emit that sequence.
None of the above are blockers — happy for this to land.
|
Commits
Review LGTM |
|
Commits
Review LGTM |
|
Commits
Commits 3 and 4 LGTM. Review One minor issue (inline): the new doc block describing Form B in |
63f35b8 to
354c606
Compare
|
Commits
Review The change cleanly separates LOB_TRIM handling from SELECT_LOB_LOCATOR, documents the three Oracle LOB storage variants thoroughly, and the integration tests cover each variant end-to-end. The Pass 3 schema/table fallback is appropriately conservative (only merging on a single unambiguous candidate). The synthesized UPDATE flow is well-documented. LGTM |
354c606 to
051cba4
Compare
…iants SecureFile (Oracle Free 23c default): emits LOB_WRITE(s) → LOB_TRIM(N). The trim finalises the value after fragments are written, so clearing fragments on LOB_TRIM was the bug. May emit no DML UPDATE — in that case a synthetic UPDATE is synthesized from the assembled accumulator. BASICFILE: emits LOB_TRIM(0) → LOB_WRITE(s) (clear-then-write). The DML UPDATE is always present; fragments arrive after the trim, so LOB_TRIM(0) must be a no-op against the fragment list. BASICFILE out-of-row (DISABLE STORAGE IN ROW): no SELECT_LOB_LOCATOR is emitted. The inferLOBLocator path creates the accumulator from an existing DML event rather than from the locator SQL, and the DML UPDATE is always present for PK matching.
Pass 3 of MergeLOBsIntoDMLEvents previously took the most-recent matching event unconditionally when no PK columns were available in the WHERE clause (ROWID-only). If two LOB-only UPDATEs for different rows in the same transaction both reached Pass 3, the second accumulator's value would silently overwrite the first on the same target event. Pass 3 now collects all candidate events for the schema/table first. It only merges when exactly one candidate exists — the unambiguous case. When multiple candidates exist, it warns and leaves the accumulator unmerged so the caller synthesizes a separate UPDATE per row instead. The comment previously described "exactly one candidate" behaviour that the code did not implement; they now agree.
051cba4 to
24fff5b
Compare
|
Commits
Review |
When updating an existing LOB field via a SQL
UPDATEcommand, OracleDB will handle that update in a number of ways.SecureFile (Oracle Free 23c default): emits LOB_WRITE(s) → LOB_TRIM(N). The trim finalises the value after fragments are written, so clearing fragments on LOB_TRIM was the bug. May emit no DML UPDATE — in that case a synthetic UPDATE is synthesized from the assembled accumulator.
BASICFILE: emits LOB_TRIM(0) → LOB_WRITE(s) (clear-then-write). The DML UPDATE is always present; fragments arrive after the trim, so LOB_TRIM(0) must be a no-op against the fragment list.
BASICFILE out-of-row (DISABLE STORAGE IN ROW): no SELECT_LOB_LOCATOR is emitted. The inferLOBLocator path creates the accumulator from an existing DML event rather than from the locator SQL, and the DML UPDATE is always present for PK matching.
This change ensures we support these three variants and includes a test to verify each.