Skip to content

Conversation

@lovromazgon
Copy link
Contributor

@lovromazgon lovromazgon commented Jul 19, 2022

Description

Updates SDK to the latest version and integrates OpenCDC record into the connector. I also took the opportunity to fix some inconsistencies, for example:

  • destination test now uses global test utilities to prepare the database
  • key column is included in payload in logrepl.SnapshotIterator to match CDC behavior and longpoll.SnapshotIterator
  • position in longpoll.SnapshotIterator was changed to match position in logrepl.SnapshotIterator
  • decoding of values was simplified and uses info from the library instead of our pgutil package which was now removed
  • introduced constants for metadata and config fields - metadata field for table now has the prefix postgres.* to prevent conflicts with metadata from other connectors
  • global statement builders were replaced with local ones for each struct that needs it

Code should be a bit simpler now, this can already be seen from the line number diff.

Depends on ConduitIO/conduit-connector-sdk#31 (needs to be merged and we need to update the connector SDK version).

Related to ConduitIO/conduit#512.

Quick checks:

  • I have followed the Code Guidelines.
  • There is no other pull request for the same update/change.
  • I have written unit tests.
  • I have made sure that the PR is of reasonable size and can be easily reviewed.

@lovromazgon lovromazgon changed the title [WIP] OpenCDC integration OpenCDC integration Jul 21, 2022
@lovromazgon lovromazgon marked this pull request as ready for review July 21, 2022 17:44
@lovromazgon lovromazgon requested a review from a team as a code owner July 21, 2022 17:44
@lovromazgon lovromazgon changed the title OpenCDC integration OpenCDC integration + other fixes Jul 21, 2022
Copy link
Contributor

@hariso hariso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

Comment on lines +74 to +79
for i, col := range cfg.Columns {
if col == cfg.KeyColumn {
s.keyColumnIndex = i
}
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@lovromazgon lovromazgon merged commit d35314b into main Aug 5, 2022
@lovromazgon lovromazgon deleted the lovro/opencdc branch August 5, 2022 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants