Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Patch for mssql source cdc update #13168

Merged
merged 30 commits into from
May 25, 2022
Merged

Patch for mssql source cdc update #13168

merged 30 commits into from
May 25, 2022

Conversation

tuliren
Copy link
Contributor

@tuliren tuliren commented May 25, 2022 •
edited
Loading

What

  • Update spec field names and descriptions.
    • We cannot reuse the replication_method field name, because it will break all existing MS SQL connectors.
    • is_cdc_only is changed to data_to_sync enum with two values: Existing and New, New Changes Only.
      • The enums are easier to understand then is_cdc_only.
      • Also in the future, we may want to support the initial_only mode in Debezium.
    • is_disable_snapshot is changed to snapshot_isolation enum with two values: Snapshot, Read Committed.
      • In the future, we may want to support more isolation levels. So an enum is better.
  • Update connector documentation.
  • Format code.
  • Add more unit tests.
  • This is a follow up PR for Source MSSQL: add option to disable Snapshot mode and initial backup #12759
    • Merge the parent PR first.
    • Then merge this PR.
  • Will publish the connector in a separate PR.

@github-actions github-actions bot added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels May 25, 2022
@tuliren tuliren marked this pull request as ready for review May 25, 2022 12:18
@tuliren tuliren mentioned this pull request May 25, 2022
Merged
38 tasks
@tuliren
Copy link
Contributor Author

tuliren commented May 25, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-mssql

🕑 connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/2385014489
✅ connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/2385014489
No Python unittests run

Build Passed

Test summary info:

All Passed

@tuliren
Copy link
Contributor Author

tuliren commented May 25, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-mssql-strict-encrypt

🕑 connectors/source-mssql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/2385047021
✅ connectors/source-mssql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/2385047021
No Python unittests run

Build Passed

Test summary info:

All Passed

@tuliren tuliren requested a review from subodh1810 May 25, 2022 14:27
@tuliren tuliren merged commit 868ed76 into master May 25, 2022
@tuliren tuliren deleted the liren/sivankumar-mssql-patch branch May 25, 2022 15:06
This was referenced May 25, 2022
Closed
Merged
@octavia-squidington-iii octavia-squidington-iii mentioned this pull request May 25, 2022
Merged
jscottpolevault pushed a commit to jscottpolevault/airbyte that referenced this pull request Jun 1, 2022
@tuliren @sivankumar86
Patch for mssql source cdc update (airbytehq#13168)
7f65724 
* MSSQL CDC feature to capture only changes option added

* MSSQL CDC feature to capture only changes option added

* added option to disable snapshot 

snapshot isolation setting is hardcode and I modified as user option

* recommitting

* docker version change

docker version change

* resolve conflict

* review 1 added

* review 1 added

* removed extra space

removed extra space

* Update spec.json

* Format java code

* Update doc and version

* Update is-cdc method and add test

* Update spec

* Update doc

* Update snapshot isolation check

* Add helper method to check snapshot level

* Introduce enums

* Format code

* Remove empty constant class

* Update expected spec

* Update spec

* Prevent npe

* Update changelog

* Use a different field name

* Fix missing return statement

Co-authored-by: Sivakumar Ramaswamy <sivakumar.ramaswamy@zip.co>
Co-authored-by: sivankumar86 <sivankumar86@users.noreply.github.com>
Co-authored-by: sivankumar86 <sivankumar86@gmail.com>
@marcosmarxm marcosmarxm mentioned this pull request Jun 2, 2022
Merged
38 tasks
@sh4sh sh4sh mentioned this pull request Jul 20, 2023
Closed
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@noahkawasaki-airbyte noahkawasaki-airbyte Awaiting requested review from noahkawasaki-airbyte

@subodh1810 subodh1810 Awaiting requested review from subodh1810

Assignees
No one assigned
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@tuliren @sivankumar86