🎉 MySQL Source : Expose serverTimezone debezium option via MySQL Source spec for CDC by akashkulk · Pull Request #17815 · airbytehq/airbyte

Akash Kulkarni (akashkulk) · 2022-10-11T00:37:23Z

What

Currently, CDC mode does not work when when the source MySQL db is set up with a server timezone that is not recognized by the ZoneID Java library (e.g. CEST, PDT).

The fix here is to expose an server timezone option in the CDC replication methods UI and add it to the debezium serverTimezone property while reading from MySQL in CDC mode.

🚨 User Impact 🚨

User will see a new option to add server timezone, with messaging that this is an 'advanced' option - only to be used when the server timezone is not IANNA compliant.

https://www.loom.com/i/2c1d080ab76541419782d508a22ab98b

github-actions · 2022-10-11T00:38:13Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

Akash Kulkarni (akashkulk) · 2022-10-11T00:39:20Z

/test connector=connectors/source-mysql-strict-encrypt

🕑 connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3223298892
❌ connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3223298892
🐛 https://gradle.com/s/gfrkyuq7cl2oa

Build Failed

Test summary info:

Could not find result summary

Akash Kulkarni (akashkulk) · 2022-10-11T00:39:28Z

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3223299137
❌ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3223299137
🐛 https://gradle.com/s/23ht6kdibqe26

Build Failed

Test summary info:

Could not find result summary

github-actions · 2022-10-11T03:30:08Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql-strict-encrypt
source-mysql

Akash Kulkarni (akashkulk) · 2022-10-11T03:31:08Z

/test connector=connectors/source-mysql-strict-encrypt

🕑 connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3223916999
✅ connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3223916999
No Python unittests run

Build Passed

Test summary info:

All Passed

Akash Kulkarni (akashkulk) · 2022-10-11T03:31:22Z

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3223918088
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3223918088
No Python unittests run

Build Passed

Test summary info:

All Passed

github-actions · 2022-10-11T16:03:35Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

github-actions · 2022-10-11T16:23:11Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql-strict-encrypt
source-mysql

github-actions · 2022-10-11T16:23:27Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

github-actions · 2022-10-11T16:26:01Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql-strict-encrypt
source-mysql

github-actions · 2022-10-11T16:27:15Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

github-actions · 2022-10-11T16:27:58Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

github-actions · 2022-10-11T23:56:35Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

github-actions · 2022-10-12T00:02:30Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

Akash Kulkarni (akashkulk) · 2022-10-12T00:03:32Z

/test connector=connectors/source-mysql-strict-encrypt

🕑 connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3231075194
✅ connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3231075194
No Python unittests run

Build Passed

Test summary info:

All Passed

github-actions · 2022-10-12T00:03:33Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

github-actions · 2022-10-12T00:03:56Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql-strict-encrypt
source-mysql

Akash Kulkarni (akashkulk) · 2022-10-12T15:11:20Z

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3235809223
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3235809223
No Python unittests run

Build Passed

Test summary info:

All Passed

github-actions · 2022-10-12T15:12:04Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

Akash Kulkarni (akashkulk) · 2022-10-12T16:37:54Z

/publish connector=connectors/source-mysql run-tests=false

🕑 Publishing the following connectors:
connectors/source-mysql
https://github.com/airbytehq/airbyte/actions/runs/3236373815

Connector	Did it publish?	Were definitions generated?
connectors/source-mysql	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

Akash Kulkarni (akashkulk) · 2022-10-12T16:38:04Z

/publish connector=connectors/source-mysql-strict-encrypt run-tests=false

🕑 Publishing the following connectors:
connectors/source-mysql-strict-encrypt
https://github.com/airbytehq/airbyte/actions/runs/3236375145

Connector	Did it publish?	Were definitions generated?
connectors/source-mysql-strict-encrypt	✅	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

github-actions · 2022-10-12T16:50:32Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql-strict-encrypt
source-mysql

github-actions · 2022-10-12T16:52:42Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql-strict-encrypt
source-mysql

…ce spec for CDC (airbytehq#17815) * Expose serverTimezone debezium option via MySQL Source spec for CDC * Fix tests * Bump dockerfile version * Update Dockerfile * Update mysql.md * Update mysql.md * Documentation * Code Review Comments * Addressing comments * Addressing comments * Addressing comments * auto-bump connector version [ci skip] Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

Expose serverTimezone debezium option via MySQL Source spec for CDC

9e5fe30

github-actions Bot added the area/connectors Connector related issues label Oct 11, 2022

Octavia Squidington IV (octavia-squidington-iv) added the connectors/source/mysql label Oct 11, 2022

Akash Kulkarni (akashkulk) changed the title ~~Expose serverTimezone debezium option via MySQL Source spec for CDC~~ 🎉 Expose serverTimezone debezium option via MySQL Source spec for CDC Oct 11, 2022

Fix tests

9df57a6

Octavia Squidington IV (octavia-squidington-iv) added the connectors/source/mysql-strict-encrypt label Oct 11, 2022

Merge branch 'master' into akash/cdc-tz

24c591c

Akash Kulkarni (akashkulk) added 2 commits October 11, 2022 09:22

Bump dockerfile version

e494bba

Update Dockerfile

7553d02

Update mysql.md

9767336

github-actions Bot added the area/documentation Improvements or additions to documentation label Oct 11, 2022

Akash Kulkarni (akashkulk) added 2 commits October 11, 2022 09:26

Update mysql.md

8620f41

Merge branch 'master' into akash/cdc-tz

aff43fa

Akash Kulkarni (akashkulk) marked this pull request as ready for review October 11, 2022 16:27

Akash Kulkarni (akashkulk) requested a review from a team as a code owner October 11, 2022 16:27

Akash Kulkarni (akashkulk) requested review from Rodi Reich Zilberman (rodireich) and Subodh Chaturvedi (subodh1810) October 11, 2022 16:27

Documentation

068fd16

Addressing comments

0bf8b88

Addressing comments

163a0dc

Akash Kulkarni (akashkulk) added 2 commits October 11, 2022 17:02

Addressing comments

f096c08

Merge branch 'master' into akash/cdc-tz

7945e53

Merge branch 'master' into akash/cdc-tz

5dc920a

Akash Kulkarni (akashkulk) changed the title ~~🎉 Expose serverTimezone debezium option via MySQL Source spec for CDC~~ 🎉 MySQL Source : Expose serverTimezone debezium option via MySQL Source spec for CDC Oct 12, 2022

Octavia Squidington III (octavia-squidington-iii) and others added 2 commits October 12, 2022 16:48

auto-bump connector version [ci skip]

93ad9cd

Merge branch 'master' into akash/cdc-tz

3cace3e

Akash Kulkarni (akashkulk) temporarily deployed to more-secrets October 12, 2022 16:51 Inactive

Merge branch 'master' into akash/cdc-tz

9f372ee

Akash Kulkarni (akashkulk) merged commit e1701c9 into master Oct 12, 2022

Akash Kulkarni (akashkulk) deleted the akash/cdc-tz branch October 12, 2022 16:53

Akash Kulkarni (akashkulk) temporarily deployed to more-secrets October 12, 2022 16:53 Inactive

This was referenced Oct 13, 2022

Bump Airbyte version from 0.40.14 to 0.40.15 #17961

Closed

Bump Airbyte version from 0.40.14 to 0.40.15 #17970

Merged

michaelsonnle (michaelsonnle) mentioned this pull request Mar 29, 2023

Source MySQL - sync CDC does not work because of changed mysql server timezone #24659

Closed

Uh oh!

Conversation

Akash Kulkarni (akashkulk) commented Oct 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

🚨 User Impact 🚨

Uh oh!

github-actions Bot commented Oct 11, 2022

Uh oh!

Akash Kulkarni (akashkulk) commented Oct 11, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Failed

Uh oh!

Akash Kulkarni (akashkulk) commented Oct 11, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Failed

Uh oh!

github-actions Bot commented Oct 11, 2022

Uh oh!

Akash Kulkarni (akashkulk) commented Oct 11, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

Akash Kulkarni (akashkulk) commented Oct 11, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

github-actions Bot commented Oct 11, 2022

Uh oh!

github-actions Bot commented Oct 11, 2022

Uh oh!

github-actions Bot commented Oct 11, 2022

Uh oh!

github-actions Bot commented Oct 11, 2022

Uh oh!

github-actions Bot commented Oct 11, 2022

Uh oh!

github-actions Bot commented Oct 11, 2022

Uh oh!

github-actions Bot commented Oct 11, 2022

Uh oh!

github-actions Bot commented Oct 12, 2022

Uh oh!

Akash Kulkarni (akashkulk) commented Oct 12, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

github-actions Bot commented Oct 12, 2022

Uh oh!

github-actions Bot commented Oct 12, 2022

Uh oh!

Akash Kulkarni (akashkulk) commented Oct 12, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

github-actions Bot commented Oct 12, 2022

Uh oh!

Akash Kulkarni (akashkulk) commented Oct 12, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Akash Kulkarni (akashkulk) commented Oct 12, 2022 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 12, 2022

Uh oh!

github-actions Bot commented Oct 12, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Akash Kulkarni (akashkulk) commented Oct 11, 2022 •

edited

Loading

Akash Kulkarni (akashkulk) commented Oct 11, 2022 •

edited by github-actions Bot

Loading

Akash Kulkarni (akashkulk) commented Oct 11, 2022 •

edited by github-actions Bot

Loading

Akash Kulkarni (akashkulk) commented Oct 11, 2022 •

edited by github-actions Bot

Loading

Akash Kulkarni (akashkulk) commented Oct 11, 2022 •

edited by github-actions Bot

Loading

Akash Kulkarni (akashkulk) commented Oct 12, 2022 •

edited by github-actions Bot

Loading

Akash Kulkarni (akashkulk) commented Oct 12, 2022 •

edited by github-actions Bot

Loading

Akash Kulkarni (akashkulk) commented Oct 12, 2022 •

edited by github-actions Bot

Loading

Akash Kulkarni (akashkulk) commented Oct 12, 2022 •

edited by github-actions Bot

Loading