Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[core] Support print options for flink cdc sql connectors #2099

Merged
merged 1 commit into from
Jun 5, 2023
Merged

[core] Support print options for flink cdc sql connectors #2099

merged 1 commit into from
Jun 5, 2023

Conversation

TyrantLucifer
Copy link
Member

close #1742

@TyrantLucifer
Copy link
Member Author

Hi @GOODBOY008 , in this task I have changed the following things:

  1. Add dependency flink-cdc-base into connectors modules in order to code reused.
  2. In connector-db2 module I try to add flink-cdc-base but this will effact the compile flink-sql-connector-db2-cdc, the error message as the following shown:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:3.1.1:shade (shade-flink) on project flink-sql-connector-db2-cdc: Error creating shaded jar: Problem shading JAR /home/tyrantlucifer/.m2/repository/com/zaxxer/HikariCP/4.0.3/HikariCP-4.0.3.jar entry META-INF/versions/11/module-info.class: java.lang.IllegalArgumentException -> [Help 1]

So I add a same class into module, for this problem do you have any good advice?

ruanhang1993
ruanhang1993 reviewed May 9, 2023
View reviewed changes
Copy link
Contributor

@ruanhang1993 ruanhang1993 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TyrantLucifer Thanks for your contribution. I left some comments.

flink-cdc-base/src/main/java/com/ververica/cdc/connectors/base/utils/OptionUtils.java Outdated Show resolved Hide resolved
flink-cdc-base/src/test/java/com/ververica/cdc/connectors/base/utils/OptionUtilsTest.java Outdated Show resolved Hide resolved
...ctor-db2-cdc/src/main/java/com/ververica/cdc/connectors/db2/table/Db2TableSourceFactory.java Outdated Show resolved Hide resolved
ruanhang1993
ruanhang1993 reviewed Jun 5, 2023
View reviewed changes
Copy link
Contributor

@ruanhang1993 ruanhang1993 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe each module invoke the base module is not a good idea. Serval connectors should not rely on the base module before they are moved to the new incremental snapshot framework.

I think we should introduce separate OptionUtils for them.

flink-cdc-base/src/main/java/com/ververica/cdc/connectors/base/utils/OptionUtils.java Outdated Show resolved Hide resolved
flink-connector-db2-cdc/src/main/java/com/ververica/cdc/connectors/db2/utils/OptionUtils.java Show resolved Hide resolved
ruanhang1993

This comment was marked as duplicate.

Sign in to view

ruanhang1993
ruanhang1993 approved these changes Jun 5, 2023
View reviewed changes
Copy link
Contributor

@ruanhang1993 ruanhang1993 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@ruanhang1993 ruanhang1993 merged commit 7504217 into apache:master Jun 5, 2023
14 checks passed
fuyun2024 added a commit to fuyun2024/flink-cdc-connectors that referenced this pull request Jun 8, 2023
@fuyun2024
oracle test
55de70b 
commit 7ced72b6426e9723f91933ef0144fdda4b9c072d
Author: Hang Ruan <ruanhang1993@hotmail.com>
Date:   Thu Jun 8 17:01:39 2023 +0800

    [mysql-cdc] Supports MYSQL_TYPE_TYPED_ARRAY column type when parsing the table map event

    This closes apache#2001

commit a847655ed68c4d5874ced69006686362f23140e7
Author: Leonard Xu <leonard@apache.org>
Date:   Thu Jun 8 15:33:54 2023 +0800

    [mysql-cdc][hotfix] Optimize the error msg when binlog expire in source

commit 32f77ef0075dfd2b522df07809ab2aea8ae6de54
Author: Leonard Xu <leonard@apache.org>
Date:   Wed Jun 7 17:49:41 2023 +0800

    [mysql-cdc] Optimize the error msg when binlog expire or server id conflict
     This closes apache#2010.

commit da5cbf54e6b219956088bb99cec44d93f1011ff7
Author: He Wang <wanghechn@qq.com>
Date:   Thu Jun 8 13:50:24 2023 +0800

    [oceanbase] add jdbc options and support oracle mode (apache#1854)

commit 0e221382601e352215091d9cd1cb59fa2175063b
Author: Leonard Xu <leonard@apache.org>
Date:   Wed Jun 7 17:34:57 2023 +0800

    [mysql-cdc] Add handler for catching async exceptions in snapshot reading executor
    This closes apache#2016.

commit 15e5e4d34dc05edcbdcfe15ff906213352a2cd02
Author: Hang Ruan <ruanhang1993@hotmail.com>
Date:   Wed Jun 7 17:26:59 2023 +0800

    [hotfix][mysql-cdc] Use session timezone instead of local time zone as the default server time zone

    This closes apache#2015.

commit 13235e2f2dca73320988a430b765f1fafa8e8f26
Author: Hang Ruan <ruanhang1993@hotmail.com>
Date:   Wed Jun 7 17:22:07 2023 +0800

    [mysql] Fix NullPointerException when database name or table name contains dot

    This closes apache#2006.

commit 6aa0fc4de353473cb7cda844a432e3dd5ddf4fea
Author: Hang Ruan <ruanhang1993@hotmail.com>
Date:   Thu Jan 12 02:18:52 2023 +0000

    [mysql] Fix NullPointerException caused by mysql ignores the capitalization when splitting chunks

commit e9fed740f55f7f1e675e5fe83f7e04efb5e3f866
Author: ehui <374488688@qq.com>
Date:   Tue Jun 6 13:06:03 2023 +0800

    [sqlserver] Fix old change data that will be captured when the latest mode starts (apache#2176)

commit d2ffa0902038e9aec0e2b92fa4e82a066d1b123f
Author: Tyrantlucifer <tyrantlucifer@apache.org>
Date:   Mon Jun 5 23:15:33 2023 +0800

    [core] support print configuration options for connectors (apache#2099)

commit a00abf88178feb6a2c699129e1f6aefb005960c7
Author: He Wang <wanghechn@qq.com>
Date:   Mon Jun 5 16:40:54 2023 +0800

    [oceanbase] support libobcdc 4.x and fix restore timestamp config (apache#2161)

commit 061602d
Author: ehui <374488688@qq.com>
Date:   Mon Jun 5 23:49:15 2023 +0800

    oralce

commit 8dc46cc
Author: ehui <374488688@qq.com>
Date:   Mon Jun 5 00:18:50 2023 +0800

    [oracle]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ruanhang1993 ruanhang1993 ruanhang1993 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[core] Format print connector config to log.
2 participants
@TyrantLucifer @ruanhang1993