[Bug] cdc 3.0.1和flink 1.16.2 集成存在依赖冲入问题

[WuChongYong]

Search before asking

• I searched in the issues and found nothing similar.

Flink version

1.16.2

Flink CDC version

3.0.1

Database and its version

mysql 5.7.26

Minimal reproduce step

guava 依赖冲突，cdc的guava依赖是

org.apache.flink
flink-shaded-guava
30.1.1-jre-16.2

flink 1.16.2的guava依赖是

org.apache.flink
flink-shaded-guava
31.1-jre-17.0

运行官网样例的api会报类找不到：
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/shaded/guava31/com/google/common/util/concurrent/ThreadFactoryBuilder

如果剔除cdc的guava包，引入31的guava包，也会出现其他类的找不到
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/shaded/guava30/com/google/common/collect/Lists

What did you expect to see?

希望3.0.1之后cdc版本能准确和flink 1.14以上的版本互相兼容

What did you see instead?

但目前3.0.1之后cdc版本能准确和flink 1.16.2 的版本不兼容

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[loserwang1024]

@WuChongYong
Unlike flink-table-common、flink-connector-base 、flink-core whose maven scope is provided , flink-shaded-guava and flink-shaded-force-shading will be included in the final jar package, causing dependency conflict.

Now we can see why dependency conflict occurs:

• Since Flink 1.18, version of guava is upgrade from 30 to 31.

• Since CDC 3.0, version of guava is also upgrade from 30 to 31

• So CDC 3.0 is compatible with Flink 1.18, CDC 2.x is is compatible with Flink 1.13-1.17. CDC 2.x and Flink 1.18 dependency conflict, CDC 3.0 and Flink 1.17 dependency conflict.

Thus, both dependencies should be shaded relocations in pom.xml.
By the way, please use English so that foreign developers can also refer to this issue.

@leonardBang , @ruanhang1993 , @PatrickRen CC, WDYT?

[YyItRoad]

和我的一样我用的是flink 1.17.2 和 flink-connector-mongodb-cdc 3.0.1也出现这个问题， 然后我将flink-connector-mongodb-cdc降到2.4.2才正常

[loserwang1024]

@WuChongYong , @YyItRoad , hi, Please give me your code so that I can reproduce it.

[WuChongYong]

you can use the flink 1.18.0 version

…

---- 回复的原邮件 ---- | 发件人 | Hongshun ***@***.***> | | 日期 | 2024年03月06日 20:25 | | 收件人 | ***@***.***> | | 抄送至 | ***@***.***>***@***.***> | | 主题 | Re: [apache/flink-cdc] [Bug] cdc 3.0.1和flink 1.16.2 集成存在依赖冲入问题 (Issue #3071) | @WuChongYong , @YyItRoad , hi, Please give me your code so that I can reproduce it. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[PatrickRen]

Considering collaboration with developers around the world, please re-create your issue in English on Apache Jira under project Flink with component tag Flink CDC. Thank you!