Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-30062][Connectors/HBase] ADDENDUM: Bundle the exact same dependencies as it was in core #4

Merged
merged 1 commit into from
Dec 12, 2022

Conversation

ferenc-csaky
Copy link
Contributor

Without this, we not package the HBase specific classes into the flink-sql-connector-hbase-2.2 artifact.

Copy link
Contributor

@zentol zentol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still a bit different than what was bundled in 1.16.0.
1.16.0:

[INFO] Including org.apache.flink:flink-connector-hbase-2.2:jar:1.16.0 in the shaded jar.
[INFO] Including org.apache.flink:flink-connector-hbase-base:jar:1.16.0 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-client:jar:2.2.3 in the shaded jar.
[INFO] Including org.apache.hbase.thirdparty:hbase-shaded-protobuf:jar:2.2.1 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-common:jar:2.2.3 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-protocol-shaded:jar:2.2.3 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-protocol:jar:2.2.3 in the shaded jar.
[INFO] Including commons-codec:commons-codec:jar:1.15 in the shaded jar.
[INFO] Including org.apache.commons:commons-lang3:jar:3.3.2 in the shaded jar.
[INFO] Including org.apache.hbase.thirdparty:hbase-shaded-miscellaneous:jar:2.2.1 in the shaded jar.
[INFO] Including com.google.protobuf:protobuf-java:jar:2.5.0 in the shaded jar.
[INFO] Including org.apache.hbase.thirdparty:hbase-shaded-netty:jar:2.2.1 in the shaded jar.
[INFO] Including org.apache.zookeeper:zookeeper:jar:3.4.14 in the shaded jar.
[INFO] Including org.apache.htrace:htrace-core4:jar:4.2.0-incubating in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-core:jar:3.2.6 in the shaded jar.
[INFO] Including org.apache.commons:commons-crypto:jar:1.0.0 in the shaded jar.
[INFO] Including io.netty:netty-all:jar:4.1.70.Final in the shaded jar.

PR:

[INFO] Including org.apache.flink:flink-connector-hbase-2.2:jar:1.0-SNAPSHOT in the shaded jar.
[INFO] Including org.apache.flink:flink-connector-hbase-base:jar:1.0-SNAPSHOT in the shaded jar.
[INFO] Including org.apache.hbase:hbase-client:jar:2.2.3 in the shaded jar.
[INFO] Including org.apache.hbase.thirdparty:hbase-shaded-protobuf:jar:2.2.1 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-common:jar:2.2.3 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-protocol-shaded:jar:2.2.3 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-protocol:jar:2.2.3 in the shaded jar.
[INFO] Including commons-codec:commons-codec:jar:1.10 in the shaded jar.
[INFO] Including org.apache.hbase.thirdparty:hbase-shaded-miscellaneous:jar:2.2.1 in the shaded jar.
[INFO] Including com.google.protobuf:protobuf-java:jar:2.5.0 in the shaded jar.
[INFO] Including org.apache.hbase.thirdparty:hbase-shaded-netty:jar:2.2.1 in the shaded jar.
[INFO] Including org.apache.zookeeper:zookeeper:jar:3.4.14 in the shaded jar.
[INFO] Including org.apache.htrace:htrace-core4:jar:4.2.0-incubating in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-core:jar:3.2.6 in the shaded jar.
[INFO] Including org.apache.commons:commons-crypto:jar:1.0.0 in the shaded jar.

So netty-all is missing, as is commons-lang3.

Now netty I get; 2.2 uses hbase-shaded-netty (but then I'd like a dedicated commit removing an excess dependency; can be a hotfix 🤷).
But the commons-lang3 dependency does still seem to be there (but it's now being excluded for some reason).

flink-sql-connector-hbase-2.2/pom.xml Outdated Show resolved Hide resolved
flink-connector-hbase-2.2/pom.xml Outdated Show resolved Hide resolved
@zentol zentol self-assigned this Dec 8, 2022
@ferenc-csaky ferenc-csaky changed the title [FLINK-30062][Connectors/HBase] ADDENDUM: Fix hbase-client 2.2 dependency handling [FLINK-30062][Connectors/HBase] ADDENDUM: Bundle the exact same dependencies as it was in core Dec 10, 2022
@ferenc-csaky
Copy link
Contributor Author

So netty-all is missing, as is commons-lang3.

Now netty I get; 2.2 uses hbase-shaded-netty (but then I'd like a dedicated commit removing an excess dependency; can be a hotfix 🤷). But the commons-lang3 dependency does still seem to be there (but it's now being excluded for some reason).

commons-lang3 got lost during the dependency convergence work, but now I added back both. I also made sure to use the same commons-codec version everywhere, cause that was also messed up. Now everything should be good.

Copy link
Contributor

@zentol zentol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found another problem:
The 1.14 sql connector now bundles commons-logging which it didn't in 1.16.0.

flink-sql-connector-hbase-2.2/pom.xml Outdated Show resolved Hide resolved
@ferenc-csaky
Copy link
Contributor Author

ferenc-csaky commented Dec 12, 2022

I compared the JAR files and all of them has the exact same content now, except the meta inf.

@zentol
Copy link
Contributor

zentol commented Dec 12, 2022

I compared the JAR files and all of them has the exact same content now, except the meta inf.

Apparently you missed #4 (review) though.

Hold on did I mix things up?...

@zentol zentol merged commit d74e518 into apache:main Dec 12, 2022
@ferenc-csaky ferenc-csaky deleted the fix-hbase-client branch December 12, 2022 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants