Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIVE-28417: Bump Log4j2 to 2.23.1 to facilitate compilation of GraalVM Native Image #5375

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

linghengqian
Copy link
Member

@linghengqian linghengqian commented Jul 31, 2024

What changes were proposed in this pull request?

  • Bump Log4j2 to 2.23.1 to facilitate compilation of GraalVM Native Image.

Why are the changes needed?

  • This PR is aimed at ensuring that HiveServer2 JDBC Driver is compatible with future versions of LOG4J as much as possible. I created a unit test at https://github.com/linghengqian/log4j2-v2231-graalvm-native-image-test to prove that the HiveServer2 JDBC Driver using apache/logging-log4j2:2.23.1 can be used under the GraalVM Native Image compiled by GraalVM CE For JDK 22.0.2.
sdk install java 8.0.422-tem
sdk install java 22.0.2-graalce
sdk use java 8.0.422-tem
sdk install maven
git clone git@github.com:linghengqian/hive.git -b log4j-bump
cd ./hive/
mvn clean install -DskipTests
mvn clean package -pl packaging -DskipTests -Pdocker
cd ../

git clone git@github.com:linghengqian/log4j2-v2231-graalvm-native-image-test.git
cd ./log4j2-v2231-graalvm-native-image-test/
sdk use java 22.0.2-graalce
./mvnw -PgenerateMetadata -DskipNativeTests -e -T1C clean test native:metadata-copy
./mvnw -PnativeTestInHive -T1C -e clean test
Moved `log4j-api` and `log4j-core` artifacts with classifier `tests` to `log4j-api-test` and `log4j-core-test` respectively. 
I took care of the [release notes](https://logging.apache.org/log4j/2.x/release-notes/2.20.0.html).

The difference in dependencies is caused by Maven:

for the log4j-core artifact with tests classifier Maven pulled the wrong dependencies (compile scope instead of test scope),
for the log4j-core-test artifact the real dependencies are no longer hidden: the artifact mostly contains JUnit 4 and JUnit 5 extensions, so it depends on all those libraries.

Does this PR introduce any user-facing change?

No.

Is the change a dependency upgrade?

How was this patch tested?

sdk install java 8.0.422-tem
sdk use java 8.0.422-tem
sdk install maven
mvn clean install -DskipTests
mvn test -Dtest=TestHplsqlLocal
mvn test -Dtest=TestHplsqlOffline

Copy link
Member Author

@linghengqian linghengqian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unexpected exception java.lang.RuntimeException: Process docker failed to run in 30 seconds
 at org.apache.hadoop.hive.ql.externalDB.AbstractExternalDB.runCmd(AbstractExternalDB.java:91)
 at org.apache.hadoop.hive.ql.externalDB.AbstractExternalDB.runCmdAndPrintStreams(AbstractExternalDB.java:106)

@linghengqian linghengqian changed the title [WIP]HIVE-28417: Bump Log4j2 to 2.23.0 to facilitate compilation of GraalVM Native Image [WIP]HIVE-28417: Bump Log4j2 to 2.23.1 to facilitate compilation of GraalVM Native Image Aug 5, 2024
Copy link

sonarcloud bot commented Aug 5, 2024

@linghengqian linghengqian changed the title [WIP]HIVE-28417: Bump Log4j2 to 2.23.1 to facilitate compilation of GraalVM Native Image HIVE-28417: Bump Log4j2 to 2.23.1 to facilitate compilation of GraalVM Native Image Aug 5, 2024
@linghengqian linghengqian marked this pull request as ready for review August 5, 2024 04:56
@@ -49,9 +44,6 @@ appender.DRFA.policies.time.modulate = true
appender.DRFA.strategy.type = DefaultRolloverStrategy
appender.DRFA.strategy.max = 30

# list of all loggers
loggers = HadoopIPC, HadoopSecurity, Hdfs, HdfsServer, HadoopMetrics2, Mortbay, Yarn, YarnServer, Tez, HadoopConf, Zookeeper, ServerCnxn, NIOServerCnxn, ClientCnxn, ClientCnxnSocket, ClientCnxnSocketNIO, DataNucleus, Datastore, JPOX, Operator, Serde2Lazy, ObjectStore, CalcitePlanner, AmazonAws, ApacheHttp, Thrift, Jetty, BlockStateChange, swo, CBORuleLogger
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@linghengqian, any reason for removing this?

Copy link
Member Author

@linghengqian linghengqian Aug 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prior to version 2.6, the properties configuration requires that you list the identifiers of the appenders, filters and loggers, in a comma separated list in properties with those names. Each of those components will then be expected to be defined in sets of properties that begin with component.<.identifier>.. The identifier does not have to match the name of the component being defined but must uniquely identify all the attributes and subcomponents that are part of the component. If the list of identifiers is not present the identifier must not contain a '.'. Each individual component MUST have a "type" attribute specified that identifies the component's Plugin type.
As of version 2.6, this list of identifiers is no longer required as names are inferred upon first usage, however if you wish to use more complex identifies you must still use the list. If the list is present it will be used.

@Aggarwal-Raghav
Copy link
Contributor

Had 1 question, added it as comment. Otherwise java and pom changes LGTM (+1 non-binding).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants