Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Hadoop libraries as runtimeOnly dependeny in transportable-udfs-test-hive/generic/trino #154

Merged
merged 2 commits into from
Jun 8, 2023

Conversation

yiqiangin
Copy link
Contributor

@yiqiangin yiqiangin commented Jun 6, 2023

Code Changes
As consumer UDFs generated by Transport Plugin use HiveTester defined in the module of transportable-udfs-hive-test to execute the unit tests on UDF generated against Hive at run time and HiveTester depends on some classes from the libraries of hadoop-common and hadoop-mapreduce-client-core, two libraries are required at run time and they are added as runtimeOnly dependencies in the module of transportable-udfs-hive-test.
Also the test classes from transportable-udfs-test-generic and transportable-udfs-test-trino depends on hdfs library of Hadoop when the tests of UDF are executed at runtime. Therefore the runtimeOnly dependencies on two libraries are added into transportable-udfs-test-generic and transportable-udfs-test-trino.

Tests

cd transport
./gradlew clean build

cd transportable-udfs-examples
./gradlew clean build

Test UDF with a locally-built TransportPlugin with this code change. 

@yiqiangin yiqiangin requested a review from wmoustafa June 6, 2023 21:09
@weijiii
Copy link
Contributor

weijiii commented Jun 6, 2023

Should you remove the testImplementation for hadoop in transportable-udfs-example/transportable-udfs-example/build.gradle then run the tests? Is it needed for the testing on Trino?
LG

wmoustafa
wmoustafa previously approved these changes Jun 6, 2023
@yiqiangin yiqiangin changed the title Add Hadoop libraries as runtimeOnly dependeny in transportable-udfs-hive-test Add Hadoop libraries as runtimeOnly dependeny in transportable-udfs-test-hive/generic/trino Jun 7, 2023
@yiqiangin yiqiangin merged commit 1adf4f4 into master Jun 8, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants