[SUPPORT] Unable to query Partitioned COW Hudi tables with metadata enabled using Trino-Hudi Connector #7583
Labels
priority:major
degraded perf; unable to move forward; potential bugs
query-engine
trino, presto, athena, impala, etc
Describe the problem you faced
Original issue: trinodb/trino#15368
The issue was resolved by placing some dependencies in the classpath. Interestingly, those dependencies are already included in the trino-hudi-bundle. This particular issues tracks any gap in packaging.
To Reproduce
Steps to reproduce the behavior:
hudi.metadata-enabled=true
.Trino Hudi Connector Properties:
Hudi Properties set while writing:
General information of table:
Total rows = 1,213,959,199
Total Partitions = 2400+
Total file objects = 120,000
Total Size on S3 = 12~13 GB
The table was upgraded from 0.9.0 to 0.10.1
Coordinator Relevant Logs:
Expected behavior
They query should work out-of-the-box without having to place jars in classpath.
Environment Description
Hudi version : 0.10.1
Spark version : 2.4
Trino version : 400
Hadoop version :
Storage (HDFS/S3/GCS..) :
Running on Docker? (yes/no) : no
Additional context
Add any other context about the problem here.
Stacktrace
Full stacktrace in
Partitioned_COW_Hudi_Coordinator_logs.log
The text was updated successfully, but these errors were encountered: