You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I tried to use spark-rapids to read data from iceberg, but failed with below exception no matter whether spark.rapids.sql.format.iceberg.enabled is true or flase or not.
24/05/17 10:50:40 ERROR GpuOverrideUtil: Encountered an exception applying GPU overrides java.lang.ClassCastException: org.apache.iceberg.BaseFileScanTask cannot be cast to org.apache.iceberg.CombinedScanTaskjava.lang.ClassCastException: org.apache.iceberg.BaseFileScanTask cannot be cast to org.apache.iceberg.CombinedScanTask at com.nvidia.spark.rapids.iceberg.spark.source.GpuSparkBatchQueryScan.isMetadataScan(GpuSparkBatchQueryScan.java:92) at com.nvidia.spark.rapids.iceberg.IcebergProviderImpl$$anon$1.tagSelfForGpu(IcebergProviderImpl.scala:51) at com.nvidia.spark.rapids.RapidsMeta.tagForGpu(RapidsMeta.scala:318) at com.nvidia.spark.rapids.RapidsMeta.$anonfun$tagForGpu$1(RapidsMeta.scala:292) at com.nvidia.spark.rapids.RapidsMeta.$anonfun$tagForGpu$1$adapted(RapidsMeta.scala:292)
cala> spark.table("local.db.demo").show()
24/05/1710:55:46ERRORGpuOverrideUtil:Encountered an exception applying GPU overrides java.lang.ClassCastException: org.apache.iceberg.BaseFileScanTask cannot be cast to org.apache.iceberg.CombinedScanTask
java.lang.ClassCastException: org.apache.iceberg.BaseFileScanTask cannot be cast to org.apache.iceberg.CombinedScanTask
at com.nvidia.spark.rapids.iceberg.spark.source.GpuSparkBatchQueryScan.isMetadataScan(GpuSparkBatchQueryScan.java:92)
at com.nvidia.spark.rapids.iceberg.IcebergProviderImpl$$anon$1.tagSelfForGpu(IcebergProviderImpl.scala:51)
at com.nvidia.spark.rapids.RapidsMeta.tagForGpu(RapidsMeta.scala:318)
at com.nvidia.spark.rapids.RapidsMeta.$anonfun$tagForGpu$1(RapidsMeta.scala:292)
at com.nvidia.spark.rapids.RapidsMeta.$anonfun$tagForGpu$1$adapted(RapidsMeta.scala:292)
at scala.collection.immutable.List.foreach(List.scala:431)
at com.nvidia.spark.rapids.RapidsMeta.tagForGpu(RapidsMeta.scala:292)
The text was updated successfully, but these errors were encountered:
Bug Desc
I tried to use spark-rapids to read data from iceberg, but failed with below exception no matter whether
spark.rapids.sql.format.iceberg.enabled
is true or flase or not.How to repro
prepare data for iceberg
and execute the below command
run with spark-rapids
and execute
The text was updated successfully, but these errors were encountered: