java.lang.ClassCastException: java.sql.Timestamp cannot be cast to java.time.Instant from DESC DETAIL

[study-day]

spark-sql> desc  delta.`/tmp/delta-table-test/aa12` ;
21/08/17 15:06:30 WARN metastore.ObjectStore: Failed to get database delta, returning NoSuchObjectException
shortField      string
intField        string
bigintField     string
doubleField     string
stringField     string
timestampField  string
decimalField    string
dateField       string
charField       string
floatField      string
complexData     string

# Partitioning
Not partitioned
Time taken: 0.042 seconds, Fetched 14 row(s)
spark-sql> select * from   delta.`/tmp/delta-table-test/aa12` ;
21/08/17 15:06:33 WARN metastore.ObjectStore: Failed to get database delta, returning NoSuchObjectException
1       10      1100    48.4    spark   2015/4/23 12:01:01      1.23    2015/4/23       aaa     2.5     'foo'#'bar'#'world'
5       17      1140    43.4    spark   2015/7/27 12:01:02      3.45    2015/7/27       bbb     2.5     'foo'#'bar'#'world'
1       11      1100    44.4    flink   2015/5/23 12:01:03      23.23   2015/5/23       ccc     2.5     'foo'#'bar'#'world'
1       10      1150    43.4    spark   2015/7/24 12:01:04      254.12  2015/7/24       ddd     2.5     'foo'#'bar'#'world'
1       10      1100    47.4    spark   2015/7/23 12:01:05      876.14  2015/7/23       eeee    3.5     'foo'#'bar'#'world'
3       14      1160    43.4    hive    2015/7/26 12:01:06      3454.32 2015/7/26       ff      2.5     'foo'#'bar'#'world'
2       10      1100    43.4    impala  2015/7/23 12:01:07      456.98  2015/7/23       ggg     2.5     'foo'#'bar'#'world'
1       10      1100    43.4    spark   2015/5/23 12:01:08      32.53   2015/5/23       hhh     2.5     'foo'#'bar'#'world'
4       16      1130    42.4    impala  2015/7/23 12:01:09      67.23   2015/7/23       iii     2.5     'foo'#'bar'#'world'
1       10      1100    43.4    spark   2015/7/23 12:01:10      832.23  2015/7/23       jjj     2.5     'foo'#'bar'#'world'
Time taken: 0.574 seconds, Fetched 10 row(s)
spark-sql> describe detail  '/tmp/delta-table-test/aa12' ;
21/08/17 15:06:38 ERROR thriftserver.SparkSQLDriver: Failed in [describe detail  '/tmp/delta-table-test/aa12' ]
java.lang.ClassCastException: java.sql.Timestamp cannot be cast to java.time.Instant
        at org.apache.spark.sql.catalyst.CatalystTypeConverters$InstantConverter$.toCatalystImpl(CatalystTypeConverters.scala:335)
        at org.apache.spark.sql.catalyst.CatalystTypeConverters$CatalystTypeConverter.toCatalyst(CatalystTypeConverters.scala:107)

[zsxwing]

This is caused by apache/spark#28729 . It will be fixed by https://issues.apache.org/jira/browse/SPARK-35204 in Spark 3.2.0.

Right now, you can run set spark.sql.datetime.java8API.enabled=false; before running the describe detail command as a workaround.

[study-day]

in the spark 3.1.2 it is default false 。so it is no effect.