[Spark-16791] [SQL] cast struct with timestamp field fails #14400
Conversation
|
@ueshin , you seem to be the original committer of this code. can you please have a look? |
| @@ -416,7 +416,7 @@ case class Cast(child: Expression, dataType: DataType) extends UnaryExpression w | |||
| } | |||
|
|
|||
| private[this] def cast(from: DataType, to: DataType): Any => Any = to match { | |||
| case dt if dt == child.dataType => identity[Any] | |||
| case dt if dt == from => identity[Any] | |||
There was a problem hiding this comment.
nit: remove extra whitespace before from.
|
Good catch! |
|
Good catch! LGTM except the style comment given by @ueshin . And please also use full word for variable name, e.g. |
…s from reviewers.
| @@ -727,6 +727,19 @@ class CastSuite extends SparkFunSuite with ExpressionEvalHelper { | |||
| } | |||
| } | |||
|
|
|||
| test("cast struct with a timestamp field") { | |||
| val originalSchema = StructType( | |||
There was a problem hiding this comment.
we usually use new StructType().add("name", TimestampType, nullable = false)
| @@ -727,6 +727,16 @@ class CastSuite extends SparkFunSuite with ExpressionEvalHelper { | |||
| } | |||
| } | |||
|
|
|||
| test("cast struct with a timestamp field") { | |||
| val originalSchema = new StructType().add( "tsField", TimestampType, nullable = false ) | |||
There was a problem hiding this comment.
please remove the space in add( xxx )
|
ok to test |
|
|
||
| val inp = Literal.create(InternalRow(0L), originalSchema) | ||
| val expected = InternalRow(0L) | ||
| checkEvaluation(cast(inp, targetSchema), expected ) |
There was a problem hiding this comment.
and remove the extra space here too
|
Test build #63052 has finished for PR 14400 at commit
|
|
@cloud-fan, any update on this? |
|
@cloud-fan, just for notice for the next patches: is there a way to run mvn in a mode that fails on style issues? it could save a lot of round-trips if it existed. (can be sbt just as well) |
|
we have an individual style check script: |
|
Test build #63076 has finished for PR 14400 at commit
|
## What changes were proposed in this pull request? a failing test case + fix to SPARK-16791 (https://issues.apache.org/jira/browse/SPARK-16791) ## How was this patch tested? added a failing test case to CastSuit, then fixed the Cast code and rerun the entire CastSuit Author: eyal farago <eyal farago> Author: Eyal Farago <eyal.farago@actimize.com> Closes #14400 from eyalfa/SPARK-16791_cast_struct_with_timestamp_field_fails. (cherry picked from commit 338a98d) Signed-off-by: Wenchen Fan <wenchen@databricks.com>
|
thanks, merging to master and 2.0! |
What changes were proposed in this pull request?
a failing test case + fix to SPARK-16791 (https://issues.apache.org/jira/browse/SPARK-16791)
How was this patch tested?
added a failing test case to CastSuit, then fixed the Cast code and rerun the entire CastSuit