Skip to content

Commit

Permalink
[SPARK-34743][SQL][TESTS] ExpressionEncoderSuite should use deepEqual…
Browse files Browse the repository at this point in the history
…s when we expect `array of array`

### What changes were proposed in this pull request?

This PR aims to make `ExpressionEncoderSuite` to use `deepEquals` instead of `equals` when `input` is `array of array`.

This comparison code itself was added by SPARK-11727 at Apache Spark 1.6.0.

### Why are the changes needed?

Currently, the interpreted mode fails for `array of array` because the following line is used.
```
Arrays.equals(b1.asInstanceOf[Array[AnyRef]], b2.asInstanceOf[Array[AnyRef]])
```

### Does this PR introduce _any_ user-facing change?

No. This is a test-only PR.

### How was this patch tested?

Pass the existing CIs.

Closes #31837 from dongjoon-hyun/SPARK-34743.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
dongjoon-hyun committed Mar 15, 2021
1 parent be888b2 commit 363a7f0
Showing 1 changed file with 3 additions and 6 deletions.
Expand Up @@ -167,10 +167,9 @@ class ExpressionEncoderSuite extends CodegenInterpretedPlanTest with AnalysisTes
encodeDecodeTest(Array.empty[Int], "empty array of int")
encodeDecodeTest(Array.empty[String], "empty array of string")

encodeDecodeTest(Array(Array(31, -123), null, Array(4, 67)), "array of array of int",
useFallback = true)
encodeDecodeTest(Array(Array(31, -123), null, Array(4, 67)), "array of array of int")
encodeDecodeTest(Array(Array("abc", "xyz"), Array[String](null), null, Array("1", null, "2")),
"array of array of string", useFallback = true)
"array of array of string")

encodeDecodeTest(Map(1 -> "a", 2 -> "b"), "map")
encodeDecodeTest(Map(1 -> "a", 2 -> null), "map with null")
Expand Down Expand Up @@ -595,10 +594,8 @@ class ExpressionEncoderSuite extends CodegenInterpretedPlanTest with AnalysisTes
val isCorrect = (input, convertedBack) match {
case (b1: Array[Byte], b2: Array[Byte]) => Arrays.equals(b1, b2)
case (b1: Array[Int], b2: Array[Int]) => Arrays.equals(b1, b2)
case (b1: Array[Array[_]], b2: Array[Array[_]]) =>
Arrays.deepEquals(b1.asInstanceOf[Array[AnyRef]], b2.asInstanceOf[Array[AnyRef]])
case (b1: Array[_], b2: Array[_]) =>
Arrays.equals(b1.asInstanceOf[Array[AnyRef]], b2.asInstanceOf[Array[AnyRef]])
Arrays.deepEquals(b1.asInstanceOf[Array[AnyRef]], b2.asInstanceOf[Array[AnyRef]])
case (left: Comparable[_], right: Comparable[_]) =>
left.asInstanceOf[Comparable[Any]].compareTo(right) == 0
case _ => input == convertedBack
Expand Down

0 comments on commit 363a7f0

Please sign in to comment.