Skip to content

Commit

Permalink
[SPARK-31980][SQL] Function sequence() fails if start and end of rang…
Browse files Browse the repository at this point in the history
…e are equal dates

### What changes were proposed in this pull request?
1. Add judge equal as bigger condition in `org.apache.spark.sql.catalyst.expressions.Sequence.TemporalSequenceImpl#eval`
2. Unit test for interval `day`, `month`, `year`

### Why are the changes needed?
Bug exists when sequence input get same equal start and end dates, which will occur `while loop` forever

### Does this PR introduce _any_ user-facing change?
Yes,
Before this PR, people will get a `java.lang.ArrayIndexOutOfBoundsException`, when eval as below:
`sql("select sequence(cast('2011-03-01' as date), cast('2011-03-01' as date), interval 1 year)").show(false)
`

### How was this patch tested?
Unit test.

Closes #28819 from TJX2014/master-SPARK-31980.

Authored-by: TJX2014 <xiaoxingstack@gmail.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
  • Loading branch information
TJX2014 authored and dongjoon-hyun committed Jun 20, 2020
1 parent 7b86838 commit 177a380
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -2628,7 +2628,7 @@ object Sequence {
val maxEstimatedArrayLength =
getSequenceLength(startMicros, stopMicros, intervalStepInMicros)

val stepSign = if (stopMicros > startMicros) +1 else -1
val stepSign = if (stopMicros >= startMicros) +1 else -1
val exclusiveItem = stopMicros + stepSign
val arr = new Array[T](maxEstimatedArrayLength)
var t = startMicros
Expand Down Expand Up @@ -2690,7 +2690,7 @@ object Sequence {
|
| $sequenceLengthCode
|
| final int $stepSign = $stopMicros > $startMicros ? +1 : -1;
| final int $stepSign = $stopMicros >= $startMicros ? +1 : -1;
| final long $exclusiveItem = $stopMicros + $stepSign;
|
| $arr = new $elemType[$arrLength];
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1836,4 +1836,22 @@ class CollectionExpressionsSuite extends SparkFunSuite with ExpressionEvalHelper
checkEvaluation(ArrayIntersect(empty, oneNull), Seq.empty)
checkEvaluation(ArrayIntersect(oneNull, empty), Seq.empty)
}

test("SPARK-31980: Start and end equal in month range") {
checkEvaluation(new Sequence(
Literal(Date.valueOf("2018-01-01")),
Literal(Date.valueOf("2018-01-01")),
Literal(stringToInterval("interval 1 day"))),
Seq(Date.valueOf("2018-01-01")))
checkEvaluation(new Sequence(
Literal(Date.valueOf("2018-01-01")),
Literal(Date.valueOf("2018-01-01")),
Literal(stringToInterval("interval 1 month"))),
Seq(Date.valueOf("2018-01-01")))
checkEvaluation(new Sequence(
Literal(Date.valueOf("2018-01-01")),
Literal(Date.valueOf("2018-01-01")),
Literal(stringToInterval("interval 1 year"))),
Seq(Date.valueOf("2018-01-01")))
}
}

0 comments on commit 177a380

Please sign in to comment.