Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-32133][SQL] Forbid time field steps for date start/end in Sequ…
…ence ### What changes were proposed in this pull request? 1.Add time field steps check for date start/end in Sequence at `org.apache.spark.sql.catalyst.expressions.Sequence.TemporalSequenceImpl` 2.Add a UT:`SPARK-32133: Sequence step must be a day interval if start and end values are dates` at `org.apache.spark.sql.catalyst.expressions.CollectionExpressionsSuite` ### Why are the changes needed? **Sequence time field steps for date start/end looks strange in spark as follows:** ``` scala> sql("select explode(sequence(cast('2011-03-01' as date), cast('2011-03-02' as date), interval 1 hour))").head(3) res0: Array[org.apache.spark.sql.Row] = _Array([2011-03-01], [2011-03-01], [2011-03-01])_ **<- strange result.** scala> sql("select explode(sequence(cast('2011-03-01' as date), cast('2011-03-02' as date), interval 1 day))").head(3) res1: Array[org.apache.spark.sql.Row] = Array([2011-03-01], [2011-03-02]) ``` **While this behavior in Prosto make sense:** ``` presto> select sequence(date('2011-03-01'),date('2011-03-02'),interval '1' hour); Query 20200624_122744_00002_pehix failed: sequence step must be a day interval if start and end values are dates presto> select sequence(date('2011-03-01'),date('2011-03-02'),interval '1' day); _col0 [2011-03-01, 2011-03-02] ``` ### Does this PR introduce _any_ user-facing change? Yes, after this patch, users will get informed `sequence step must be a day interval if start and end values are dates` when use time field steps for date start/end in Sequence. ### How was this patch tested? Unit test. Closes #28926 from TJX2014/master-SPARK-31982-sequence-cross-dst-follow-presto. Authored-by: TJX2014 <xiaoxingstack@gmail.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
- Loading branch information