Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-29502][SQL] typed interval expression should fail for invalid format #26151

Closed
wants to merge 3 commits into from

Conversation

cloud-fan
Copy link
Contributor

What changes were proposed in this pull request?

This is a followup of #25241 .

The typed interval expression should fail for invalid format.

Why are the changes needed?

Te be consistent with the typed timestamp/date expression

Does this PR introduce any user-facing change?

Yes. But this feature is not released yet.

How was this patch tested?

updated test

@cloud-fan
Copy link
Contributor Author

cc @wangyum @dongjoon-hyun

@@ -435,8 +435,7 @@ class ExpressionParserSuite extends AnalysisTest {
val intervalLiteral = Literal(CalendarInterval.fromString("interval 3 month 1 hour"))
assertEqual("InterVal 'interval 3 month 1 hour'", intervalLiteral)
assertEqual("INTERVAL '3 month 1 hour'", intervalLiteral)
assertEqual("Interval 'interval 3 monthsss 1 hoursss'",
Literal(null, CalendarIntervalType))
intercept("Interval 'interval 3 monthsss 1 hoursss'")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can refer to the timestamp/date tests just a few lines above.

@SparkQA
Copy link

SparkQA commented Oct 17, 2019

Test build #112226 has finished for PR 26151 at commit ee86886.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

val ex = new ParseException(e.getMessage, ctx)
ex.setStackTrace(e.getStackTrace)
throw ex
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message is still inconsistent. Invalid interval vs Cannot parse the DATE value:

scala> spark.sql("select Interval '10000 secondss'").show(false)
org.apache.spark.sql.catalyst.parser.ParseException:
Invalid interval: 10000 secondss(line 1, pos 7)
scala> spark.sql("select date '10000 secondss'").show(false)
org.apache.spark.sql.catalyst.parser.ParseException:
Cannot parse the DATE value: 10000 secondss(line 1, pos 7)

== SQL ==
select date '10000 secondss'
-------^^^

@SparkQA
Copy link

SparkQA commented Oct 18, 2019

Test build #112254 has finished for PR 26151 at commit 5a375eb.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@wangyum
Copy link
Member

wangyum commented Oct 18, 2019

retest this please

@SparkQA
Copy link

SparkQA commented Oct 18, 2019

Test build #112258 has finished for PR 26151 at commit 5a375eb.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@@ -435,8 +435,7 @@ class ExpressionParserSuite extends AnalysisTest {
val intervalLiteral = Literal(CalendarInterval.fromString("interval 3 month 1 hour"))
assertEqual("InterVal 'interval 3 month 1 hour'", intervalLiteral)
assertEqual("INTERVAL '3 month 1 hour'", intervalLiteral)
assertEqual("Interval 'interval 3 monthsss 1 hoursss'",
Literal(null, CalendarIntervalType))
intercept("Interval 'interval 3 monthsss 1 hoursss'")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you check the error message?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test suite defines the intercept method and uses it in many places. I don't want to break the rule here. It's definitely better to update the intercept method and take a error message parameter. This should be done in a followup.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the method has messages arg

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

damn, the date and timestamp tests don't specify message, and I thought it doesn't have the message arg... Let me update it now.

@SparkQA
Copy link

SparkQA commented Oct 18, 2019

Test build #112283 has finished for PR 26151 at commit b27be9f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@wangyum wangyum closed this in 2437878 Oct 18, 2019
@wangyum
Copy link
Member

wangyum commented Oct 18, 2019

Merged to master.

@dongjoon-hyun
Copy link
Member

Late LGTM. Thank you all!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
6 participants