Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-31710][SQL][FOLLOWUP] Replace CAST by TIMESTAMP_SECONDS in benchmarks #28843

Conversation

MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Jun 16, 2020

What changes were proposed in this pull request?

Replace CAST(... AS TIMESTAMP by TIMESTAMP_SECONDS in the following benchmarks:

  • ExtractBenchmark
  • DateTimeBenchmark
  • FilterPushdownBenchmark
  • InExpressionBenchmark

Why are the changes needed?

The benchmarks fail w/o the changes:

[info] Running benchmark: datetime +/- interval
[info]   Running case: date + interval(m)
[error] Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve 'CAST(`id` AS TIMESTAMP)' due to data type mismatch: cannot cast bigint to timestamp,you can enable the casting by setting spark.sql.legacy.allowCastNumericToTimestamp to true,but we strongly recommend using function TIMESTAMP_SECONDS/TIMESTAMP_MILLIS/TIMESTAMP_MICROS instead.; line 1 pos 5;
[error] 'Project [(cast(cast(id#0L as timestamp) as date) + 1 months) AS (CAST(CAST(id AS TIMESTAMP) AS DATE) + INTERVAL '1 months')#2]
[error] +- Range (0, 10000000, step=1, splits=Some(1))

Does this PR introduce any user-facing change?

No

How was this patch tested?

By running the affected benchmarks.

@cloud-fan
Copy link
Contributor

thanks for fixing it! It's benchmark only so we don't need to wait for jenkins.

merging to master!

@cloud-fan cloud-fan closed this in 3643565 Jun 16, 2020
@SparkQA
Copy link

SparkQA commented Jun 16, 2020

Test build #124125 has finished for PR 28843 at commit 97b5f97.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MaxGekk MaxGekk deleted the GuoPhilipse-31710-fix-compatibility-followup branch December 11, 2020 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
3 participants