Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-36119][SQL] Add new SQL function to_timestamp_ltz #33318

Closed
wants to merge 1 commit into from

Conversation

gengliangwang
Copy link
Member

What changes were proposed in this pull request?

Add new SQL function to_timestamp_ltz
syntax:

to_timestamp_ltz(timestamp_str_column[, fmt])
to_timestamp_ltz(timestamp_column)
to_timestamp_ltz(date_column)

Why are the changes needed?

As the result of to_timestamp become consistent with the SQL configuration spark.sql.timestmapType and there is already a SQL function to_timestmap_ntz, we need new function to_timestamp_ltz to construct timestamp with local time zone values.

Does this PR introduce any user-facing change?

Yes, a new function for constructing timestamp with local time zone values

How was this patch tested?

Unit test

@gengliangwang
Copy link
Member Author

This is supposed to be the last for *_ltz functions in my plan. We will have to_timestmap_ltz, make_timestamp_ltz, and type constructor timestmap_ltz.

@SparkQA
Copy link

SparkQA commented Jul 13, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45480/

@gengliangwang
Copy link
Member Author

@MaxGekk @HyukjinKwon Thanks for the review.
Merging to master/3.2

gengliangwang added a commit that referenced this pull request Jul 13, 2021
### What changes were proposed in this pull request?

Add new SQL function `to_timestamp_ltz`
syntax:
```
to_timestamp_ltz(timestamp_str_column[, fmt])
to_timestamp_ltz(timestamp_column)
to_timestamp_ltz(date_column)
```

### Why are the changes needed?

As the result of to_timestamp become consistent with the SQL configuration spark.sql.timestmapType and there is already a SQL function to_timestmap_ntz, we need new function to_timestamp_ltz to construct timestamp with local time zone values.

### Does this PR introduce _any_ user-facing change?

Yes, a new function for constructing timestamp with local time zone values

### How was this patch tested?

Unit test

Closes #33318 from gengliangwang/to_timestamp_ltz.

Authored-by: Gengliang Wang <gengliang@apache.org>
Signed-off-by: Gengliang Wang <gengliang@apache.org>
(cherry picked from commit 01ddaf3)
Signed-off-by: Gengliang Wang <gengliang@apache.org>
@SparkQA
Copy link

SparkQA commented Jul 13, 2021

Test build #140966 has finished for PR 33318 at commit 8428398.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • case class ParseToTimestampLTZ(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
5 participants