Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add timestamptz #3660

Merged
merged 1 commit into from Sep 30, 2022
Merged

add timestamptz #3660

merged 1 commit into from Sep 30, 2022

Conversation

waitingkuo
Copy link
Contributor

Which issue does this PR close?

Closes #3659

Rationale for this change

Fix the timezone as UTC for now before we figure out how to deal with non-utc timezone

casting between timestamp and timestamptz has no ambiguity under this situation

select from_unixtime(0)::timestamptz, arrow_typeof(from_unixtime(0)::timestamptz);
+------------------------+-------------------------------------+
| fromunixtime(Int64(0)) | arrowtypeof(fromunixtime(Int64(0))) |
+------------------------+-------------------------------------+
| 1970-01-01 00:00:00    | Timestamp(Nanosecond, Some("UTC"))  |
+------------------------+-------------------------------------+
1 row in set. Query took 0.002 seconds.

What changes are included in this PR?

Are there any user-facing changes?

@github-actions github-actions bot added core Core datafusion crate sql labels Sep 29, 2022
@waitingkuo
Copy link
Contributor Author

@avantgardnerio @alamb @liukun4515
this is based on our discussion #3148 (comment)

  1. Add timestamp but not timestamp with timezone
  2. Hard code the server "local" time zone to always be UTC, such that show time zone always returns UTC and set time zone 'UTC' is the only set command that will not produce an error
  3. Add timestamp with timezone, but it will always be UTC due to the above
  4. Add support for set time zone 'UTC+8'
  5. Add support for set timezone 'US/Mountain

1 and 2 were done. this pr is to enable 3

@waitingkuo
Copy link
Contributor Author

note that there're some functions don't work with timestamptz due to mismatch function signatures
e.g.

select extract(hour from now());
Plan("Coercion from [Utf8, Timestamp(Nanosecond, Some(\"UTC\"))] to the signature OneOf([Exact([Utf8, Date32]), Exact([Utf8, Date64]), Exact([Utf8, Timestamp(Second, None)]), Exact([Utf8, Timestamp(Microsecond, None)]), Exact([Utf8, Timestamp(Millisecond, None)]), Exact([Utf8, Timestamp(Nanosecond, None)])]) failed.")

i'd like to add these in separate pr

@codecov-commenter
Copy link

Codecov Report

Merging #3660 (6c5e7ff) into master (a9f7cac) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #3660   +/-   ##
=======================================
  Coverage   86.01%   86.02%           
=======================================
  Files         300      300           
  Lines       56583    56595   +12     
=======================================
+ Hits        48672    48684   +12     
  Misses       7911     7911           
Impacted Files Coverage Δ
datafusion/core/tests/sql/timestamp.rs 99.65% <100.00%> (+<0.01%) ⬆️
datafusion/sql/src/planner.rs 81.32% <100.00%> (+0.02%) ⬆️
datafusion/core/src/physical_plan/metrics/value.rs 87.06% <0.00%> (-0.50%) ⬇️
datafusion/common/src/scalar.rs 85.25% <0.00%> (-0.07%) ⬇️
datafusion/expr/src/logical_plan/plan.rs 77.66% <0.00%> (+0.32%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM -- thank you @waitingkuo

@alamb alamb merged commit bfc5ff0 into apache:master Sep 30, 2022
@ursabot
Copy link

ursabot commented Sep 30, 2022

Benchmark runs are scheduled for baseline = 65a5c6b and contender = bfc5ff0. bfc5ff0 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core datafusion crate sql
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add TIMESTAMPTZ
4 participants