Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-37110: [C++] Expression: SmallestTypeFor lost tz for Scalar #37135

Merged
merged 4 commits into from
Aug 14, 2023

Conversation

mapleFU
Copy link
Member

@mapleFU mapleFU commented Aug 12, 2023

Rationale for this change

This patch ( #15180 ) adds a SmallestTypeFor to handling expression type. However, it lost timezone when handling.

What changes are included in this PR?

Add timezone in SmallestTypeFor

Are these changes tested?

Currently not

Are there any user-facing changes?

Yeah it's a bugfix

@github-actions
Copy link

⚠️ GitHub issue #37110 has been automatically assigned in GitHub to PR creator.

@mapleFU mapleFU requested a review from bkietz August 12, 2023 05:49
@mapleFU
Copy link
Member Author

mapleFU commented Aug 12, 2023

@bkietz @pitrou Would you mind take a look?

Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch, @mapleFU ! LGTM but just a suggestion.

cmp(field_ref("ts_s"),
literal(std::make_shared<TimestampScalar>(0, TimeUnit::NANO, "UTC"))),
cmp(field_ref("ts_s"),
literal(std::make_shared<TimestampScalar>(0, TimeUnit::SECOND, "UTC"))));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are at it and the original PR didn't add many tests, can we also test non-zero values? For example (untested):

Suggested change
literal(std::make_shared<TimestampScalar>(0, TimeUnit::SECOND, "UTC"))));
literal(std::make_shared<TimestampScalar>(0, TimeUnit::SECOND, "UTC"))));
ExpectBindsTo(
cmp(field_ref("ts_s"),
literal(std::make_shared<TimestampScalar>(123000, TimeUnit::NANO, "UTC"))),
cmp(field_ref("ts_s"),
literal(std::make_shared<TimestampScalar>(123, TimeUnit::MICRO, "UTC"))));

Copy link
Member Author

@mapleFU mapleFU Aug 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added, I also found that we need a UTC type here.

field("ts_s_utc", timestamp(TimeUnit::SECOND, "UTC"))

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Aug 14, 2023
@pitrou pitrou merged commit baf3eb9 into apache:main Aug 14, 2023
35 of 36 checks passed
@pitrou pitrou removed the awaiting committer review Awaiting committer review label Aug 14, 2023
@mapleFU mapleFU deleted the expr/add-tz branch August 14, 2023 17:51
@conbench-apache-arrow
Copy link

After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit baf3eb9.

There were 2 benchmark results indicating a performance regression:

The full Conbench report has more details. It also includes information about possible false positives for unstable benchmarks that are known to sometimes produce them.

@brokenjacobs
Copy link

Great find!

loicalleyne pushed a commit to loicalleyne/arrow that referenced this pull request Nov 13, 2023
…pache#37135)

### Rationale for this change

This patch ( apache#15180 ) adds a `SmallestTypeFor` to handling expression type. However, it lost timezone when handling.

### What changes are included in this PR?

Add `timezone` in `SmallestTypeFor`

### Are these changes tested?

Currently not

### Are there any user-facing changes?

Yeah it's a bugfix

* Closes: apache#37110

Authored-by: mwish <maplewish117@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[C++][Python] Cannot filter dataset with a timestamp (with timezone) column
3 participants