Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-37799: [C++] Compute: CommonTemporal support time32 and time64 casting #37949

Merged
merged 4 commits into from
Oct 2, 2023

Conversation

mapleFU
Copy link
Member

@mapleFU mapleFU commented Sep 29, 2023

Rationale for this change

The original problem in mentioned in #37799

  1. SECOND in Parquet would always cast to MILLS
  2. eq not support eq(time32[s], time32[ms)

This patch is as advice in #37799 (comment) . We tent to add time32 and time64 in CommonTemporal.

What changes are included in this PR?

Support time32 and time64 with different time unit in arrow::compute::internal::CommonTemporal.

Are these changes tested?

Yes

Are there any user-facing changes?

bugfix

@github-actions
Copy link

⚠️ GitHub issue #37799 has been automatically assigned in GitHub to PR creator.

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Sep 29, 2023
@mapleFU mapleFU marked this pull request as ready for review September 29, 2023 08:55
@mapleFU mapleFU requested a review from bkietz September 29, 2023 08:56
@mapleFU
Copy link
Member Author

mapleFU commented Sep 29, 2023

@bkietz I've do a basic impl, would you mind take a look?

@github-actions github-actions bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Sep 29, 2023
Copy link
Member

@bkietz bkietz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this! Just a few comments

cpp/src/arrow/compute/kernels/codegen_internal.cc Outdated Show resolved Hide resolved
cpp/src/arrow/compute/kernels/codegen_internal.cc Outdated Show resolved Hide resolved
cpp/src/arrow/dataset/file_parquet_test.cc Outdated Show resolved Hide resolved
cpp/src/arrow/dataset/file_parquet_test.cc Outdated Show resolved Hide resolved
cpp/src/arrow/dataset/file_parquet_test.cc Outdated Show resolved Hide resolved
@github-actions github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Sep 29, 2023
@mapleFU
Copy link
Member Author

mapleFU commented Sep 29, 2023

@bkietz I've resolve the comments now

@github-actions github-actions bot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels Oct 2, 2023
@bkietz bkietz merged commit 22df70a into apache:main Oct 2, 2023
33 of 37 checks passed
@bkietz bkietz removed the awaiting merge Awaiting merge label Oct 2, 2023
@conbench-apache-arrow
Copy link

After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit 22df70a.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about possible false positives for unstable benchmarks that are known to sometimes produce them.

@mapleFU mapleFU deleted the time32-time64-cast branch October 3, 2023 16:41
JerAguilon pushed a commit to JerAguilon/arrow that referenced this pull request Oct 23, 2023
…64 casting (apache#37949)

### Rationale for this change

The original problem in mentioned in apache#37799

1. `SECOND` in Parquet would always cast to `MILLS`
2. `eq` not support `eq(time32[s], time32[ms)`

This patch is as advice in apache#37799 (comment) . We tent to add time32 and time64 in `CommonTemporal`.

### What changes are included in this PR?

Support time32 and time64 with different time unit in `arrow::compute::internal::CommonTemporal`.

### Are these changes tested?

Yes

### Are there any user-facing changes?

bugfix

* Closes: apache#37799

Authored-by: mwish <maplewish117@gmail.com>
Signed-off-by: Benjamin Kietzman <bengilgit@gmail.com>
JerAguilon pushed a commit to JerAguilon/arrow that referenced this pull request Oct 23, 2023
…64 casting (apache#37949)

### Rationale for this change

The original problem in mentioned in apache#37799

1. `SECOND` in Parquet would always cast to `MILLS`
2. `eq` not support `eq(time32[s], time32[ms)`

This patch is as advice in apache#37799 (comment) . We tent to add time32 and time64 in `CommonTemporal`.

### What changes are included in this PR?

Support time32 and time64 with different time unit in `arrow::compute::internal::CommonTemporal`.

### Are these changes tested?

Yes

### Are there any user-facing changes?

bugfix

* Closes: apache#37799

Authored-by: mwish <maplewish117@gmail.com>
Signed-off-by: Benjamin Kietzman <bengilgit@gmail.com>
loicalleyne pushed a commit to loicalleyne/arrow that referenced this pull request Nov 13, 2023
…64 casting (apache#37949)

### Rationale for this change

The original problem in mentioned in apache#37799

1. `SECOND` in Parquet would always cast to `MILLS`
2. `eq` not support `eq(time32[s], time32[ms)`

This patch is as advice in apache#37799 (comment) . We tent to add time32 and time64 in `CommonTemporal`.

### What changes are included in this PR?

Support time32 and time64 with different time unit in `arrow::compute::internal::CommonTemporal`.

### Are these changes tested?

Yes

### Are there any user-facing changes?

bugfix

* Closes: apache#37799

Authored-by: mwish <maplewish117@gmail.com>
Signed-off-by: Benjamin Kietzman <bengilgit@gmail.com>
dgreiss pushed a commit to dgreiss/arrow that referenced this pull request Feb 19, 2024
…64 casting (apache#37949)

### Rationale for this change

The original problem in mentioned in apache#37799

1. `SECOND` in Parquet would always cast to `MILLS`
2. `eq` not support `eq(time32[s], time32[ms)`

This patch is as advice in apache#37799 (comment) . We tent to add time32 and time64 in `CommonTemporal`.

### What changes are included in this PR?

Support time32 and time64 with different time unit in `arrow::compute::internal::CommonTemporal`.

### Are these changes tested?

Yes

### Are there any user-facing changes?

bugfix

* Closes: apache#37799

Authored-by: mwish <maplewish117@gmail.com>
Signed-off-by: Benjamin Kietzman <bengilgit@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[C++][Parquet] Dataset cannot filter timestamp with time_unit SECOND
2 participants