Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-41121: [C++] Fix: left anti join filter empty rows. #41122

Merged
merged 3 commits into from
Apr 15, 2024

Conversation

Light-City
Copy link
Contributor

@Light-City Light-City commented Apr 10, 2024

Rationale for this change

Since the left anti filter implementation is based on the left semi filter, and an assertion error occurs when the left semi filter rows are empty, this problem should be fixed.

What changes are included in this PR?

swiss_join.cc and hash_join_node_test.cc

Are these changes tested?

Yes

Are there any user-facing changes?

No

Copy link

⚠️ GitHub issue #41121 has been automatically assigned in GitHub to PR creator.

@Light-City
Copy link
Contributor Author

I don’t know why lint ci hangs without any prompt?

@ZhangHuiGui
Copy link
Collaborator

I don’t know why lint ci hangs without any prompt?

Seems you need to format your codes:

 TEST(HashJoin, FilterEmptyRows) {
   BatchesWithSchema input_left;
-  input_left.batches = {ExecBatchFromJSON({int32(), utf8(), int32()}, R"([[2, "Jarry", 28]])")};
+  input_left.batches = {
+      ExecBatchFromJSON({int32(), utf8(), int32()}, R"([[2, "Jarry", 28]])")};

@Light-City
Copy link
Contributor Author

I don’t know why lint ci hangs without any prompt?

Seems you need to format your codes:

 TEST(HashJoin, FilterEmptyRows) {
   BatchesWithSchema input_left;
-  input_left.batches = {ExecBatchFromJSON({int32(), utf8(), int32()}, R"([[2, "Jarry", 28]])")};
+  input_left.batches = {
+      ExecBatchFromJSON({int32(), utf8(), int32()}, R"([[2, "Jarry", 28]])")};

thanks, I changed.

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Apr 10, 2024
@Light-City
Copy link
Contributor Author

test_table.py failed .The test failure has nothing to do with my PR, it seems to be asof join

Copy link
Collaborator

@zanmato1984 zanmato1984 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great catch, thank you!

One minor suggestion though.

cpp/src/arrow/acero/swiss_join.cc Outdated Show resolved Hide resolved
@zanmato1984
Copy link
Collaborator

test_table.py failed .The test failure has nothing to do with my PR, it seems to be asof join

Yeah, seems unrelated to this PR. Could be the same as #41149.

Copy link
Collaborator

@zanmato1984 zanmato1984 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@zanmato1984
Copy link
Collaborator

We should wait for some committer to approve and merge. Thank you again for the fix!

cc @westonpace

@Light-City
Copy link
Contributor Author

We should wait for some committer to approve and merge. Thank you again for the fix!

cc @westonpace

Okay, thank you

@mapleFU
Copy link
Member

mapleFU commented Apr 11, 2024

@zanmato1984 should this better in 16.0?

@mapleFU mapleFU requested a review from bkietz April 11, 2024 16:14
@zanmato1984
Copy link
Collaborator

@zanmato1984 should this better in 16.0?

Yes, it would be nice to have it in 16.0. The condition to trigger this bug is not very rare: left-semi join with no rows matched by key meanwhile filter referring left table columns only.

cc @raulcd . If we have a chance, please consider including this PR in 16.0. Thank you.

@mapleFU mapleFU added the Priority: Blocker Marks a blocker for the release label Apr 12, 2024
@mapleFU
Copy link
Member

mapleFU commented Apr 12, 2024

I've change this patch to blocker

@pitrou pitrou removed the Priority: Blocker Marks a blocker for the release label Apr 15, 2024
@pitrou
Copy link
Member

pitrou commented Apr 15, 2024

While this would indeed be nice to have, I've removed the "Blocker" label since this doesn't seem to be a regression.

@pitrou
Copy link
Member

pitrou commented Apr 15, 2024

Rebased on git main for CI.

@pitrou
Copy link
Member

pitrou commented Apr 15, 2024

@github-actions crossbow submit -g cpp

Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems ok on the face of it, but I can't vouch for the correctness.

Copy link

Revision: 5d59164

Submitted crossbow builds: ursacomputing/crossbow @ actions-63062f6aac

Task Status
test-alpine-linux-cpp GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind GitHub Actions
test-cuda-cpp GitHub Actions
test-debian-12-cpp-amd64 GitHub Actions
test-debian-12-cpp-i386 GitHub Actions
test-fedora-39-cpp GitHub Actions
test-ubuntu-20.04-cpp GitHub Actions
test-ubuntu-20.04-cpp-bundled GitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizer GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-emscripten GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions

@pitrou pitrou merged commit 924fbec into apache:main Apr 15, 2024
35 of 36 checks passed
@pitrou pitrou removed the awaiting committer review Awaiting committer review label Apr 15, 2024
raulcd pushed a commit that referenced this pull request Apr 15, 2024
### Rationale for this change

Since the left anti filter implementation is based on the left semi filter, and an assertion error occurs when the left semi filter rows are empty, this problem should be fixed.

### What changes are included in this PR?

swiss_join.cc and hash_join_node_test.cc

### Are these changes tested?
Yes

### Are there any user-facing changes?
No

* GitHub Issue: #41121

Lead-authored-by: light-city <455954986@qq.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Copy link

After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 924fbec.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 7 possible false positives for unstable benchmarks that are known to sometimes produce them.

tolleybot pushed a commit to tmct/arrow that referenced this pull request May 2, 2024
…41122)

### Rationale for this change

Since the left anti filter implementation is based on the left semi filter, and an assertion error occurs when the left semi filter rows are empty, this problem should be fixed.

### What changes are included in this PR?

swiss_join.cc and hash_join_node_test.cc

### Are these changes tested?
Yes

### Are there any user-facing changes?
No

* GitHub Issue: apache#41121

Lead-authored-by: light-city <455954986@qq.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Signed-off-by: Antoine Pitrou <antoine@python.org>
vibhatha pushed a commit to vibhatha/arrow that referenced this pull request May 25, 2024
…41122)

### Rationale for this change

Since the left anti filter implementation is based on the left semi filter, and an assertion error occurs when the left semi filter rows are empty, this problem should be fixed.

### What changes are included in this PR?

swiss_join.cc and hash_join_node_test.cc

### Are these changes tested?
Yes

### Are there any user-facing changes?
No

* GitHub Issue: apache#41121

Lead-authored-by: light-city <455954986@qq.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants