Skip to content

Parallelize hjoin with BulkCompare #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
arukuka merged 8 commits into from
Oct 3, 2022
Merged

Parallelize hjoin with BulkCompare #43

arukuka merged 8 commits into from
Oct 3, 2022

Conversation

@mdonaka
Copy link

@mdonaka mdonaka commented Sep 27, 2022 •
edited
Loading

Summary

Accelerate intersectionSortedValueIndex

Purpose

Accelerate hjoin

Contents

  • Fix ID of hjoin test table
  • Parallelize intersectionSortedValueIndex with BulkCompare
    • Step1: Split into blocks using parallel binary search
    • Step2: Parallel scanning per block

Testing Methods Performed

  • CI
  • make test t=LibClient p=benchmark
  • make test t=ComputationContainer p=benchmark

Benchmark

2-patry

(second) N=10 N=100 N=1,000 N=10,000 N=100,000
PR ver 4 15 24 115 973
old ver 6 15 111 1027 9989

@mdonaka mdonaka requested review from arukuka and come2ry September 27, 2022 11:15
@mdonaka mdonaka force-pushed the feature/nakata/hjoin_bulkcompare branch from dded257 to d2c436c Compare September 28, 2022 11:43
@mdonaka mdonaka marked this pull request as ready for review September 28, 2022 12:09
arukuka
arukuka requested changes Sep 29, 2022
View reviewed changes
Copy link
Member

@arukuka arukuka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your great work!
I commented on the part that I was curious about, so please check it.

@arukuka
Copy link
Member

arukuka commented Sep 30, 2022

I will rebase latest main branch

@arukuka arukuka force-pushed the feature/nakata/hjoin_bulkcompare branch from 122eb39 to 59dabc3 Compare September 30, 2022 09:13
@mdonaka mdonaka force-pushed the feature/nakata/hjoin_bulkcompare branch from 59dabc3 to bc47e3f Compare September 30, 2022 09:28
@arukuka arukuka merged commit 4361dca into main Oct 3, 2022
@arukuka arukuka deleted the feature/nakata/hjoin_bulkcompare branch October 3, 2022 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@arukuka arukuka arukuka approved these changes

@come2ry come2ry Awaiting requested review from come2ry

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@mdonaka @arukuka