Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is this the expected way to parallelize? #849

Closed
ahadnagy opened this issue Jan 6, 2022 · 0 comments
Closed

Is this the expected way to parallelize? #849

ahadnagy opened this issue Jan 6, 2022 · 0 comments

Comments

@ahadnagy
Copy link

ahadnagy commented Jan 6, 2022

Hi everyone,

I'm trying to get a few TPC-H-like queries running by building the query plans manually.

For example, TPC-H 13:

auto plan = PlanBuilder()
             .values({tables.get("customer")})
             .hashJoin(
                 {0},
                 {0},
                 PlanBuilder()
                     .values({tables.get("orders")})
                     .planNode(),
                 R"(not REGEXP_LIKE(o_comment, '^special.*?requests'))",
                 {0, 1, 2, 3})
             .singleAggregation({0}, {"count(o_orderkey)"})
             .project({"c_custkey", "a0"}, {"c_custkey", "c_count"})
             .singleAggregation({1}, {"count(1)"})
             .project({"c_count", "a0"}, {"c_count", "custdist"})
             .orderBy({0, 1}, {kDescNullsLast, kDescNullsLast}, false)
             .limit(0, 1, false)
             .planNode();
auto result = readSingleValue(plan);

I tried to parallelize the execution using the CursorParameters:

params.maxDrivers = 16;
params.numResultDrivers = 1;

This appears to be working, although it's on a single core ~95% of the time. params.numResultDrivers = 1; seem fishy to me, but without that the execution hangs.

Is this the correct approach, or am I missing something?
Thanks in advance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant