Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare subquery pruning #2560

Merged
merged 27 commits into from
Jun 27, 2023
Merged

Prepare subquery pruning #2560

merged 27 commits into from
Jun 27, 2023

Conversation

dey4ss
Copy link
Member

@dey4ss dey4ss commented Apr 5, 2023

This PR prepares the codebase for subquery pruning. I.e., the chunk pruning code is moved to an own file. Also, we introduce a check that ensures acyclic operator task graphs. All in all, these changes are to be merged previously so we can focus on the actual dynamic subquery pruning when we open a dedicated PR for it. Besides, the usual Monk (style, fixing comments, auto to stick, const, superfluous includes, ...).

@dey4ss dey4ss added the FullCI Run all CI tests (slow, but required for merge) label Apr 5, 2023
@dey4ss dey4ss requested a review from Bouncner April 5, 2023 20:37
@dey4ss
Copy link
Member Author

dey4ss commented Apr 5, 2023

chunk_pruning_utils.cpp is just moved from chunk_pruning_rule.cpp with tiny adaptions (auto to stick). Additional includes and the currently unused

std::set<ChunkID> compute_chunk_exclude_list(const PredicatePruningChain& predicate_pruning_chain,
                                             const std::shared_ptr<StoredTableNode>& stored_table_node);

in chunk_pruning_utils will be used once the subquery pruning PR is opened.

Same goes for

std::shared_ptr<AbstractOperator> _translate_node_recursively(const std::shared_ptr<AbstractLQPNode>& node) const;

in lpq_translator (we will need this once we assign PredicateNodes to StoredTableNodes and have to set the TableScans to the GetTables when translating an LQP – this can only be done after each node is translated).

This is a cherry-pick of the new LoCs. Großes Indianerehrenwort 🤞

@dey4ss dey4ss mentioned this pull request Apr 12, 2023
16 tasks
src/lib/expression/lqp_subquery_expression.hpp Outdated Show resolved Hide resolved
src/lib/expression/pqp_subquery_expression.hpp Outdated Show resolved Hide resolved
src/lib/logical_query_plan/abstract_lqp_node.cpp Outdated Show resolved Hide resolved
src/lib/operators/abstract_operator.cpp Show resolved Hide resolved
src/lib/operators/pqp_utils.cpp Outdated Show resolved Hide resolved
src/lib/utils/chunk_pruning_utils.hpp Outdated Show resolved Hide resolved
src/test/lib/operators/get_table_test.cpp Show resolved Hide resolved
src/test/lib/operators/pqp_utils_test.cpp Show resolved Hide resolved
src/test/lib/scheduler/operator_task_test.cpp Outdated Show resolved Hide resolved
Copy link
Collaborator

@Bouncner Bouncner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One typo and one tiny issue remaining (comment). Apart from that, all good.

src/lib/utils/chunk_pruning_utils.hpp Outdated Show resolved Hide resolved
src/test/lib/operators/get_table_test.cpp Show resolved Hide resolved
@Bouncner
Copy link
Collaborator

Second time ASAN issues. I thought it's a glitch in the CI, but it might be an actual issue.
Do you know how to check for the error?

@dey4ss
Copy link
Member Author

dey4ss commented Jun 21, 2023

Second time ASAN issues. I thought it's a glitch in the CI, but it might be an actual issue.
Do you know how to check for the error?

Just ran the failing stage again on one of our servers (no problems there), might try it on the CI machine later.

Edit: Ran it on the CI server, no issue. Restarted the CI pipeline.

@dey4ss
Copy link
Member Author

dey4ss commented Jun 22, 2023

I'm really giving it up now. CI failed on another sanitizer test case that did not fail anywhere else (but seems reasonable and is fixed). At least the sanitizer tests should work now.

@dey4ss dey4ss merged commit cbe009f into master Jun 27, 2023
4 checks passed
@dey4ss dey4ss deleted the dey4ss/prepare_subquery_pruning branch June 27, 2023 14:08
nikriek pushed a commit that referenced this pull request Oct 28, 2023
This PR prepares the codebase for subquery pruning.
nikriek pushed a commit that referenced this pull request Nov 6, 2023
This PR prepares the codebase for subquery pruning.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FullCI Run all CI tests (slow, but required for merge)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants