-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make PageProcessor preserve input block laziness #1075
Conversation
fa91b33
to
f49bddb
Compare
ScanFilterAndProject can have different source node ID and plan node ID.
038b4f4
to
2cd3415
Compare
Benchmarks (no perf regression for non-lazy pages).
|
Some queries (significant CPU reduction for TopN queries with filter): No work processor pipelines:
with work processor pipelines:
|
part of: #49 |
2cd3415
to
becb49f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly just suggestions, and requests for more explanations (and code comments in one case).
I'm curious why we are only applying lazyness to some cases, how those were choose, and if we should expose it to more cases?
presto-main/src/main/java/io/prestosql/operator/project/InputPageProjection.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/io/prestosql/operator/project/DictionaryAwarePageProjection.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/io/prestosql/operator/project/PageProcessor.java
Show resolved
Hide resolved
presto-main/src/main/java/io/prestosql/operator/project/PageProcessor.java
Show resolved
Hide resolved
becb49f
to
c97a15b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ac
presto-main/src/main/java/io/prestosql/operator/project/PageProcessor.java
Show resolved
Hide resolved
c97a15b
to
3278c62
Compare
3278c62
to
ba52e16
Compare
Followup:
WorkProcessorOperator
(@Praveen2112 ?)FilterAndProjectOperator
toWorkProcessorOperator
This way we get (node-local) dynamic filtering for semi joins and
IN (subquery), NOT IN (subquery), x >= ALL (subquery), etc
expressions