-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize remaining filter evaluation #7433
Conversation
✅ Deploy Preview for meta-velox canceled.
|
This pull request was exported from Phabricator. Differential Revision: D51039406 |
Summary: Currently when we have a remaining filter separated by ANDs and same field is multi-referenced from different clauses, we eagerly materialize that field. If there is a very selective condition in the AND clauses, we lose the opportunity to avoid reading unneeded rows using that condition on the mutli-referenced field. Since AND always reduces the selectivity, there is no need to eagerly evaluate the multi-referenced fields. It gives 2.6x performance gain in some queries by evaluating the field lazily. Differential Revision: D51039406
d354626
to
925c1c7
Compare
This pull request was exported from Phabricator. Differential Revision: D51039406 |
Summary: Currently when we have a remaining filter separated by ANDs and same field is multi-referenced from different clauses, we eagerly materialize that field. If there is a very selective condition in the AND clauses, we lose the opportunity to avoid reading unneeded rows using that condition on the mutli-referenced field. Since AND always reduces the selectivity, there is no need to eagerly evaluate the multi-referenced fields. It gives 2.6x performance gain in some queries by evaluating the field lazily. Differential Revision: D51039406
925c1c7
to
b2009c1
Compare
This pull request was exported from Phabricator. Differential Revision: D51039406 |
Summary: Currently when we have a remaining filter separated by ANDs and same field is multi-referenced from different clauses, we eagerly materialize that field. If there is a very selective condition in the AND clauses, we lose the opportunity to avoid reading unneeded rows using that condition on the mutli-referenced field. Since AND always reduces the selectivity, there is no need to eagerly evaluate the multi-referenced fields. It gives 2.6x performance gain in some queries by evaluating the field lazily. Reviewed By: laithsakka Differential Revision: D51039406
b2009c1
to
7b16729
Compare
This pull request was exported from Phabricator. Differential Revision: D51039406 |
Summary: Currently when we have a remaining filter separated by ANDs and same field is multi-referenced from different clauses, we eagerly materialize that field. If there is a very selective condition in the AND clauses, we lose the opportunity to avoid reading unneeded rows using that condition on the mutli-referenced field. Since AND always reduces the selectivity, there is no need to eagerly evaluate the multi-referenced fields. It gives 2.6x performance gain in some queries by evaluating the field lazily. Reviewed By: laithsakka Differential Revision: D51039406
7b16729
to
2a2ed7e
Compare
This pull request was exported from Phabricator. Differential Revision: D51039406 |
Summary: Currently when we have a remaining filter separated by ANDs and same field is multi-referenced from different clauses, we eagerly materialize that field. If there is a very selective condition in the AND clauses, we lose the opportunity to avoid reading unneeded rows using that condition on the mutli-referenced field. Since AND always reduces the selectivity, there is no need to eagerly evaluate the multi-referenced fields. It gives 2.6x performance gain in some queries by evaluating the field lazily. Reviewed By: laithsakka Differential Revision: D51039406
2a2ed7e
to
1c0742a
Compare
This pull request was exported from Phabricator. Differential Revision: D51039406 |
This pull request has been merged in fc8a69f. |
Conbench analyzed the 1 benchmark run on commit There were no benchmark performance regressions. 🎉 The full Conbench report has more details. |
Summary:
Currently when we have a remaining filter separated by ANDs and same field is multi-referenced from different clauses, we eagerly materialize that field. If there is a very selective condition in the AND clauses, we lose the opportunity to avoid reading unneeded rows using that condition on the mutli-referenced field.
Since AND always reduces the selectivity, there is no need to eagerly evaluate the multi-referenced fields. It gives 2.6x performance gain in some queries by evaluating the field lazily.
Differential Revision: D51039406