Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement PassThroughTransformOperator to optimize select queries #6972

Closed
GSharayu opened this issue May 25, 2021 · 1 comment
Closed

Implement PassThroughTransformOperator to optimize select queries #6972

GSharayu opened this issue May 25, 2021 · 1 comment
Assignees

Comments

@GSharayu
Copy link
Contributor

For general select queries

e.g SELECT * FROM Foo ..... LIMIT N

e.g SELECT col1, col2 ..... FROM Foo .... LIMIT N

the columns are identifier expressions and not some transform functions. So, we should have a PassThroughTransformOperator that avoids the if conditional check repeatedly in TransformBlock.

If the columns selected in the query are identifier expressions then no need to make the check again and again for every block fetch of 10K records. Since during query planning time, we have the information available in TransformPlanNode if selected expressions are identifiers or not, we can make an optimization to have a specialized TransformOperator and TransformBlock that simply does a pass through and avoids the repeated if check at query runtime.

This is something that would have been done in pinot if there was run time query specific code generation. Since we currently don't have it, we can handwrite the specialized operator.

@siddharthteotia
Copy link
Contributor

Resolved by PR #6973

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants