Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of SHOW PARTITIONS #6753

Closed
electrum opened this issue Dec 2, 2016 · 1 comment
Closed

Improve performance of SHOW PARTITIONS #6753

electrum opened this issue Dec 2, 2016 · 1 comment
Labels

Comments

@electrum
Copy link
Contributor

electrum commented Dec 2, 2016

ShowQueriesRewrite calls getLayouts() with an alwaysTrue() constraint. This fetches all partition names for the table. It then rewrites to a query that queries an internal table which then does a second getLayouts() call with alwaysTrue(). In neither case are the query predicates passed in.

The first call is done to validate that exactly one layout exists and that the table has partition columns. We could keep the partition values, but the only feasible way to do that today would be to write to a VALUES node and that might not work well for a huge number of partitions.

MetadataQueryOptimizer has a similarly inefficient call to getLayouts() with alwaysTrue().

I think the API is the wrong shape for this. We could fix it by adding a separate ConnectorMetadata method to return the discrete predicates from a layout.

Additionally, we need to pass in a proper Constraint object, though this will require extracting the predicate logic from AddExchanges. Another potential issue is that the predicate function is extremely expensive with many object allocations and duplicated work on every invocation.

@stale
Copy link

stale bot commented Apr 3, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@stale stale bot added the stale label Apr 3, 2019
@stale stale bot closed this as completed Apr 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant