You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ShowQueriesRewrite calls getLayouts() with an alwaysTrue() constraint. This fetches all partition names for the table. It then rewrites to a query that queries an internal table which then does a second getLayouts() call with alwaysTrue(). In neither case are the query predicates passed in.
The first call is done to validate that exactly one layout exists and that the table has partition columns. We could keep the partition values, but the only feasible way to do that today would be to write to a VALUES node and that might not work well for a huge number of partitions.
MetadataQueryOptimizer has a similarly inefficient call to getLayouts() with alwaysTrue().
I think the API is the wrong shape for this. We could fix it by adding a separate ConnectorMetadata method to return the discrete predicates from a layout.
Additionally, we need to pass in a proper Constraint object, though this will require extracting the predicate logic from AddExchanges. Another potential issue is that the predicate function is extremely expensive with many object allocations and duplicated work on every invocation.
The text was updated successfully, but these errors were encountered:
ShowQueriesRewrite
callsgetLayouts()
with analwaysTrue()
constraint. This fetches all partition names for the table. It then rewrites to a query that queries an internal table which then does a secondgetLayouts()
call withalwaysTrue()
. In neither case are the query predicates passed in.The first call is done to validate that exactly one layout exists and that the table has partition columns. We could keep the partition values, but the only feasible way to do that today would be to write to a
VALUES
node and that might not work well for a huge number of partitions.MetadataQueryOptimizer
has a similarly inefficient call togetLayouts()
withalwaysTrue()
.I think the API is the wrong shape for this. We could fix it by adding a separate
ConnectorMetadata
method to return the discrete predicates from a layout.Additionally, we need to pass in a proper
Constraint
object, though this will require extracting the predicate logic fromAddExchanges
. Another potential issue is that the predicate function is extremely expensive with many object allocations and duplicated work on every invocation.The text was updated successfully, but these errors were encountered: