Information schema fixups #2488
Conversation
| log.debug( | ||
| "Building information schema table (queryId=%s; catalog=%s; table=%s; prefixes=%s, limit=%s)", | ||
| session.getQueryId(), | ||
| informationSchemaTableHandle.getCatalogName(), |
There was a problem hiding this comment.
can we simply log informationSchemaTableHandle?
| @@ -13,6 +13,7 @@ | |||
| */ | |||
There was a problem hiding this comment.
It was removed previously
| @@ -111,6 +116,7 @@ public InformationSchemaPageSource( | |||
| else { | |||
| checkArgument(prefixes.equals(defaultPrefixes(catalogName)), "Catalog-wise tables have prefixes other than the default one"); | |||
| } | |||
|
|
|||
| @@ -101,6 +101,11 @@ public InformationSchemaPageSource( | |||
| table = tableHandle.getTable(); | |||
| prefixIterator = Suppliers.memoize(() -> { | |||
| Set<QualifiedTablePrefix> prefixes = tableHandle.getPrefixes(); | |||
| if (!tableHandle.getLimit().isPresent()) { | |||
| // no limit is used, therefore it doesn't make sense to split information schema query into smaller ones | |||
There was a problem hiding this comment.
Listing schemas explicitly here also helps return results to client sooner. And with respect to Hive Connector, it doesn't really increase the number of metadata calls to HMS. Having said that, I think it's fine to make the change as it may help in case of other connectors.
There was a problem hiding this comment.
And with respect to Hive Connector, it doesn't really increase the number of metadata calls to HMS. Having said that, I think it's fine to make the change as it may help in case of other connectors.
JDBC connectors can take advantage of single query. For example, they can query their own information_schema, so instead of X requests you get one
When limit is not used, then listing schemas explicitly causes additional connector metadata queries. However, since all metadata rows need to be read, it doesn't make sense to do so.
sopel39
force-pushed
the
ks/metadata_fixups
branch
from
3d6c905
to
3958333
Compare
No description provided.