Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -33,30 +33,30 @@
* (for views) at read time.
* <p>
* Catalogs build the metadata via {@link TableInfo.Builder} (for data-source tables) or
* {@link ViewInfo.Builder} (for views). A {@code MetadataOnlyTable} wrapping a
* {@link ViewInfo.Builder} (for views). A {@code MetadataTable} wrapping a
* {@link TableInfo} can be returned from {@link TableCatalog#loadTable(Identifier)} for a
* data-source table; a {@code MetadataOnlyTable} wrapping a {@link ViewInfo} can be returned
* from {@link RelationCatalog#loadRelation(Identifier)} as the single-RPC perf opt-in for a view.
* Downstream consumers distinguish the two by checking
* data-source table; a {@code MetadataTable} wrapping a {@link ViewInfo} can be returned
* from {@link TableViewCatalog#loadTableOrView(Identifier)} as the single-RPC perf opt-in
* for a view. Downstream consumers distinguish the two by checking
* {@code getTableInfo() instanceof ViewInfo}.
*
* @since 4.2.0
*/
@Evolving
public class MetadataOnlyTable implements Table {
public class MetadataTable implements Table {
private final TableInfo info;
private final String name;

/**
* @param info metadata for the table or view. Pass a {@link ViewInfo} for a view.
* @param name human-readable name for this table, used by places that read {@link #name()}
* (e.g. the {@code Name} row of {@code DESCRIBE TABLE EXTENDED}). Catalogs
* returning a {@code MetadataOnlyTable} from {@link TableCatalog#loadTable} or
* {@link RelationCatalog#loadRelation} should typically pass
* returning a {@code MetadataTable} from {@link TableCatalog#loadTable} or
* {@link TableViewCatalog#loadTableOrView} should typically pass
* {@code ident.toString()}, matching the quoted multi-part form used elsewhere
* for v2 identifiers.
*/
public MetadataOnlyTable(TableInfo info, String name) {
public MetadataTable(TableInfo info, String name) {
this.info = Objects.requireNonNull(info, "info should not be null");
this.name = Objects.requireNonNull(name, "name should not be null");
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
* Catalog API for connectors that expose tables.
* <p>
* Connectors that expose <i>only</i> tables implement this interface. Connectors that expose
* both tables and views must implement {@link RelationCatalog} (which extends both this
* both tables and views must implement {@link TableViewCatalog} (which extends both this
* interface and {@link ViewCatalog} and adds the cross-cutting contract for the combined
* case); the methods on this interface remain table-only -- they do not interact with views.
* <p>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,15 @@
* Catalog API for connectors that expose both tables and views in a single shared identifier
* namespace.
* <p>
* Connectors that expose <i>both</i> tables and views must implement {@code RelationCatalog};
* Connectors that expose <i>both</i> tables and views must implement {@code TableViewCatalog};
* implementing {@link TableCatalog} and {@link ViewCatalog} directly without
* {@code RelationCatalog} is rejected at catalog initialization. Connectors that expose only
* {@code TableViewCatalog} is rejected at catalog initialization. Connectors that expose only
* tables implement just {@link TableCatalog}; connectors that expose only views implement just
* {@link ViewCatalog}; this interface is not relevant to them.
*
* <h2>Two principles</h2>
*
* A {@code RelationCatalog} follows two rules that, taken together, define every cross-cutting
* A {@code TableViewCatalog} follows two rules that, taken together, define every cross-cutting
* subtlety:
* <ol>
* <li><b>Orthogonal interfaces.</b> Every {@link TableCatalog} method behaves as if views did
Expand Down Expand Up @@ -97,38 +97,38 @@
* <h2>Single-RPC perf entry points</h2>
*
* The orthogonal {@link TableCatalog} and {@link ViewCatalog} answer two cross-cutting
* questions in two round trips each. {@code RelationCatalog} adds dedicated methods so a
* questions in two round trips each. {@code TableViewCatalog} adds dedicated methods so a
* catalog can answer both in one round trip:
* <ul>
* <li>{@link #loadRelation(Identifier)} -- the resolver's per-identifier read path. Returns
* a regular {@link Table} for a table, or a {@link MetadataOnlyTable} wrapping a
* <li>{@link #loadTableOrView(Identifier)} -- the resolver's per-identifier read path. Returns
* a regular {@link Table} for a table, or a {@link MetadataTable} wrapping a
* {@link ViewInfo} for a view. Saves the {@code loadTable} -> {@code loadView} fallback
* on a cold cache.</li>
* <li>{@link #listRelationSummaries(String[])} -- a unified listing of tables and views with the
* kind preserved on each {@link TableSummary}. Default impl performs both
* <li>{@link #listTableAndViewSummaries(String[])} -- a unified listing of tables and views
* with the kind preserved on each {@link TableSummary}. Default impl performs both
* {@link TableCatalog#listTableSummaries} and {@link ViewCatalog#listViews}; override to
* fetch in one round trip.</li>
* </ul>
*
* @since 4.2.0
*/
@Evolving
public interface RelationCatalog extends TableCatalog, ViewCatalog {
public interface TableViewCatalog extends TableCatalog, ViewCatalog {

/**
* Load metadata for an identifier that may resolve to either a table or a view.
* <p>
* For a table, returns the table's {@link Table}. For a view, returns a
* {@link MetadataOnlyTable} wrapping a {@link ViewInfo}; callers discriminate via
* {@link MetadataTable} wrapping a {@link ViewInfo}; callers discriminate via
* {@code getTableInfo() instanceof ViewInfo}. This lets the resolver answer in a single RPC
* instead of falling back from {@link TableCatalog#loadTable} to {@link ViewCatalog#loadView}.
*
* @param ident the identifier
* @return a {@link Table} for tables, or a {@link MetadataOnlyTable} wrapping a
* @return a {@link Table} for tables, or a {@link MetadataTable} wrapping a
* {@link ViewInfo} for views
* @throws NoSuchTableException if neither a table nor a view exists at {@code ident}
*/
Table loadRelation(Identifier ident) throws NoSuchTableException;
Table loadTableOrView(Identifier ident) throws NoSuchTableException;

/**
* List the tables and views in a namespace, returned as {@link TableSummary} entries with
Expand All @@ -144,7 +144,7 @@ public interface RelationCatalog extends TableCatalog, ViewCatalog {
* @throws NoSuchTableException if a table listed by the underlying enumeration disappears
* before its summary can be assembled (default impl only)
*/
default TableSummary[] listRelationSummaries(String[] namespace)
default TableSummary[] listTableAndViewSummaries(String[] namespace)
throws NoSuchNamespaceException, NoSuchTableException {
TableSummary[] tableSummaries = listTableSummaries(namespace);
Identifier[] viewIdentifiers = listViews(namespace);
Expand All @@ -162,14 +162,14 @@ default TableSummary[] listRelationSummaries(String[] namespace)
/**
* {@inheritDoc}
* <p>
* The default implementation derives from {@link #loadRelation}: a {@link MetadataOnlyTable}
* The default implementation derives from {@link #loadTableOrView}: a {@link MetadataTable}
* wrapping a {@link ViewInfo} is rejected as not-a-table; anything else is returned. Override
* only if a tables-only path is materially cheaper than the unified one.
*/
@Override
default Table loadTable(Identifier ident) throws NoSuchTableException {
Table t = loadRelation(ident);
if (t instanceof MetadataOnlyTable mot && mot.getTableInfo() instanceof ViewInfo) {
Table t = loadTableOrView(ident);
if (t instanceof MetadataTable mt && mt.getTableInfo() instanceof ViewInfo) {
throw new NoSuchTableException(ident);
}
return t;
Expand All @@ -178,7 +178,7 @@ default Table loadTable(Identifier ident) throws NoSuchTableException {
/**
* {@inheritDoc}
* <p>
* The default implementation derives from {@link #loadRelation}: a {@link MetadataOnlyTable}
* The default implementation derives from {@link #loadTableOrView}: a {@link MetadataTable}
* wrapping a {@link ViewInfo} is unwrapped and returned; anything else (table or absent) is
* surfaced as {@link NoSuchViewException}. Override only if a views-only path is materially
* cheaper than the unified one.
Expand All @@ -187,11 +187,11 @@ default Table loadTable(Identifier ident) throws NoSuchTableException {
default ViewInfo loadView(Identifier ident) throws NoSuchViewException {
Table t;
try {
t = loadRelation(ident);
t = loadTableOrView(ident);
} catch (NoSuchTableException e) {
throw new NoSuchViewException(ident);
}
if (t instanceof MetadataOnlyTable mot && mot.getTableInfo() instanceof ViewInfo vi) {
if (t instanceof MetadataTable mt && mt.getTableInfo() instanceof ViewInfo vi) {
return vi;
}
throw new NoSuchViewException(ident);
Expand All @@ -200,14 +200,14 @@ default ViewInfo loadView(Identifier ident) throws NoSuchViewException {
/**
* {@inheritDoc}
* <p>
* The default implementation derives from {@link #loadRelation}: returns {@code true} only if
* The default implementation derives from {@link #loadTableOrView}: returns {@code true} only if
* the entry exists and is not a view. Override only if a cheaper existence-check path exists.
*/
@Override
default boolean tableExists(Identifier ident) {
try {
Table t = loadRelation(ident);
return !(t instanceof MetadataOnlyTable mot && mot.getTableInfo() instanceof ViewInfo);
Table t = loadTableOrView(ident);
return !(t instanceof MetadataTable mt && mt.getTableInfo() instanceof ViewInfo);
} catch (NoSuchTableException e) {
return false;
}
Expand All @@ -216,14 +216,14 @@ default boolean tableExists(Identifier ident) {
/**
* {@inheritDoc}
* <p>
* The default implementation derives from {@link #loadRelation}: returns {@code true} only if
* The default implementation derives from {@link #loadTableOrView}: returns {@code true} only if
* the entry exists and is a view. Override only if a cheaper existence-check path exists.
*/
@Override
default boolean viewExists(Identifier ident) {
try {
Table t = loadRelation(ident);
return t instanceof MetadataOnlyTable mot && mot.getTableInfo() instanceof ViewInfo;
Table t = loadTableOrView(ident);
return t instanceof MetadataTable mt && mt.getTableInfo() instanceof ViewInfo;
} catch (NoSuchTableException e) {
return false;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
* Catalog API for connectors that expose views.
* <p>
* Connectors that expose <i>only</i> views implement this interface. Connectors that expose
* both tables and views must implement {@link RelationCatalog} (which extends both this
* both tables and views must implement {@link TableViewCatalog} (which extends both this
* interface and {@link TableCatalog} and adds the cross-cutting contract for the combined
* case); the methods on this interface remain view-only -- they do not interact with tables.
* <p>
Expand Down Expand Up @@ -126,7 +126,7 @@ ViewInfo createView(Identifier ident, ViewInfo info)
* concurrent {@code CREATE VIEW} won the race in the
* default impl's gap between {@link #replaceView} and
* the fallback {@link #createView}, or, in a
* {@link RelationCatalog}, a table sits at {@code ident}
* {@link TableViewCatalog}, a table sits at {@code ident}
* @throws NoSuchNamespaceException if the identifier's namespace does not exist (optional)
*/
default ViewInfo createOrReplaceView(Identifier ident, ViewInfo info)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@
* query output column names. Schema and user TBLPROPERTIES are inherited from {@link TableInfo}
* via the typed builder.
* <p>
* {@code ViewInfo} extends {@link TableInfo} so that a {@link RelationCatalog} can opt into the
* single-RPC perf path by returning a {@link MetadataOnlyTable} wrapping a {@code ViewInfo}
* from {@link RelationCatalog#loadRelation} for a view identifier. Pure {@link ViewCatalog}
* {@code ViewInfo} extends {@link TableInfo} so that a {@link TableViewCatalog} can opt into the
* single-RPC perf path by returning a {@link MetadataTable} wrapping a {@code ViewInfo}
* from {@link TableViewCatalog#loadTableOrView} for a view identifier. Pure {@link ViewCatalog}
* implementations never see {@code TableInfo}; the typed setters on {@link Builder} cover
* everything they need to construct a {@code ViewInfo}.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1131,8 +1131,8 @@ class Analyzer(
* so surfacing a downstream "view not found" would hide the real reason.
*
* Lookup order against a non-session catalog:
* 1. If the catalog is a [[RelationCatalog]], [[RelationCatalog.loadRelation]] is called
* once. A returned [[MetadataOnlyTable]] wrapping a [[ViewInfo]] is interpreted as a
* 1. If the catalog is a [[TableViewCatalog]], [[TableViewCatalog.loadTableOrView]] is
* called once. A returned [[MetadataTable]] wrapping a [[ViewInfo]] is interpreted as a
* view; other results are tables.
* 2. Otherwise, [[TableCatalog.loadTable]] is tried (when implemented), then
* [[ViewCatalog.loadView]] as the fallback view-resolution path (when implemented).
Expand All @@ -1150,13 +1150,13 @@ class Analyzer(
throw QueryCompilationErrors.missingCatalogViewsAbilityError(catalog)
}
catalog match {
case mc: RelationCatalog =>
// Single-RPC perf path: loadRelation returns a Table for a table or a
// MetadataOnlyTable wrapping a ViewInfo for a view. NoSuchTable means
case mc: TableViewCatalog =>
// Single-RPC perf path: loadTableOrView returns a Table for a table or a
// MetadataTable wrapping a ViewInfo for a view. NoSuchTable means
// neither exists.
try {
Some(mc.loadRelation(ident) match {
case t: MetadataOnlyTable if t.getTableInfo.isInstanceOf[ViewInfo] =>
Some(mc.loadTableOrView(ident) match {
case t: MetadataTable if t.getTableInfo.isInstanceOf[ViewInfo] =>
ResolvedPersistentView(
catalog, ident, V1Table.toCatalogTable(catalog, ident, t))
case table =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,10 @@ import org.apache.spark.sql.connector.catalog.{
ChangelogInfo,
Identifier,
LookupCatalog,
MetadataOnlyTable,
RelationCatalog,
MetadataTable,
Table,
TableCatalog,
TableViewCatalog,
V1Table,
V2TableWithV1Fallback,
ViewCatalog,
Expand Down Expand Up @@ -262,8 +262,8 @@ class RelationResolution(
.orElse {
val writePrivileges = u.options.get(UnresolvedRelation.REQUIRED_WRITE_PRIVILEGES)
val finalOptions = u.clearWritePrivileges.options
// For a `RelationCatalog` with no time-travel / write privileges, the single-RPC
// `loadRelation` answers both "is there a table?" and "is there a view?" in one
// For a `TableViewCatalog` with no time-travel / write privileges, the single-RPC
// `loadTableOrView` answers both "is there a table?" and "is there a view?" in one
// call. Time-travel and write privileges apply to tables only, so for those the
// lookup falls through to the table-only `loadTable` path below; views are not
// reachable via the v2 fallback in those cases.
Expand All @@ -272,9 +272,10 @@ class RelationResolution(
// mixin): `CatalogV2Util.loadTable` would call `asTableCatalog` and throw
// MISSING_CATALOG_ABILITY.TABLES, masking the legitimate view-resolution path.
val tableOrView: Option[Table] = catalog match {
case mc: RelationCatalog if finalTimeTravelSpec.isEmpty && writePrivileges == null =>
case mc: TableViewCatalog
if finalTimeTravelSpec.isEmpty && writePrivileges == null =>
try {
Some(mc.loadRelation(ident))
Some(mc.loadTableOrView(ident))
} catch {
case _: NoSuchTableException => None
}
Expand All @@ -299,7 +300,7 @@ class RelationResolution(
catalog match {
case vc: ViewCatalog =>
try {
Some(new MetadataOnlyTable(vc.loadView(ident), ident.toString))
Some(new MetadataTable(vc.loadView(ident), ident.toString))
} catch {
case _: NoSuchViewException => None
}
Expand All @@ -313,7 +314,7 @@ class RelationResolution(
// `table` is `tableOrView` filtered to tables only -- used for cache lookup since
// we don't share-cache views.
val table: Option[Table] = tableOrView.filter {
case t: MetadataOnlyTable if t.getTableInfo.isInstanceOf[ViewInfo] => false
case t: MetadataTable if t.getTableInfo.isInstanceOf[ViewInfo] => false
case _ => true
}

Expand Down Expand Up @@ -426,10 +427,10 @@ class RelationResolution(
|| !v1Table.catalogTable.tracksPartitionsInCatalog =>
createDataSourceV1Scan(v1Table.v1Table)

// MetadataOnlyTable is a sentinel meaning "interpret via v1", so unlike the V1Table
// MetadataTable is a sentinel meaning "interpret via v1", so unlike the V1Table
// case above we apply no session-catalog / tracksPartitionsInCatalog guard -- any catalog
// returning MetadataOnlyTable has opted into v1 read semantics.
case t: MetadataOnlyTable =>
// returning MetadataTable has opted into v1 read semantics.
case t: MetadataTable =>
createDataSourceV1Scan(V1Table.toCatalogTable(catalog, ident, t))

case table =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1058,7 +1058,7 @@ class SessionCatalog(
// so the SubqueryAlias qualifier reflects the real catalog + multi-part namespace.
// Fall back to the historical 3-part form for v1 session-catalog tables -- we intentionally
// always include `SESSION_CATALOG_NAME` here and ignore
// `LEGACY_NON_IDENTIFIER_OUTPUT_CATALOG_NAME` to preserve pre-v2-MetadataOnlyTable behavior.
// `LEGACY_NON_IDENTIFIER_OUTPUT_CATALOG_NAME` to preserve pre-v2-MetadataTable behavior.
val multiParts = metadata.multipartIdentifier.getOrElse {
val qualifiedIdent = qualifyIdentifier(metadata.identifier)
Seq(CatalogManager.SESSION_CATALOG_NAME, qualifiedIdent.database.get, qualifiedIdent.table)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -447,7 +447,7 @@ case class CatalogTable(
ignoredProperties: Map[String, String] = Map.empty,
viewOriginalText: Option[String] = None,
// Multi-part identifier [catalog, namespace..., name] for tables synthesized from a v2
// `MetadataOnlyTable` whose namespace has more than one part -- the v1 `identifier:
// `MetadataTable` whose namespace has more than one part -- the v1 `identifier:
// TableIdentifier` (single-string database) cannot carry that losslessly. `None` for
// v1-native tables; callers should use `fullIdent` which falls back to `identifier.nameParts`.
multipartIdentifier: Option[Seq[String]] = None)
Expand Down
Loading