-
Notifications
You must be signed in to change notification settings - Fork 359
Open
Description
Issue with Environment : SQLMesh with Iceberg Catalog
https://github.com/TobikoData/sqlmesh/blob/v0.218.0/sqlmesh/core/engine_adapter/base.py
def drop_data_object_on_type_mismatch(
self, data_object: t.Optional[DataObject], expected_type: DataObjectType
) -> bool:
"""Drops a data object if it exists and is not of the expected type.
Args:
data_object: The data object to check.
expected_type: The expected type of the data object.
Returns:
True if the data object was dropped, False otherwise.
"""
if data_object is None or data_object.type == expected_type:
return False
logger.warning(
"Target data object '%s' is a %s and not a %s, dropping it",
[data_object.to](http://data_object.to/)_table().sql(dialect=self.dialect),
data_object.type.value,
expected_type.value,
)
self.drop_data_object(data_object)
return True
I'm encountering an issue with Iceberg catalogs related to the drop_data_object_on_type_mismatch method from PR #5087 #5087. @Iaroslav (Tobiko)
Problem:
In Iceberg catalogs, views appear as "table" objects in information_schema.tables rather than information_schema.views.
This causes:
- data_object.type returns "table"
- expected_type is "view"
- Method drops the physical object before the virtual layer is created
Root Cause:
Iceberg's unified metadata approach treats both views and tables as table-type objects for consistency.
Question:
Should SQLMesh have special handling for Iceberg catalogs where view/table detection works differently, or should the type mismatch logic be more lenient for certain catalog types?
Reactions are currently unavailable