-
Notifications
You must be signed in to change notification settings - Fork 356
Description
Bug: Snowflake Iceberg tables fail on CLONE and ALTER — missing ICEBERG keyword in DDL
Description
When using table_format = iceberg with Snowflake, SQLMesh generates standard DDL statements (CREATE TABLE ... CLONE, ALTER TABLE ...) that Snowflake rejects for Iceberg tables. Snowflake requires dedicated Iceberg-prefixed commands:
CREATE ICEBERG TABLE ... CLONEinstead ofCREATE TABLE ... CLONEALTER ICEBERG TABLE ...instead ofALTER TABLE ...
This causes ProgrammingError failures during both the virtual layer update (clone phase) and schema migration (alter phase).
How to Reproduce
- Configure a SQLMesh project targeting Snowflake with Iceberg table format
- Define any materialized model (e.g.,
INCREMENTAL_BY_TIME_RANGE) withtable_format iceberg - Run
sqlmesh planto create a dev environment that triggers cloning or schema migration
Error Messages
Clone error (during virtual layer update):
ProgrammingError: 002008 (55000): SQL compilation error:
Invalid operation The table STAGING__STG_CLOUDFRONT_LOGS__788519766 is an Iceberg table.
Clone Iceberg table should use CREATE ICEBERG TABLE CLONE command.
Please ensure the appropriate command format is used.
Alter error (during schema migration):
ProgrammingError: 091367 (42601): SQL compilation error:
The table STAGING__STG_TIVO_CONSENT_PREFERENCES__1774833790__dev is an Iceberg table.
Iceberg tables should use ALTER ICEBERG TABLE commands.
Please ensure the appropriate command format is used.
Root Cause
The table_format attribute is available on the model (model.table_format) and is correctly used during table creation (_create_table / ctas generate CREATE ICEBERG TABLE), but it is never propagated to the clone and alter code paths:
-
Clone:
EngineAdapter.clone_table()in base.py hardcodeskind="TABLE"inexp.Create. The Snowflake override in snowflake.py handlesTransientPropertybut not the Iceberg table_format. The evaluator's _clone_snapshot_in_dev() does not pass table_format to clone_table(). -
Alter:
TableAlterOperation.expressionin schema_diff.py hardcodeskind="TABLE"inexp.Alter. The base alter_table() method does not accept additional context, and the evaluator's migrate() does not pass table_format through.
Affected Code
| File | Method | Issue |
|---|---|---|
| sqlmesh/core/engine_adapter/base.py | clone_table() | Hardcodes kind="TABLE" |
| sqlmesh/core/engine_adapter/base.py | alter_table() | No mechanism to receive table_format |
| sqlmesh/core/engine_adapter/snowflake.py | clone_table() | Handles TransientProperty but not Iceberg |
| sqlmesh/core/engine_adapter/snowflake.py | — | No alter_table() override for Iceberg |
| sqlmesh/core/snapshot/evaluator.py | _clone_snapshot_in_dev() | Doesn't pass table_format to clone_table() |
| sqlmesh/core/snapshot/evaluator.py | migrate() | Doesn't pass table_format to alter_table() |
Environment
- SQLMesh version: 0.231.2
- Engine: Snowflake with Iceberg table format
- Trigger:
sqlmesh planwith dev environments (clone + migration phases)