Skip to content

Fix(snowflake): extract date from datetime values, quote audit's this_model#1269

Merged
georgesittas merged 3 commits intomainfrom
jo/snowflake_fixes
Aug 5, 2023
Merged

Fix(snowflake): extract date from datetime values, quote audit's this_model#1269
georgesittas merged 3 commits intomainfrom
jo/snowflake_fixes

Conversation

@georgesittas
Copy link
Contributor

These changes ensure that we can successfully run the quickstart example in Snowflake (tested).

The two issues I faced when I tried to run sqlmesh plan for it were:

  1. Got the error Failed to cast variant value .... to DATE when trying to evaluate the seed model. The problem was that the dataframe we pass in write_pandas had a column of type datetime64[ns], but the underlying column's type was DATE (ds's type). See related SO discussion which I included in a comment here.

  2. After fixing the above issue, I got another error when trying to evaluate the full model, specifically when trying to run its assert_positive_order_ids audit. The reason was that in Audit.render_query we were creating a query with a table whose name is produced by the snapshot's table_name method. Since the resulting name is prefixed with the (case-sensitive) physical schema, we would then normalize it in the renderer's render method and hence produce an invalid schema reference. For example, sqlmesh__SQLMESH_EXAMPLE.<model_name> would be converted into SQLMESH__SQLMESH_EXAMPLE.<model_name>, which didn't exist. I fixed this error by enforcing quotes in said name before rendering, which I believe should be ok because the model name is already normalized, only the schema isn't.

@georgesittas georgesittas changed the title Fix(snowflake): extract date part of datetime values, quote audit's this_model Fix(snowflake): extract date from datetime values, quote audit's this_model Aug 5, 2023
@georgesittas georgesittas merged commit f98583c into main Aug 5, 2023
@georgesittas georgesittas deleted the jo/snowflake_fixes branch August 5, 2023 03:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants