Fix(snowflake): extract date from datetime values, quote audit's this_model#1269
Merged
georgesittas merged 3 commits intomainfrom Aug 5, 2023
Merged
Fix(snowflake): extract date from datetime values, quote audit's this_model#1269georgesittas merged 3 commits intomainfrom
georgesittas merged 3 commits intomainfrom
Conversation
tobymao
approved these changes
Aug 5, 2023
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
These changes ensure that we can successfully run the quickstart example in Snowflake (tested).
The two issues I faced when I tried to run
sqlmesh planfor it were:Got the error
Failed to cast variant value .... to DATEwhen trying to evaluate the seed model. The problem was that the dataframe we pass inwrite_pandashad a column of typedatetime64[ns], but the underlying column's type wasDATE(ds's type). See related SO discussion which I included in a comment here.After fixing the above issue, I got another error when trying to evaluate the full model, specifically when trying to run its
assert_positive_order_idsaudit. The reason was that inAudit.render_querywe were creating a query with a table whose name is produced by the snapshot'stable_namemethod. Since the resulting name is prefixed with the (case-sensitive) physical schema, we would then normalize it in the renderer'srendermethod and hence produce an invalid schema reference. For example,sqlmesh__SQLMESH_EXAMPLE.<model_name>would be converted intoSQLMESH__SQLMESH_EXAMPLE.<model_name>, which didn't exist. I fixed this error by enforcing quotes in said name before rendering, which I believe should be ok because the model name is already normalized, only the schema isn't.