Feat: allow user-supplied queries to generate unit test data#2579
Feat: allow user-supplied queries to generate unit test data#2579georgesittas merged 2 commits intomainfrom
Conversation
7550c84 to
215b6ed
Compare
sqlmesh/core/test/definition.py
Outdated
There was a problem hiding this comment.
this is confusing, but i think it makes sense, we need to normalize by the model, that's always true or else references will be wrong.
the parsing should be supplied by the actual test engine so there are no nuanences with transpilation
There was a problem hiding this comment.
both points are exactly what I had in mind 👍
215b6ed to
12771bf
Compare
|
Allowing users to supply arbitrary SQL to produce a resultset for tests is a nice escape hatch imo. Of course in an ideal world SQLGlot would be able to generate the correct query 100% of the time, but the variations in SQL syntax and obscure data types between engines are essentially infinite. I think allowing users to un-block themselves by supplying the query manually when they encounter an edge case will go a long way to reducing friction |
12771bf to
2c8ae82
Compare
2c8ae82 to
c580b0c
Compare
Fixes #2542, TL;DR this PR adds a new format for specifying unit test fixture/expected data: SQL. :-)
create_viewmethod because the fully-qualified view name in theCREATE VIEW <name> AS ...statement won't be quoted if we setquote_identifiers=False, which may cause problems.@erindru I ran your example against Trino and it seemed to work fine. I feel like the flexibility this feature provides will allow users to bypass conversion issues in unit tests pretty easily. Thanks for the idea!
EDIT: will make a followup PR to update the unit test documentation.