Merged
Conversation
izeigerman
reviewed
Mar 4, 2025
izeigerman
approved these changes
Mar 4, 2025
tobymao
approved these changes
Mar 4, 2025
afzaljasani
pushed a commit
that referenced
this pull request
Apr 8, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The generation for brackets in DuckDB changed after v26.6.0, in order to support the latest semantics, introduced in v1.2.0.
In order to allow folks to use both the old and the new semantics, we introduced a new dialect setting that controls the version of the dialect, e.g.
dialect="duckdb, version=1.1.0".Dialect settings aren't properly supported in SQLMesh right now, because we hard code dialect strings in various places, but most importantly in the engine adapters:
https://github.com/TobikoData/sqlmesh/blob/5fb2c818a720a823d3f8ef45d047bb33f92376fc/sqlmesh/core/engine_adapter/bigquery.py#L62
This means that even if someone includes some settings in the config's default dialect, I expect that they won't be used when generating ASTs within the adapter module, e.g. when executing code, which can lead to issues.
Granted, only the DuckDB dialect is affected by this change, so perhaps this doesn't need to be a blocker for merging this PR, assuming that we can suggest upgrading DuckDB to the latest version as a workaround. But it may be an issue for other dialects in the future, so I'll leave this be for a while until we discuss how we want to solve this internally.
EDIT: we discussed this internally and concluded that roundtripping DuckDB SQL should work regardless of the version. The issue would only arise if someone wanted to transpile to duckdb and the version of the latter was <v1.2.0. This tradeoff is fine for now, happy to revisit this if needed.