Skip to content

Fix: address staged file path edge case in snowflake parsing#4233

Merged
georgesittas merged 1 commit intomainfrom
jo/fix_staged_file_path
Apr 23, 2025
Merged

Fix: address staged file path edge case in snowflake parsing#4233
georgesittas merged 1 commit intomainfrom
jo/fix_staged_file_path

Conversation

@georgesittas
Copy link
Contributor

The behavior today in main is as follows:

MODEL (
  name test_schema.@{table_name},
  kind FULL,
  blueprints (
    (table_name := "test_table", stage_path := 'foo'),
  )
);

SELECT * FROM @stage_path (file_format => 'bar') LIMIT 100
(.venv) ➜  sqlmesh render test_schema.test_table
Error: Failed to resolve macros for
SELECT
  *
FROM @stage_path(file_format => 'bar')
LIMIT 100
Macro 'stage_path' does not exist. at '.../playground/models/test_model.sql'

This is because the parsing is off:

>>> from sqlmesh.core.dialect import parse_one
>>> parse_one("""
... SELECT
...   *
... FROM @stage_path (FILE_FORMAT => 'foo')
... LIMIT 100
... """, dialect="snowflake")
Select(
  expressions=[
    Star()],
  limit=Limit(
    expression=Literal(this=100, is_string=False)),
  from=From(
    this=Table(
      this=MacroFunc(
        this=Anonymous(
          this=stage_path,
          expressions=[
            Kwarg(
              this=Var(this=FILE_FORMAT),
              expression=Literal(this='foo', is_string=True))])))))

Notice the MacroFunc here. Instead, we want this to be a StagedFilePath, because we can't disambiguate it from Snowflake's stage file syntax.

Alternatively, if I change the variable reference to @{stage_path}, then it fails with "Expecting ). Line 9, Col: 43." (=> highlighted).

@georgesittas georgesittas requested a review from tobymao April 23, 2025 17:15
@georgesittas georgesittas enabled auto-merge (squash) April 23, 2025 17:23
@georgesittas georgesittas disabled auto-merge April 23, 2025 17:25
@georgesittas georgesittas force-pushed the jo/fix_staged_file_path branch from 5217a36 to 2f7fd9e Compare April 23, 2025 17:27
@georgesittas georgesittas force-pushed the jo/fix_staged_file_path branch from 2f7fd9e to c3730ab Compare April 23, 2025 17:27
@georgesittas georgesittas enabled auto-merge (squash) April 23, 2025 17:28
@georgesittas georgesittas merged commit ef8834d into main Apr 23, 2025
22 checks passed
@georgesittas georgesittas deleted the jo/fix_staged_file_path branch April 23, 2025 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants