Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{TRY_}PARSE_JSON fails when fields are named as reserved keywords using colon notation #3061

Closed
seve-martinez opened this issue Mar 1, 2024 · 0 comments

Comments

@seve-martinez
Copy link

seve-martinez commented Mar 1, 2024

When using Snowflake, if a JSON blob contains a reserved keyword such as from, SqlGlot will interpret as a reserved keyword and fail to parse if using colon notation.

Fully reproducible code snippet
Please include a fully reproducible code snippet or the input sql, dialect, and expected output.

Dialect Snowflake

from sqlglot import parse_one, exp
from sqlglot.dialects import Snowflake
from sqlglot.optimizer import build_scope, find_all_in_scope
from sqlglot.optimizer.qualify import qualify


qry = """
  with a as (
    select '{"from": 12345, "to": 67890}' as json_blob
  )
  
  select try_parse_json(json_blob):from from a
"""

ast = parse_one(qry, dialect=Snowflake)
q_ast = qualify(ast, dialect=Snowflake)

root = build_scope(q_ast)

for scope in root.traverse():
    for column in set(scope.columns):
        print(column)

Will produce:

    ast = parse_one(qry, dialect=Snowflake)
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/__init__.py", line 123, in parse_one
    result = dialect.parse(sql, **opts)
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/dialects/dialect.py", line 444, in parse
    return self.parser(**opts).parse(self.tokenize(sql), sql)
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 1047, in parse
    return self._parse(
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 1113, in _parse
    expressions.append(parse_method(self))
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 1321, in _parse_statement
    expression = self._parse_set_operations(expression) if expression else self._parse_select()
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 2266, in _parse_select
    this = self._parse_statement()
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 1321, in _parse_statement
    expression = self._parse_set_operations(expression) if expression else self._parse_select()
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 2306, in _parse_select
    projections = self._parse_projections()
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 2254, in _parse_projections
    return self._parse_expressions()
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 5216, in _parse_expressions
    return self._parse_csv(self._parse_expression)
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 5170, in _parse_csv
    parse_result = parse_method()
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 3415, in _parse_expression
    return self._parse_alias(self._parse_conjunction())
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 3418, in _parse_conjunction
    return self._parse_tokens(self._parse_equality, self.CONJUNCTION)
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 5184, in _parse_tokens
    this = parse_method()
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 3421, in _parse_equality
    return self._parse_tokens(self._parse_comparison, self.EQUALITY)
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 5184, in _parse_tokens
    this = parse_method()
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 3424, in _parse_comparison
    return self._parse_tokens(self._parse_range, self.COMPARISON)
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 5184, in _parse_tokens
    this = parse_method()
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 3431, in _parse_range
    expression = self.RANGE_PARSERS[self._prev.token_type](self, this)
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/dialects/snowflake.py", line 294, in _parse_colon_get_path
    this = self.expression(exp.GetPath, this=this, expression=path)
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 1177, in expression
    return self.validate_expression(instance)
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 1197, in validate_expression
    self.raise_error(error_message)
  File "/Users/smartinez/.cbds-envs/af2-py310/lib/python3.10/site-packages/sqlglot/parser.py", line 1157, in raise_error
    raise error
sqlglot.errors.ParseError: Required keyword: 'expression' missing for <class 'sqlglot.expressions.GetPath'>. Line 6, Col: 39.
   (
    select '{"from": 12345, "to": 67890}' as json_blob
  )
  
  select try_parse_json(json_blob):from from a

As an aside, it works if using bracket notation:

---
select try_parse_json(json_blob['from']

Official Documentation
Please include links to official SQL documentation related to your issue.

https://docs.snowflake.com/en/sql-reference/functions/try_parse_json

@seve-martinez seve-martinez changed the title {TRY_}PARSE_JSON fails when fields are named as reserved keywords {TRY_}PARSE_JSON fails when fields are named as reserved keywords using colon notation Mar 1, 2024
@tobymao tobymao closed this as completed Mar 1, 2024
georgesittas added a commit that referenced this issue Mar 1, 2024
Co-authored-by: tobymao <toby.mao@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants