Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug(postgres): cannot parse types from information_schema #8845

Closed
gforsyth opened this issue Apr 1, 2024 · 0 comments · Fixed by #9055
Closed

bug(postgres): cannot parse types from information_schema #8845

gforsyth opened this issue Apr 1, 2024 · 0 comments · Fixed by #9055
Labels
bug Incorrect behavior inside of ibis postgres The PostgreSQL backend

Comments

@gforsyth
Copy link
Member

gforsyth commented Apr 1, 2024

For the information_schema database, if you attempt to pull a table the types have information_schema. prepended, which breaks our type parsing.

I can't get this to replicate for other databases, so I don't think it's an issue beyond info schema, but still might trip people up.

Additionally, there are several types in Postgres that are only present in information_schema:
https://www.postgresql.org/docs/current/infoschema-datatypes.html

[ins] In [1]: import ibis

[ins] In [2]: con = ibis.postgres.connect(
         ...:     user="postgres",
         ...:     password="postgres",
         ...:     host="0.0.0.0",
         ...:     port=5432,
         ...:     database="ibis_testing",
         ...: )

[ins] In [3]: con.table("domains", database="information_schema")
---------------------------------------------------------------------------
ParseError                                Traceback (most recent call last)
File /nix/store/lj17gpqka5s8v7cj3dfqqp4af45wa1pv-python3-3.11.8-env/lib/python3.11/site-packages/sqlglot/parser.py:1185, in Parser.parse_into(self, expression_types, raw_tokens, sql)
   1184 try:
-> 1185     return self._parse(parser, raw_tokens, sql)
   1186 except ParseError as e:

File /nix/store/lj17gpqka5s8v7cj3dfqqp4af45wa1pv-python3-3.11.8-env/lib/python3.11/site-packages/sqlglot/parser.py:1224, in Parser._parse(self, parse_method, raw_tokens, sql)
   1223 if self._index < len(self._tokens):
-> 1224     self.raise_error("Invalid expression / Unexpected token")
   1226 self.check_errors()

File /nix/store/lj17gpqka5s8v7cj3dfqqp4af45wa1pv-python3-3.11.8-env/lib/python3.11/site-packages/sqlglot/parser.py:1265, in Parser.raise_error(self, message, token)
   1264 if self.error_level == ErrorLevel.IMMEDIATE:
-> 1265     raise error
   1267 self.errors.append(error)

ParseError: Invalid expression / Unexpected token. Line 1, Col: 18.
  information_schema.sql_identifier

The above exception was the direct cause of the following exception:

ParseError                                Traceback (most recent call last)
Cell In[3], line 1
----> 1 con.table("domains", database="information_schema")

File ~/github.com/ibis-project/ibis/ibis/backends/sql/__init__.py:137, in SQLBackend.table(self, name, schema, database)
    134     catalog = table_loc.catalog or None
    135     database = table_loc.db or None
--> 137 table_schema = self.get_schema(name, catalog=catalog, database=database)
    138 return ops.DatabaseTable(
    139     name,
    140     schema=table_schema,
    141     source=self,
    142     namespace=ops.Namespace(catalog=catalog, database=database),
    143 ).to_expr()

File ~/github.com/ibis-project/ibis/ibis/backends/postgres/__init__.py:585, in Backend.get_schema(self, name, catalog, database)
    581 if not rows:
    582     raise com.IbisError(f"Table not found: {name!r}")
    584 return sch.Schema(
--> 585     {
    586         col: type_mapper.from_string(typestr, nullable=nullable)
    587         for col, typestr, nullable in rows
    588     }
    589 )

File ~/github.com/ibis-project/ibis/ibis/backends/postgres/__init__.py:586, in <dictcomp>(.0)
    581 if not rows:
    582     raise com.IbisError(f"Table not found: {name!r}")
    584 return sch.Schema(
    585     {
--> 586         col: type_mapper.from_string(typestr, nullable=nullable)
    587         for col, typestr, nullable in rows
    588     }
    589 )

File ~/github.com/ibis-project/ibis/ibis/backends/sql/datatypes.py:195, in SqlglotType.from_string(cls, text, nullable)
    192 if dtype := cls.unknown_type_strings.get(text.lower()):
    193     return dtype
--> 195 sgtype = sg.parse_one(text, into=sge.DataType, read=cls.dialect)
    196 return cls.to_ibis(sgtype, nullable=nullable)

File /nix/store/lj17gpqka5s8v7cj3dfqqp4af45wa1pv-python3-3.11.8-env/lib/python3.11/site-packages/sqlglot/__init__.py:122, in parse_one(sql, read, dialect, into, **opts)
    119 dialect = Dialect.get_or_raise(read or dialect)
    121 if into:
--> 122     result = dialect.parse_into(into, sql, **opts)
    123 else:
    124     result = dialect.parse(sql, **opts)

File /nix/store/lj17gpqka5s8v7cj3dfqqp4af45wa1pv-python3-3.11.8-env/lib/python3.11/site-packages/sqlglot/dialects/dialect.py:495, in Dialect.parse_into(self, expression_type, sql, **opts)
    492 def parse_into(
    493     self, expression_type: exp.IntoType, sql: str, **opts
    494 ) -> t.List[t.Optional[exp.Expression]]:
--> 495     return self.parser(**opts).parse_into(expression_type, self.tokenize(sql), sql)

File /nix/store/lj17gpqka5s8v7cj3dfqqp4af45wa1pv-python3-3.11.8-env/lib/python3.11/site-packages/sqlglot/parser.py:1190, in Parser.parse_into(self, expression_types, raw_tokens, sql)
   1187         e.errors[0]["into_expression"] = expression_type
   1188         errors.append(e)
-> 1190 raise ParseError(
   1191     f"Failed to parse '{sql or raw_tokens}' into {expression_types}",
   1192     errors=merge_errors(errors),
   1193 ) from errors[-1]

ParseError: Failed to parse 'information_schema.sql_identifier' into <class 'sqlglot.expressions.DataType'>
@gforsyth gforsyth added bug Incorrect behavior inside of ibis postgres The PostgreSQL backend labels Apr 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Incorrect behavior inside of ibis postgres The PostgreSQL backend
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant