'trino' not in table.dialect_options #21541

justineyster · 2022-09-21T16:14:39Z

A clear and concise description of what the bug is.

This seems related to #20043.

Our Superset/Trino integration has been broken by a recent update but I can't yet figure out a fix.

Our dashboards and graphs that utilize Trino are failing with DB engine Error: 'TableClause' object has no attribute 'dialect_options'.

In the logs I'm also seeing 'trino' not in table.dialect_options and WARNING:superset.db_engine_specs:Unable to load SQLAlchemy dialect <class 'es.opendistro.sqlalchemy.ESHTTPSDialect'>: No module named 'sqlalchemy_trino'.

We're on version 1.5.1 of Superset installed on a Python 3.9 Docker image via pip with extras, i.e. pip install apache-superset[db2,druid,elasticsearch,mysql,postgres,thumbnails,trino].

I see there has been a lot of activity lately around the Trino engine in this repo. I'm hoping someone with more intimate understanding of the SQLAlchemy dialect situation might be able to shed some light on what's going on, and how to fix it.

How to reproduce the bug

Create a Superset dashboard with a Trino data source
Attempt to load the dashboard

Expected results

Dashboard loads and can access Trino.

Actual results

Errors mentioned above (stack trace at bottom).

Screenshots

Environment

(please complete the following information):

browser type and version: Chrome
superset version: 1.5.1
python version: 3.9
node.js version: N/A
any feature flags active:

Checklist

Make sure to follow these steps before submitting your issue - thank you!

I have checked the superset logs for python stacktraces and included it here as text if there are any.
I have reproduced the issue with at least the latest released version of superset.
I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

AttributeError: 'TableClause' object has no attribute 'dialect_options'
2022-09-21 15:28:03,660:ERROR:superset.views.base:'TableClause' object has no attribute 'dialect_options'
Traceback (most recent call last):
File "/opt/app-root/lib64/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request
  rv = self.dispatch_request()
File "/opt/app-root/lib64/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
  return self.view_functions[rule.endpoint](**req.view_args)
File "/opt/app-root/lib64/python3.9/site-packages/flask_appbuilder/security/decorators.py", line 104, in wraps
  return f(self, *args, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/superset/views/base_api.py", line 112, in wraps
  raise ex
File "/opt/app-root/lib64/python3.9/site-packages/superset/views/base_api.py", line 109, in wraps
  duration, response = time_function(f, self, *args, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/superset/utils/core.py", line 1468, in time_function
  response = func(*args, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/superset/utils/log.py", line 245, in wrapper
  value = f(*args, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/superset/charts/data/api.py", line 244, in data
  return self._get_data_response(
File "/opt/app-root/lib64/python3.9/site-packages/superset/charts/data/api.py", line 390, in _get_data_response
  result = command.run(force_cached=force_cached)
File "/opt/app-root/lib64/python3.9/site-packages/superset/charts/data/commands/get_data_command.py", line 43, in run
  payload = self._query_context.get_payload(
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_context.py", line 96, in get_payload
  return self._processor.get_payload(cache_query_context, force_cached)
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_context_processor.py", line 376, in get_payload
  query_results = [
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_context_processor.py", line 377, in <listcomp>
  get_query_results(
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_actions.py", line 200, in get_query_results
  return result_func(query_context, query_obj, force_cached)
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_actions.py", line 104, in _get_full
  payload = query_context.get_df_payload(query_obj, force_cached=force_cached)
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_context.py", line 115, in get_df_payload
  return self._processor.get_df_payload(query_obj, force_cached)
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_context_processor.py", line 126, in get_df_payload
  query_result = self.get_query_result(query_obj)
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_context_processor.py", line 186, in get_query_result
  result = query_context.datasource.query(query_object.to_dict())
File "/opt/app-root/lib64/python3.9/site-packages/superset/connectors/sqla/models.py", line 1693, in query
  query_str_ext = self.get_query_str_extended(query_obj)
File "/opt/app-root/lib64/python3.9/site-packages/superset/connectors/sqla/models.py", line 835, in get_query_str_extended
  sql = self.database.compile_sqla_query(sqlaq.sqla_query)
File "/opt/app-root/lib64/python3.9/site-packages/superset/models/core.py", line 466, in compile_sqla_query
  sql = str(qry.compile(engine, compile_kwargs={"literal_binds": True}))
File "<string>", line 1, in <lambda>
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/elements.py", line 481, in compile
  return self._compiler(dialect, bind=bind, **kw)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/elements.py", line 487, in _compiler
  return dialect.statement_compiler(dialect, self, **kw)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/compiler.py", line 592, in __init__
  Compiled.__init__(self, dialect, statement, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/compiler.py", line 322, in __init__
  self.string = self.process(self.statement, **compile_kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/compiler.py", line 352, in process
  return obj._compiler_dispatch(self, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/visitors.py", line 96, in _compiler_dispatch
  return meth(self, **kw)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/compiler.py", line 2201, in visit_select
  text = self._compose_select_body(
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/compiler.py", line 2292, in _compose_select_body
  [
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/compiler.py", line 2293, in <listcomp>
  f._compiler_dispatch(self, asfrom=True, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/visitors.py", line 96, in _compiler_dispatch
  return meth(self, **kw)
File "/opt/app-root/lib64/python3.9/site-packages/trino/sqlalchemy/compiler.py", line 121, in visit_table
  return self.add_catalog(sql, table)
File "/opt/app-root/lib64/python3.9/site-packages/trino/sqlalchemy/compiler.py", line 132, in add_catalog
  'trino' not in table.dialect_options
AttributeError: 'TableClause' object has no attribute 'dialect_options'

The text was updated successfully, but these errors were encountered:

justineyster · 2022-09-21T16:31:00Z

Note that we only encountered this error after upgrading the version of our UBI Python 3.9 image from 1-57 to 1-73: https://catalog.redhat.com/software/containers/ubi8/python-39/6065b24eb92fbda3a4c65d8f

Reverting back to 1-57 fixes the issue. But we'd much rather be on the latest patched version of the image.

rylativity · 2022-09-21T20:20:46Z

Exact same issue, error, and stacktrace as above, but using apache/superset:latest image from Docker Hub for testing and development.
And also like Justin, I was using Superset with Trino for a while without issue, and this error only started appearing recently.

iercan · 2022-09-22T12:15:53Z

We have same issue. We are using 1.5.0 docker image. Interestingly it gives this error with certain datasets.

iercan · 2022-09-22T12:23:09Z

Seem like issue is related with sqlalchemy-trino. I've downgraded it to 0.4.1 from 0.5.0 and problem solved for me.

justineyster · 2022-09-22T14:39:03Z

@iercan My understanding is that the sqlalchemy-trino project is deprecated and was removed from Superset for that reason. You can see a note about that here, which also states that the project has been merged into the upstream Trino package itself: https://github.com/dungdm93/sqlalchemy-trino. Personally, I'd rather not re-introduce a deprecated dependency.

iercan · 2022-09-22T14:45:21Z

You can install trino==0.315.0. Seems like latest version is problematic.

justineyster · 2022-09-22T16:05:53Z

@iercan Thanks for that info! I see that a duplicate issue has been raised here: trinodb/trino-python-client#237

EMCP · 2022-10-08T13:18:44Z

just a heads up.. I just tried

Superset v2.0.0
Trino 399
HIVE Metastore + S3 parquet files

and the latest trino==0.317.0 library.. it fails just to connect.. so I need to back down to v0.315.0 to connect and attempt to get going at all

georgewfisher · 2022-10-10T18:21:47Z

Using the latest release of Superset 2.0.0 with Trino 381 I can now load charts without this error. Previously I had this issue.

I have explicitly tried Superset 2.0.0, Trino 381, and "trino": ["trino>=0.317.0"], and that combination works.
Trino Python library fix for this issue is here: trinodb/trino-python-client#240

EMCP · 2022-10-10T18:24:20Z

I was on 398 Trino, now 399.. with that pypi version of the Trino Library I was not even able to connect to Trino in the GUI.. if I can I will try again with 317 driver but for now backed off the 315... I get connected but have now hit some sort of permissions issue possibly .. posted that to Trino slack

…

On Mon, Oct 10, 2022, 20:22 georgewfisher ***@***.***> wrote: Using the latest release of Superset 2.0.0 with Trino 381 I can now load charts without this error. Previously I had this issue. I have explicitly tried Superset 2.0.0, Trino 381, and "trino": ["trino>=0.317.0"], and that combination works. Trino Python library fix for this issue is here: trinodb/trino-python-client#240 <trinodb/trino-python-client#240> — Reply to this email directly, view it on GitHub <#21541 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA4FJSQIYGAII344R62VVSLWCRNEPANCNFSM6AAAAAAQSHNJ7M> . You are receiving this because you commented.Message ID: ***@***.***>

iercan · 2022-10-13T10:59:10Z

Latest python client solved this issue. Trino 399 with python client 0.318 works without error now. FYI.

EMCP · 2022-10-13T11:46:37Z

I can confirm with trino 399 + hive-metastore 3.x + S3 + trino==0.318.0 I can successfully start connecting and building datasets and charts!

rusackas · 2024-02-09T19:00:34Z

Are people still seeing this in Superset 3.0/3.1? This thread has been quiet for an awfully long time.

justineyster added the #bug Bug report label Sep 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

'trino' not in table.dialect_options #21541

'trino' not in table.dialect_options #21541

justineyster commented Sep 21, 2022 •

edited

justineyster commented Sep 21, 2022

rylativity commented Sep 21, 2022 •

edited

iercan commented Sep 22, 2022

iercan commented Sep 22, 2022

justineyster commented Sep 22, 2022

iercan commented Sep 22, 2022

justineyster commented Sep 22, 2022

EMCP commented Oct 8, 2022

georgewfisher commented Oct 10, 2022

EMCP commented Oct 10, 2022 via email

iercan commented Oct 13, 2022 •

edited

EMCP commented Oct 13, 2022

rusackas commented Feb 9, 2024

'trino' not in table.dialect_options #21541

'trino' not in table.dialect_options #21541

Comments

justineyster commented Sep 21, 2022 • edited

How to reproduce the bug

Expected results

Actual results

Screenshots

Environment

Checklist

Additional context

justineyster commented Sep 21, 2022

rylativity commented Sep 21, 2022 • edited

iercan commented Sep 22, 2022

iercan commented Sep 22, 2022

justineyster commented Sep 22, 2022

iercan commented Sep 22, 2022

justineyster commented Sep 22, 2022

EMCP commented Oct 8, 2022

georgewfisher commented Oct 10, 2022

EMCP commented Oct 10, 2022 via email

iercan commented Oct 13, 2022 • edited

EMCP commented Oct 13, 2022

rusackas commented Feb 9, 2024

justineyster commented Sep 21, 2022 •

edited

rylativity commented Sep 21, 2022 •

edited

iercan commented Oct 13, 2022 •

edited