Skip to content

Trino hook run does not fail for bad query #20633

@sushi30

Description

@sushi30

Apache Airflow Provider(s)

trino

Versions of Apache Airflow Providers

Due to trinodb/trino-python-client#95, trino hook does not capture error when using the run command.

Apache Airflow version

2.2.2

Operating System

Centos 7

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

What happened

No response

What you expected to happen

Trino hook should fail when query fails.

How to reproduce

Just use the trino hook with a nonsense query like:

select aaaaa

Anything else

As a workaround I implemented this custom hook:

from airflow.providers.trino.hooks.trino import TrinoHook as BaseTrinoHook
class TrinoHook(BaseTrinoHook):
    def run(self, hql, autocommit: bool = False, parameters: Optional[dict] = None) -> None:
        """Due to issue https://github.com/trinodb/trino-python-client/issues/95 trino-python-client does not
        capture errors unless using fetchall. This hook is a workaround until this is fixed."""
        sql = [hql] if isinstance(hql, str) else hql
        sql = [self._strip_sql(s) for s in sql]
        with closing(self.get_conn()) as conn:
            if self.supports_autocommit:
                self.set_autocommit(conn, autocommit)
            with closing(conn.cursor()) as cur:
                for sql_statement in sql:
                    self._run_command(cur, sql_statement, parameters)
                cur.fetchone()

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions