Sqlsensor for bigquery fails on GoogleBaseHook with default credentials

[ying-w]

Apache Airflow version

2.5.2

What happened

Using sqlsensor with bigquery authenticated using secrets backend will fail with error

The connection type is not supported by SqlSensor. The associated hook should be a subclass of DbApiHook. Got GoogleBaseHook

Overriding the sensor as described here can be used as a workaround.

What you think should happen instead

When using SqlSensor, GoogleBaseHook should be coerced into BigQueryHook or whatever type the database being queried is. Google credentials can be used for many different functions. I do understand that there is probably some ambiguity around which sql backend trying to access GCP sql products but is there a way to infer or pass in? I would ideally want to reference the same credentials for multiple GCP operations.

How to reproduce

from airflow.providers.common.sql.sensors.sql import SqlSensor and then specify a connection using secrets backend where secrets value starts with google-cloud-platform://

These errors do not occur when directly calling bigquery functions like BigQueryInsertJobOperator() and using the same connection in gcp_conn_id=

Operating System

OS X 13.2.1

Versions of Apache Airflow Providers

apache-airflow-providers-google==8.11.0

Deployment

Astronomer

Deployment details

local testing

Anything else

Maybe this is unexpected behavior / better documenting error message rather than bug

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[hussein-awala]

If you want to use SqlSensor with BigQuery, you should use a connection of type gcpbigquery, in this case the SqlSensor will initiate an instance from BigQueryHook which extends DbApiHook.

But when you provide a connection of type google_cloud_platform, the sensor will use GoogleBaseHook. This connection can be used to communicate with GCP API, or in the sensors and operators created in the provider google. For now, the only way to initiate and use the correct hook in the other providers is providing a connection from the exact type defined in the hook and not a connection defined in the parent of the hook class.