Fix sql_warehouse_name resolution: handle 'warehouses' API response key

[ataulmujeeb-cyber]

Summary

Fixes #63285

The _get_sql_endpoint_by_name method in DatabricksSqlHook fails with AirflowException: Can't list Databricks SQL endpoints whenever sql_warehouse_name (or sql_endpoint_name) is used instead of http_path.

Root Cause

The LIST_SQL_ENDPOINTS_ENDPOINT constant was updated to use the current Databricks API path (GET /api/2.0/sql/warehouses), but the response parsing in _get_sql_endpoint_by_name still checks for the legacy "endpoints" key. The current API returns data under "warehouses":

API Path	Response JSON Key
GET /api/2.0/sql/endpoints (legacy)	"endpoints"
GET /api/2.0/sql/warehouses (current)	"warehouses"

Since the code calls the new path but checks for the old key, the condition if "endpoints" not in result is always True, and the method always raises an exception.

Fix

Updated _get_sql_endpoint_by_name to check for both "warehouses" (current) and "endpoints" (legacy) response keys:

warehouses = result.get("warehouses") or result.get("endpoints")
if not warehouses:
    raise AirflowException("Can't list Databricks SQL warehouses")

This ensures backward compatibility if any Databricks workspace still returns the legacy format.

Changes

• providers/databricks/src/airflow/providers/databricks/hooks/databricks_sql.py: Updated _get_sql_endpoint_by_name to handle both "warehouses" and "endpoints" response keys
• providers/databricks/tests/unit/databricks/hooks/test_databricks_sql.py:
  • Updated existing fixture to use the current "warehouses" response format
  • Added TestGetSqlEndpointByName test class with 4 tests covering:
    • Current API response format ("warehouses" key)
    • Legacy API response format ("endpoints" key)
    • Warehouse name not found error
    • Empty API response error

Affected Components

This bug affects all operators and sensors that use sql_warehouse_name / sql_endpoint_name:

• DatabricksSqlSensor
• DatabricksPartitionSensor
• DatabricksSqlOperator
• Direct usage of DatabricksSqlHook with sql_endpoint_name

Versions Tested

• apache-airflow-providers-databricks==7.9.1
• Astronomer Runtime 3.1-11 (managed Astronomer deployment)
• Python 3.12

Workaround (for users on affected versions)

Use http_path directly instead of sql_warehouse_name:

# Instead of: sql_warehouse_name="My Warehouse"
http_path="/sql/1.0/warehouses/<warehouse_id>"

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack