Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sqlalchemy_url property to DbApiHook class #38871

Merged
merged 23 commits into from
Apr 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
c8dc5ce
Add sqlalchemy_url property to DbApiHook
rawwar Apr 9, 2024
d7a8a07
update get_sqlalchemy_url docstring
rawwar Apr 9, 2024
f520867
update drivername property docstring
rawwar Apr 9, 2024
f93e0f1
fix spelling
rawwar Apr 9, 2024
c9358d0
add sqlalchemy_url property to DbApiHook
rawwar Apr 10, 2024
6f10dd0
Merge branch 'main' into kalyan/connections/dbapihook-add-sa_uri
rawwar Apr 10, 2024
8e04a5c
fix typo
rawwar Apr 10, 2024
2e68951
Merge branch 'main' into kalyan/connections/dbapihook-add-sa_uri
rawwar Apr 10, 2024
7b0d999
Merge branch 'main' into kalyan/connections/dbapihook-add-sa_uri
rawwar Apr 10, 2024
248bd3f
Merge branch 'main' into kalyan/connections/dbapihook-add-sa_uri
rawwar Apr 10, 2024
af98f39
Merge branch 'main' into kalyan/connections/dbapihook-add-sa_uri
rawwar Apr 10, 2024
7226163
Merge branch 'main' into kalyan/connections/dbapihook-add-sa_uri
rawwar Apr 11, 2024
6df807c
precommit failure fix
rawwar Apr 11, 2024
08e1721
Merge branch 'main' into kalyan/connections/dbapihook-add-sa_uri
rawwar Apr 11, 2024
a68d30c
Merge branch 'main' into kalyan/connections/dbapihook-add-sa_uri
rawwar Apr 11, 2024
052974c
Merge branch 'main' into kalyan/connections/dbapihook-add-sa_uri
rawwar Apr 12, 2024
4bfae8c
Update sql.py
rawwar Apr 12, 2024
f0e32df
Merge branch 'main' of https://github.com/apache/airflow into kalyan/…
rawwar Apr 12, 2024
1227df8
Merge branch 'main' into kalyan/connections/dbapihook-add-sa_uri
rawwar Apr 12, 2024
62591f5
fix error message
rawwar Apr 12, 2024
c9a9e87
Merge branch 'kalyan/connections/dbapihook-add-sa_uri' of github.com:…
rawwar Apr 12, 2024
6af6ce2
Merge branch 'main' into kalyan/connections/dbapihook-add-sa_uri
rawwar Apr 12, 2024
88cdd8d
Merge branch 'main' into kalyan/connections/dbapihook-add-sa_uri
rawwar Apr 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
17 changes: 17 additions & 0 deletions airflow/providers/common/sql/hooks/sql.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@

if TYPE_CHECKING:
from pandas import DataFrame
from sqlalchemy.engine import URL

from airflow.providers.openlineage.extractors import OperatorLineage
from airflow.providers.openlineage.sqlparser import DatabaseInfo
Expand Down Expand Up @@ -209,6 +210,22 @@ def get_uri(self) -> str:
conn.schema = self.__schema or conn.schema
return conn.get_uri()

@property
def sqlalchemy_url(self) -> URL:
"""
Return a Sqlalchemy.engine.URL object from the connection.

Needs to be implemented in the provider subclass to return the sqlalchemy.engine.URL object.

:return: the extracted sqlalchemy.engine.URL object.
"""
qualname = f"{self.__class__.__module__}.{self.__class__.__qualname__}"
if qualname != "airflow.providers.common.sql.hooks.sql.DbApiHook":
msg = f"{qualname!r} does not implement/support built SQLAlchemy URL."
else:
msg = "`sqlalchemy_url` property should be implemented in the provider subclass."
raise NotImplementedError(msg)

def get_sqlalchemy_engine(self, engine_kwargs=None):
"""
Get an sqlalchemy_engine object.
Expand Down
3 changes: 3 additions & 0 deletions airflow/providers/common/sql/hooks/sql.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ from airflow.hooks.base import BaseHook as BaseHook
from airflow.providers.openlineage.extractors import OperatorLineage as OperatorLineage
from airflow.providers.openlineage.sqlparser import DatabaseInfo as DatabaseInfo
from pandas import DataFrame as DataFrame
from sqlalchemy.engine import URL as URL
from typing import Any, Callable, Generator, Iterable, Mapping, Protocol, Sequence, TypeVar, overload

T = TypeVar("T")
Expand All @@ -66,6 +67,8 @@ class DbApiHook(BaseHook):
def placeholder(self): ...
def get_conn(self): ...
def get_uri(self) -> str: ...
@property
def sqlalchemy_url(self) -> URL: ...
def get_sqlalchemy_engine(self, engine_kwargs: Incomplete | None = None): ...
def get_pandas_df(
self, sql, parameters: list | tuple | Mapping[str, Any] | None = None, **kwargs
Expand Down