airbyte.caches.motherduck
A MotherDuck implementation of the PyAirbyte cache, built on DuckDB.
Usage Example
```python from airbyte as ab from airbyte.caches import MotherDuckCache
cache = MotherDuckCache( database="mydatabase", schema_name="myschema", api_key=ab.get_secret("MOTHERDUCK_API_KEY"), )
1"""A MotherDuck implementation of the PyAirbyte cache, built on DuckDB. 2 3## Usage Example 4 5```python 6from airbyte as ab 7from airbyte.caches import MotherDuckCache 8 9cache = MotherDuckCache( 10 database="mydatabase", 11 schema_name="myschema", 12 api_key=ab.get_secret("MOTHERDUCK_API_KEY"), 13) 14""" 15 16from __future__ import annotations 17 18from overrides import overrides 19from pydantic import Field 20 21from airbyte._processors.sql.motherduck import MotherDuckSqlProcessor 22from airbyte.caches.duckdb import DuckDBCache 23from airbyte.secrets import SecretString 24 25 26class MotherDuckCache(DuckDBCache): 27 """Cache that uses MotherDuck for external persistent storage.""" 28 29 db_path: str = Field(default="md:") 30 database: str 31 api_key: SecretString 32 33 _sql_processor_class = MotherDuckSqlProcessor 34 35 @overrides 36 def get_sql_alchemy_url(self) -> SecretString: 37 """Return the SQLAlchemy URL to use.""" 38 return SecretString( 39 f"duckdb:///md:{self.database}?motherduck_token={self.api_key}" 40 # f"&schema={self.schema_name}" # TODO: Debug why this doesn't work 41 ) 42 43 @overrides 44 def get_database_name(self) -> str: 45 """Return the name of the database.""" 46 return self.database
27class MotherDuckCache(DuckDBCache): 28 """Cache that uses MotherDuck for external persistent storage.""" 29 30 db_path: str = Field(default="md:") 31 database: str 32 api_key: SecretString 33 34 _sql_processor_class = MotherDuckSqlProcessor 35 36 @overrides 37 def get_sql_alchemy_url(self) -> SecretString: 38 """Return the SQLAlchemy URL to use.""" 39 return SecretString( 40 f"duckdb:///md:{self.database}?motherduck_token={self.api_key}" 41 # f"&schema={self.schema_name}" # TODO: Debug why this doesn't work 42 ) 43 44 @overrides 45 def get_database_name(self) -> str: 46 """Return the name of the database.""" 47 return self.database
Cache that uses MotherDuck for external persistent storage.
db_path: str
Normally db_path is a Path object.
The database name will be inferred from the file name. For example, given a db_path
of
/path/to/my/my_dbairbyte.caches.duckdb
, the database name is my_db
.
api_key: airbyte.secrets.base.SecretString
36 @overrides 37 def get_sql_alchemy_url(self) -> SecretString: 38 """Return the SQLAlchemy URL to use.""" 39 return SecretString( 40 f"duckdb:///md:{self.database}?motherduck_token={self.api_key}" 41 # f"&schema={self.schema_name}" # TODO: Debug why this doesn't work 42 )
Return the SQLAlchemy URL to use.
@overrides
def
get_database_name(self) -> str:
44 @overrides 45 def get_database_name(self) -> str: 46 """Return the name of the database.""" 47 return self.database
Return the name of the database.
Inherited Members
- pydantic.main.BaseModel
- BaseModel
- Config
- dict
- json
- parse_obj
- parse_raw
- parse_file
- from_orm
- construct
- copy
- schema
- schema_json
- validate
- update_forward_refs