Repository navigation
Releases: datamindedbe/incubator-dbt-polars
Releases · datamindedbe/incubator-dbt-polars
Release list
v0.5.0
Implements
- [Databricks Catalog]: Add support for iceberg tables in the Databricks catalog.
- [Fabric Catalog]: New catalog for Delta and Iceberg tables in a Microsoft Fabric Lakehouse.
- [Snowflake Catalog]: New catalog for Snowflake-managed Iceberg tables through Snowflake Horizon Catalog.
compute_threadsin the profile limits how many of thethreadsmodels run Polars at the same time; models waiting for an Iceberg commit let others compute.- Enable lazy evaluation of incremental models with
sync_all_columnsfor delta and iceberg tables.
Fixes
- [Iceberg Catalog] Improve efficiency of merge and delete+insert models using composite keys (#45).
- [Iceberg Catalog] Merge and delete+insert no longer fail when a
unique_keycolumn contains NULL values. - [Databricks Catalog] Dropping a Delta table on Azure now deletes its storage folder. Requires installing
dbt-polars-catalog-databricks[azure].
v0.4.0
Implements
- A source can be read from a path relative to its catalog with
external.location: a file with any extension (data.CSV) or a glob (orders/*.CSV) (#29)
Fixes
- Fail with error on SQL statements executed through hooks (#36)
- Resolve polar types in read options (
schema,override_schema) (#33) - Fail with error on unknown
read_optionskeys (#34) - Lazy append on incremental materialization (#31)
- Full refresh of an incremental model are now atomic
- Fail with error on unknown model config keys
v0.3.1
Implements
current_timestampmacro evaluates to naive timestamp when dbt run started.dateadd,datediff,date_spineandget_intervals_betweenmacro.date_truncmacro supportsweekandquarter.last_daymacro forweek,month,quarterandyear.- Move documentation to GitHub pages
Fixes
- Omit sensitive fields in the profile from
dbt debug - [Databricks Catalog]: Request READ instead of READ_WRITE credentials for read-only access
- [Databricks Catalog]: Update the description of all columns in a table with a single query
- Queried fail when a CTE has the same name as a model it selects from
- User .str.date to cast strings to date as this is the only supported way to do this conversion in Polars 2.0
- Throw an error when a user uses a hook or contract which is not yet supported
v0.3.0
Implements
- Catalogs are now published as separate packages
- Move tests into a package module (
dbt.adapters.polars.testing) for reusability in catalogs maintained outside this repo - [Databricks Catalog]: Add field
table_formatin the profile. Only allowed value isdelta.
Other improvements
- Stop using
pull_request_targetin CI
Documentation
- [Readme] Add list of key dbt-polars features