1.12.1

Core Library

Quality of Life (fixing annoying little things)

• 2529-INFO_TABLES_QUERY_THRESHOLD-as-paramterer-from-config by @amirdataops in #2600
• warn when resolving configs or secrets with placeholder values by @djudjuu in #2636
• Prevent unecessary preliminary connection in dataset by @sh-rp in #2645
• QoL: warning with hint to provide data types for columns with exclusively None values by @anuunchin in #2633
• Fix issue 2690: switch to packaging to remove warning on import dlt by @djudjuu in #2707
• qol: exception formatting by @zilto in #2715
• Regular and standalone resources are now the same thing. Both provide nice typed callables, allow to be renamed and allow to inject secrets and configs in the same way - also when part of an inner function. This unifies injection behavior for all our decorators.
  In the example below (1) access_token secrets is allowed in inner resource (2) limit argument with default will be injected from ie. LIMIT env variable which was skipped before

@dlt.source
def source():
    @dlt.resource(write_disposition="merge", primary_key="_id")
    def documents(access_token=dlt.secrets.value, limit=10):
        yield from generate_json_like_data(access_token, limit)

    return documents

⚠️ Still we do not recommend to define parametrized inner resources.

• You can now return data from resources instead of yielding single item. We do not recommend that for code readability.dlt always wraps resources in generators so your return will be converted to yield.
• To return a DltResource from a resource function you must explicitly type the return value:

@dlt.resource
def rv_resource(name: str) -> DltResource:
    return dlt.resource([1, 2, 3], name=name, primary_key="value")

• normalizes config resolve behavior: default values can be overridden from providers but explicit cannot.
• ⚠️ previously, if those were instances of base configurations, behavior was inconsistent (explicit values were treated like defaults).
• ⚠️ if native value is found for a config and it does not accept native values, config resolution will fail, previously it was ignored
• We use custom, consistent wrap and unwrap of functions. our decorators preserve both typing and runtime signature of decorated functions. makefun got removed.
• if Incremental initializes from another Incremental as native value, it copies original type correctly
• dlt.resource can define configuration section (also using lambdas)

Bugfixes and improvements

• feat: Expand sql table resource config by @xneg in #2396

• Added write_disposition to sql table config
• Added primary_key and merge_key to sql table config

• Feat: Support clustered tables with custom column order in BigQuery destination by @hsm207 in #2638
• Feat: Add configuration propagation to deltalake.write_deltalake (#2629) by @gaatjeniksaan in #2640
• Add support for creating custom integer-range partition table in BigQuery by @hsm207 in #2676
• Upsert merge strategy for iceberg by @anuunchin in #2671
• Feat/add athena dabatase location option by @eric-pinkham-rw in #2708
• motherduck destination config improvement: uppercase env var by @djudjuu in #2703
• adds parquet support to postgres via adbc by @rudolfix in #2685
• 2681 - fixes null on non null column arrow by @rudolfix in #2721

• removes cffi version of psycopg2

• mssql and snowflake bugfixes by @rudolfix in #2756
• allows to configure configs and pragmas for duckdb, improves sql_client, tests @rudolfix in #2730
• logs resolved traces thread-wise, clears log between pipeline runs @rudolfix in #2730

Chores & tech debt
We switch to uv in the coming days and:

• Simplify workflow files by @sh-rp in #2663
• fix/2677: remove recursive filewatching by @zilto in #2678
• QoL: improved __repr__() for public interface by @zilto in #2630
• fix: incrementally watch files by @zilto in #2697
• Simplify pipeline test utils by @sh-rp in #2566 (we use data access and dataset for testing now)
• added constants for load_id col in _dlt_loads table by @zilto in #2729
• Update github workflow setup by @sh-rp in #2728
• fixes leaking datasets tests by @rudolfix in #2730

🧪 Upgrades to data access

• Normalize model files by @sh-rp in #2507
• dlt.transformation implementation by @sh-rp in #2528
• [transformations] decouples sqlglot lineage and schema generation from destination identifiers by @rudolfix in #2705

• All SQL queries are destination agnostic. For example
• Column lineage is computed and inferred. x-annotation hints are propagated
• SqlModel represent SQL query and is processed in extract, normalize and loaded in load step
• you can use scalar() on data access expressions ie.

# get latest processed package id
max_load_id = pipeline.dataset()._dlt_loads.load_id.max().scalar()

🧪 Cool experimental stuff:

Check out our new embedded pipeline explorer app

• dlt marimo app pre-release version by @sh-rp in #2662
• add marimo flag to show command by @sh-rp in #2741

dlt pipeline <name> show --marimo
dlt pipeline <name> show --marimo --edit

use edit option to enable Notebook/edit mode in Marimo + very cool Ibis dataset explorer

Docs

• docs: dlt+ iceberg destination partitioning by @burnash in #2686
• docs: fix invalid bigquery reference in athena destination by @goober in #2700
• docs: rest_api: clarify dlt resource and rest_api specific parameters by @burnash in #2710
• docs: plus: add merge strategies for dlt+ Iceberg destination by @burnash in #2749
• rest_api: document pagination hierarchy and add tests by @burnash in #2745
• docs: add session parameter to rest_api client configuration by @burnash in #2746
• docs: fix incorrect github_source function calls in tutorial by @axelearning in #2768

We updated contribution guidelines

• changes contrib and README by @rudolfix in #2666

• By default we do not accept more destinations (except a few like DuckLake or Trino)
• Each PR needs a test and (possibly) docs entry

New Contributors

• @amirdataops made their first contribution in #2600
• @goober made their first contribution in #2700
• @eric-pinkham-rw made their first contribution in #2708
• @axelearning made their first contribution in #2768

Full Changelog: 1.11.0...1.12.0

1.12.0

Core Library

Quality of Life (fixing annoying little things)

• 2529-INFO_TABLES_QUERY_THRESHOLD-as-paramterer-from-config by @amirdataops in #2600
• warn when resolving configs or secrets with placeholder values by @djudjuu in #2636
• Prevent unecessary preliminary connection in dataset by @sh-rp in #2645
• QoL: warning with hint to provide data types for columns with exclusively None values by @anuunchin in #2633
• Fix issue 2690: switch to packaging to remove warning on import dlt by @djudjuu in #2707
• qol: exception formatting by @zilto in #2715
• Regular and standalone resources are now the same thing. Both provide nice typed callables, allow to be renamed and allow to inject secrets and configs in the same way - also when part of an inner function. This unifies injection behavior for all our decorators.
  In the example below (1) access_token secrets is allowed in inner resource (2) limit argument with default will be injected from ie. LIMIT env variable which was skipped before

@dlt.source
def source():
    @dlt.resource(write_disposition="merge", primary_key="_id")
    def documents(access_token=dlt.secrets.value, limit=10):
        yield from generate_json_like_data(access_token, limit)

    return documents

⚠️ Still we do not recommend to define parametrized inner resources.

• You can now return data from resources instead of yielding single item. We do not recommend that for code readability.dlt always wraps resources in generators so your return will be converted to yield.
• To return a DltResource from a resource function you must explicitly type the return value:

@dlt.resource
def rv_resource(name: str) -> DltResource:
    return dlt.resource([1, 2, 3], name=name, primary_key="value")

• normalizes config resolve behavior: default values can be overridden from providers but explicit cannot.
• ⚠️ previously, if those were instances of base configurations, behavior was inconsistent (explicit values were treated like defaults).
• ⚠️ if native value is found for a config and it does not accept native values, config resolution will fail, previously it was ignored
• We use custom, consistent wrap and unwrap of functions. our decorators preserve both typing and runtime signature of decorated functions. makefun got removed.
• if Incremental initializes from another Incremental as native value, it copies original type correctly
• dlt.resource can define configuration section (also using lambdas)

Bugfixes and improvements

• feat: Expand sql table resource config by @xneg in #2396

• Added write_disposition to sql table config
• Added primary_key and merge_key to sql table config

• Feat: Support clustered tables with custom column order in BigQuery destination by @hsm207 in #2638
• Feat: Add configuration propagation to deltalake.write_deltalake (#2629) by @gaatjeniksaan in #2640
• Add support for creating custom integer-range partition table in BigQuery by @hsm207 in #2676
• Upsert merge strategy for iceberg by @anuunchin in #2671
• Feat/add athena dabatase location option by @eric-pinkham-rw in #2708
• motherduck destination config improvement: uppercase env var by @djudjuu in #2703
• adds parquet support to postgres via adbc by @rudolfix in #2685
• 2681 - fixes null on non null column arrow by @rudolfix in #2721

• removes cffi version of psycopg2

• mssql and snowflake bugfixes by @rudolfix in #2756
• allows to configure configs and pragmas for duckdb, improves sql_client, tests @rudolfix in #2730
• logs resolved traces thread-wise, clears log between pipeline runs @rudolfix in #2730

Chores & tech debt
We switch to uv in the coming days and:

• Simplify workflow files by @sh-rp in #2663
• fix/2677: remove recursive filewatching by @zilto in #2678
• QoL: improved __repr__() for public interface by @zilto in #2630
• fix: incrementally watch files by @zilto in #2697
• Simplify pipeline test utils by @sh-rp in #2566 (we use data access and dataset for testing now)
• added constants for load_id col in _dlt_loads table by @zilto in #2729
• Update github workflow setup by @sh-rp in #2728
• fixes leaking datasets tests by @rudolfix in #2730

🧪 Upgrades to data access

• Normalize model files by @sh-rp in #2507
• dlt.transformation implementation by @sh-rp in #2528
• [transformations] decouples sqlglot lineage and schema generation from destination identifiers by @rudolfix in #2705

• All SQL queries are destination agnostic. For example
• Column lineage is computed and inferred. x-annotation hints are propagated
• SqlModel represent SQL query and is processed in extract, normalize and loaded in load step
• you can use scalar() on data access expressions ie.

# get latest processed package id
max_load_id = pipeline.dataset()._dlt_loads.load_id.max().scalar()

🧪 Cool experimental stuff:

Check out our new embedded pipeline explorer app

• dlt marimo app pre-release version by @sh-rp in #2662
• add marimo flag to show command by @sh-rp in #2741

dlt pipeline <name> show --marimo
dlt pipeline <name> show --marimo --edit

use edit option to enable Notebook/edit mode in Marimo + very cool Ibis dataset explorer

Docs

• docs: dlt+ iceberg destination partitioning by @burnash in #2686
• docs: fix invalid bigquery reference in athena destination by @goober in #2700
• docs: rest_api: clarify dlt resource and rest_api specific parameters by @burnash in #2710
• docs: plus: add merge strategies for dlt+ Iceberg destination by @burnash in #2749
• rest_api: document pagination hierarchy and add tests by @burnash in #2745
• docs: add session parameter to rest_api client configuration by @burnash in #2746
• docs: fix incorrect github_source function calls in tutorial by @axelearning in #2768

We updated contribution guidelines

• changes contrib and README by @rudolfix in #2666

• By default we do not accept more destinations (except a few like DuckLake or Trino)
• Each PR needs a test and (possibly) docs entry

New Contributors

• @amirdataops made their first contribution in #2600
• @goober made their first contribution in #2700
• @eric-pinkham-rw made their first contribution in #2708
• @axelearning made their first contribution in #2768

Full Changelog: 1.11.0...1.12.0

1.11.0

Core Library

• feat: adds iceberg table properties configuration for athena (#2546) by @olexanderos in #2555
• use pendulum.parse instead of fromisoformat when deserializing pua json to avoid losing timezone info by @sh-rp in #2514
• #2192 - adds base64 encoded PEM, private_key_path for Snowflake auth, improves docs by @rudolfix in #2569
• cli: do not use field name as a placeholder name in generated TOML files by @burnash in #2468
• Speedup CI: Cache google secrets by @sh-rp in #2581
• Fix: drop corresponding staging table when original table is dropped by @anuunchin in #2567
• normalize_py_arrow_item now replaces load id column with the right one by @anuunchin in #2526
• explicit snowflake autocommit=true when connection opens by @rudolfix in #2593
• list secrets in vault config provider to avoid calls to backend, now it is called only for known keys by @rudolfix in #2597
• Enabling 'model' loader_file_format for athena, synapse and dremio by @anuunchin in #2556
• refactor init-command for use in dlt project by @djudjuu in #2568
• allows to pass config section to dlt.resource, fixes a few edge cases when configs for standalone resources are resolved @rudolfix in #2583
• enables fsspec per-thread instance cache and updates documentation, this prevents excessive memory usage reported by users by @rudolfix in #2621
• bumps pendulum to 3.0.1, removes dlt flavored pendulum in Python 3.13 by @rudolfix in #2624
• Feat/2609-clickhouse-precision-integer-mapping by @hsm207 in #2627
• removes airflow Airflow DummyOperator import, supports up to v 2.10 by @rudolfix in #2628
• fixes fsspec instantiation in fs source (where kwargs were ignored) by @rudolfix in #2634

Docs

• Docs: add documentation for CT columns by @akelad in #2552
• docs: marimo docs page added by @zilto in #2584
• athena: fix a typo in the athena_adapter docstring by @burnash in #2599
• adds proper docs for Google Secrets vault config provider @rudolfix in #2597
• Extract dataset code snippets into tests snippets system by @sh-rp in #2598
• fix some typos in cursor-restapi docs by @hsm207 in #2608
• docs: Fix incorrect nesting in secrets.toml by @agrueneberg in #2614
• fixes parquet data writer settings docs & rewrites configuration docs by @rudolfix in #2583
• Added dedup sort example by @dat-a-man in #2235
• docs: add advanced project tutorial by @sh-rp in #2338
• docs: split incremental loading page by @burnash in #2592
• Added info about dlt's internal tables by @dat-a-man in #2525
• Added section using xmin for Change Data Capture (CDC) to pg_replication docs by @dat-a-man in #2535
• Added Transform data with add_map documentation by @dat-a-man in #2500
• cli: fix 404 in the "ai" command output by @burnash in #2643
• [doc/dlt+] renames sidebar and adds intro snippet for snowflake+ by @rudolfix in #2642

New Contributors

• @olexanderos made their first contribution in #2555
• @agrueneberg made their first contribution in #2614

Full Changelog: 1.10.0...1.11.0

1.10.0

Core Library

• feat(paginator): enhance JSONResponseCursorPaginator to support cursor placement in request JSON body by @kang8 in #2446
• fix: better import exception for numpy by @zilto in #2397
• Fix read parquet chunk size by @diwu-sf in #2456
• rest_api: enhance placeholder expansion to preserve value types by @burnash in #2462
• list-destination flag added by @zilto in #2441
• Fixes bigquery, ibis tests by @anuunchin in #2470
• add ignore_unknown_values option to BigQuery destination by @xxntti3n in #2455
• Fix: insert-from-staging replace strategy incorrectly using staging-optimized pattern by @anuunchin in #2435
• fix: index invariant add_row_hash_to_table by @hagelborn in #2491
• adds condition to check XDG_DATA_HOME to set global_dir value by @goosethedev in #2361
• removes location tag from athena iceberg, fixes catalog name, allows for additional props @rudolfix in #2478
• adds default dataset/schema to ibis expressions, support for athena and databricks backends @rudolfix in #2478
• 2457-refactors iceberg and duckdb cache support by @rudolfix in #2430
• fixes wrong resolve for WithLocalFiles configuration by @rudolfix in #2430
• converts Iceberg fileio into dlt credentials by @rudolfix in #2430
• bumps and simplifies deltalake, enables streaming appends and upserts by @rudolfix in #2430
• fixes nullability warning on duckdb ALTER by @rudolfix in #2430
• adds replace strategy selector, internal x-replace-strategy hint, removes sql_params by @rudolfix in #2430
• borrows and returns sqlalchemy connections in destination and several other critical fixes to sqllite, connection management, merge operations by @rudolfix in #2430
• executes all sql jobs (replace, copy, merge) in a transaction on all destinations by @rudolfix in #2430
• shows info on locations for config providers when displaying exceptions, hides warnings when project context is present by @rudolfix in #2430
• Fix/add s3 region for redshift staging destination related to 2349 by @yannik207 in #2389
• Fix: sqlalchemy table names of type sqlalchemy.sql.elements.quoted_name causing deserialization error by @anuunchin in
  #2496
• Preserve type of emtpy lists in incremental to allow to materialize empty resources by @sh-rp in #2511
• Fix clickhouse sql syntax for sentinel table creation by @sh-rp in #2510
• cli: dlt ai setup $IDE add cursor rules to REST API source to your dlt project (and more) @zilto in #2503
• "model" item_format: support for lazily evaluated tables/data frames by @anuunchin in #2423
• adds arrow ipc nested type encoding to json type by @rudolfix in #2519
• [chore] imports addon function when generating destination test cases to extend implemented destinations and test cases. allows for followup job in job client tests by @rudolfix in #2519
• allows iceberg to infer unified schema, allows csv files, uses explicit select list in sql_client filesystem by @rudolfix in #2519
• allows to use context manager on dataset to keep internal connection open + lifecycle tests by @rudolfix in #2519
• uses union by name when reading parquet via duckdb to see all columns when schema evolves by @rudolfix in #2519
• makes vault provider to check known toml fragments also for non-secrets by @rudolfix in #2519
• handles non existing iceberg tables in sql client and during registration by @rudolfix in #2519
• refactors how deterministic 'temp' tables are named for destinations that do not support temp tables by @rudolfix in #2519
• fixes open cursor not closed in mssql and athena sql_client by @rudolfix in #2519

Docs

• rest_api: docs: remove redundant authentication types section by @burnash in #2461
• docs: MCP installation docs page by @zilto in #2463
• Add snowflake plus source docs by @VioletM in #2450
• Docs: tip on explicitly disablint merg_key hint by @anuunchin in #2448
• Docs: BigQuery incorrect reference by @michelzurkirchen in #2490
• Added info about additional permissions to iceberg docs by @dat-a-man in #2453
• update dlt and dlt+ installations by @Soot3 in #2444
• docs: plus: snowflake_plus: added details on catalog_sync by @burnash in #2494
• docs: update incremental loading section in REST API documentation by @burnash in #2474
• Docs: fix example drop pipeline commands by @michelzurkirchen in #2506
• docs: extract Iceberg docs from delta-iceberg page by @burnash in #2472
• Update iceberg destination docs by @VioletM in #2538
• docs: snowflake_plus: document iceberg_mode and base_location templating by @burnash in #2524
• MCP sse-proxy docs by @djudjuu in #2504

Verified Sources (and rules)

We support dlt ai command via https://github.com/dlt-hub/verified-sources/tree/master/ai

• adds cursor rules for building REST API Sources dlt-hub/verified-sources#608 @adrianbr @zilto @rudolfix
  If you have better cursor rules or rules for other workflows contribute them here

New Contributors

• @diwu-sf made their first contribution in #2456
• @xxntti3n made their first contribution in #2455
• @Soot3 made their first contribution in #2444
• @hagelborn made their first contribution in #2491
• @goosethedev made their first contribution in #2361
• @yannik207 made their first contribution in #2389

Full Changelog: 1.9.0...1.10.0

1.9.0

Core Library

• Apply hints for nested tables: set data types, data contract via apply_hints or resource decorator by @steinitzu @rudolfix @zilto in #2165
• Apply hints for nested tables: convert nested tables into root tables by setting primary/merge keys and write disposition by @steinitzu @rudolfix @zilto in #2165
• rest_api: interpolate json and headers for top resources by @burnash in #2437
• rest_api: added parametric expressions for headers by @francescomucio in #2262
• feat(rest_client): allow specifying headers on a per-request basis by @joscha in #2434
• rest_api: add tests for handling escaped braces in string interpolation by @burnash in #2416
• Fix/133 ibis iceberg delta by @anuunchin in #2371
• Adding sql dialect to destination capabilities by @anuunchin in #2393
• un-deprecates force_iceberg on athena via new destination cap settings by @rudolfix in #2417
• (bugfix) correctly handles bucket url with path for Snowflake S3 Stage + copy command generator refactor by @stevenayers in #2354
• Optional engine_adapter_callback by @michelzurkirchen in #2427

Docs

• Apply hints for nested tables: basic use case doc by @rudolfix
• • Apply hints for nested tables: primary key example by @rudolfix
• Add documentation for connecting to a database over ssh by @dat-a-man in #2238
• Added docs for data security and encryption by @dat-a-man in #2326
• Updated the adjust a schema and sql configuration docs by @dat-a-man in #2387
• docs/project-entityfactory-refactor (dlt+) by @djudjuu in #2414
• Comment consistently by @rosner in #2418
• docs: mention python _ctypes support by @djudjuu in #2383

Verified Sources

• mongodb: Allow projections and mongoarrow schema by @zilto @anuunchin dlt-hub/verified-sources#592
• pg_replication: allows schema replications for all tables + leaking connection fixes and other edge cases + example of auto adding new tables by @rudolfix dlt-hub/verified-sources#604

New Contributors

• @djudjuu made their first contribution in #2414
• @rosner made their first contribution in #2418
• @stevenayers made their first contribution in #2354
• @michelzurkirchen made their first contribution in #2427

Full Changelog: 1.8.1...1.9.0

1.8.1

Bugfixes

• adds support for http and path style for filesystem sql_client with s3 by @sh-rp in #2343
• fix: required fields in nested pydantic models need to follow parent nullability by @joscha in #2199
• does not default to empty body in dependent resources in rest-api by @rudolfix in #2388
• clean up: remove redundant imports by @burnash in #2353

Full Changelog: 1.8.0...1.8.1

1.8.0

Core Library

• rewrites python - arrow backend for sql_database, better type conversion by @zilto in #2295
• Updated sql database docs for selecting tables and columns via config.toml by @dat-a-man in #2243
• limit schema and state retrieval queries by @sh-rp in #2345
• Adds resource field reference syntax to template strings by @burnash in #2210
• Feat/2251-use vectorized scanner for Snowflake/Parquet by @anuunchin in #2329
• feat: support encoding custom types in json modules by @joscha in #2286
• allows to apply table level hints via make_hints by @rudolfix in #2365
• removes memory leak in schema cache, skips schema validation in internal storage by @rudolfix in #2373

Docs

• Add missing import by @arjunanan6 in #2362
• docs: standardize capitalization in titles and labels by @burnash in #2359
• docs: plus: fix sample Claude config by @burnash in #2357
• docs: add warning about environment variables in Claude config by @burnash in #2368
• update docusaurus & other docs dependency updates by @sh-rp in #2331

Others

• authorizes workflows to be run if label is assigned (allowing to run full ci on forks) by @rudolfix in #2358

New Contributors

• @arjunanan6 made their first contribution in #2362
• @stantsvek made their first contribution in #2337

Full Changelog: 1.7.0...1.8.0

1.7.0

dlt 1.7.0 is mostly a maintenance release that should not impact your existing pipelines when upgrading.

What's Changed

Improvements

• Enabling available subcommands when cli isn't installed by @anuunchin in #2272
• renames tmp_dir to local_dir by @rudolfix in #2305
• feat: adding HeaderCursorPaginator by @ciklista in #2281
• rest_api: add tests for query string parameters by @burnash in #2268
• adds common tmp dir and ref importer by @rudolfix in #2276

Bugs

• fixes custom destination to go line by line in typed jsonl by @sh-rp in #2314
• #2282 fix import run dir module by @rudolfix in #2313
• fix upsert primary key selection bug by @rudolfix in #2283
• Correct type hints of construction parameter total_path for OffsetPaginator and PageNumberPaginator by @Quarkay in #2312

Docs

• docs: Add HeaderCursorPaginator documentation to RESTClient by @burnash in #2308
• Updated performance docs for setting dlt data dir by @dat-a-man in #2247
• update minimum supported Python version installation.md by @gregorywaynepower in #2332
• Add dlt plus docs by @akelad in #2285
• Orchestra docs (#2237) by @adrianbr in #2333

New Contributors

• @ciklista made their first contribution in #2281
• @Quarkay made their first contribution in #2312
• @gregorywaynepower made their first contribution in #2332

Full Changelog: 1.6.1...1.7.0

1.6.1

Core Library

Python version support changes:

• we drop Python 3.8 support
• we add Python 3.13 support

Python 3.13 package depends on dlt-pendulum (datetime library). It is a fork of pendulum that we are going to support until we are able to merge our changes into the original repo. There are a few nice things in it (we got rid of test dependencies and now it works on Alpine linux etc.)
Note: other Python versions still use original pendulum.

A big improvement of Databricks destination:

• uses internal staging by default (Volumes), no need to declare staging filesystem pipeline
• uses predefined staging volume or creates one automatically
• can figure out credentials if running in Databricks notebook
• can take any credentials and warehouse settings when present in the environment
  overall you can run dlt pipelines in a Databrick notebook without any additional configuration

Improvements to SQLALchemy destination:

• we configured several destinations like oracle, db2, sap hana, exasol, trino etc. but still need our community help to try them out.

More stuff

• recognizes ARRAY data type and converts to nested, fixes minimal nested types handling for sqlalchemy backend by @rudolfix in #2177
• #2020 #2151 - fixes dbt 1.8.6 and arrow dict types by @rudolfix in #2175
• fix: unravel Optional to inner generic arg from instance by @otosky in #2172
• bug: support lag for non-UTC datetime cursor fields by @hairrrrr in #2170
• Added note for missing credentials error for windows authentication by @dat-a-man in #2095
• unifies handling of naive datetime for arrow and object incrementals by @rudolfix in #2184
• Force use of typeddict from typingexentions, pin poetry in tests, simple disable python 3.8 by @sh-rp in #2185
• drop python 3.8, enable python 3.13, and enable full linting for 3.12 by @sh-rp in #2194
• add downloads badge by @sh-rp in #2211
• fix: streamlit sidebar button by @zilto in #2217
• Declare pip dependency for cli extras by @jonbiemond in #2207
• #2110 adds several not found error message patterns to sqlalchemy destination by @rudolfix in #2213
• Add python 3.12 and 3.13 to bug report template by @sh-rp in #2239
• Update branch naming guide in contributing doc to add a "keep" prefix by @sh-rp in #2242
• typed entity registries by @rudolfix in #2236
• Fix: Support azure default credentials by @gaatjeniksaan in #2244
• added exception for unsupported arrow types by @zilto in #2246
• Nicer cli help output and generated cli reference by @sh-rp in #2232
• #2226 enables dbt 1.9 (also enables or fixes clickhouse, databricks and synapse profiles) by @rudolfix in #2250
• #2166 databricks direct loading by @donotpush in #2219
• #2249 sqlalchemy indexes off by default by @rudolfix in #2253
• chore: takes dlt-pendulum from pypi by @rudolfix in #2254

Docs

• Docs: fix credentials link (add slash) by @AstrakhantsevaAA in #2162
• docs: Fix typo in file name in "Create a pipeline" walkthrough by @lpillmann in #2255
• Performance page update with parallelism pitfalls section by @sh-rp in #2240
• Updated credentials docs "Added section Destination configuration via Python" by @dat-a-man in #2182

New Contributors

• @otosky made their first contribution in #2172
• @zilto made their first contribution in #2217
• @jonbiemond made their first contribution in #2207
• @gaatjeniksaan made their first contribution in #2244
• @lpillmann made their first contribution in #2255

Full Changelog: 1.5.0...1.6.0

1.6.0a0

Alpha release 1.6.0a0

This is alpha release before Python version support changes.

• we drop Python 3.8 support
• we add Python 3.13 support

Python 3.13 requires pendulum (our datetime library) release. It is available via dltHub PyPI. If you are on Python 3.13 use the following command to install dlt

pip install dlt==1.6.0a0 --index=https://pypi.dlthub.com

Other changes

• recognizes ARRAY data type and converts to nested, fixes minimal nested types handling for sqlalchemy backend by @rudolfix in #2177
• #2020 #2151 - fixes dbt 1.8.6 and arrow dict types by @rudolfix in #2175
• fix: unravel Optional to inner generic arg from instance by @otosky in #2172
• bug: support lag for non-UTC datetime cursor fields by @hairrrrr in #2170
• unifies handling of naive datetime for arrow and object incrementals by @rudolfix in #2184
• Force use of typeddict from typingexentions, pin poetry in tests, simple disable python 3.8 by @sh-rp in #2185

New Contributors

• @otosky made their first contribution in #2172

Full Changelog: 1.5.0...1.6.0a0