Skip to content

v0.20.0

Choose a tag to compare

@amotl amotl released this 21 Oct 12:09
· 30 commits to main since this release

What's Changed

  • Keep datetime normalization compatible with dlt 1.31 while retaining support for
    older dlt releases.

  • Extract fragments of code into dedicated package [dlt-filesystem].

  • Notifications: a job report through Apprise when a run ends.
    --notify <apprise-url> and --notify-config <path-or-url>, both repeatable, send one
    short report per run, for its final outcome, to any target Apprise supports.
    Inline URLs receive every report, config entries route by their tags
    (success, failure, always), and --notify-on failure keeps a job quiet
    while it works. Every target is checked before the load starts, and a report
    that cannot be sent afterwards only logs a warning. The report names schemes,
    stages and exception types, never URIs, query text or error messages. Install
    omniload[notify]; run_ingest takes the same options as notify,
    notify_config and notify_on. See [Notifications].

  • Turso, read and written. turso:///path/to/app.db is an in-process file and
    turso://host?authToken=... a remote database, Turso Cloud or a self-hosted
    tursodb --sync-server (tls=false for plain HTTP), each a source and a
    destination through pyturso's own SQLAlchemy dialects
    (pip install 'omniload[turso]', not available on Windows). A remote database
    is read and written through a replica bootstrapped for the run and discarded
    after it; a destination run pushes the replica once the load has committed,
    and cannot be combined with --pipelines-dir. The destination
    writes to the main schema with append and replace only: a merge load into
    Turso deletes rows it should keep (tursodatabase/turso#6279), so merge,
    delete+insert, scd2 and a source's own merge resources are refused before
    anything is written, as is --full-refresh.

Details

  • fix(deps): update dependency stripe to v16 by @renovate[bot] in #387
  • Extract fragments of code into dedicated package dlt-filesystem by @amotl in #385
  • fix(deps): update dependency sqlalchemy-hana to v5 by @renovate[bot] in #391
  • fix(deps): update dependency confluent-kafka to <2.17 by @renovate[bot] in #393
  • fix(deps): update dependency dlt to >=1.22,<1.32 by @renovate[bot] in #394
  • fix(deps): update dependency pyarrow to v26 by @renovate[bot] in #397
  • Chore: Remove an unused ty: ignore in the AppsFlyer test by @hampsterx in #405
  • Chore: Fix broken link in Snowflake documentation page by @amotl in #406
  • fix(dlt): support dlt 1.31 datetime normalization by @hampsterx in #410
  • Chore: Cap SQLAlchemy below 2.0.55 and fix the Snowflake docs link by @hampsterx in #411
  • fix(dlt): parameterize incremental annotations for dlt 1.31 by @hampsterx in #412
  • Tests: Run MongoDB 8.2 so the container starts on current runners by @hampsterx in #413
  • chore(deps): update astral-sh/setup-uv action to v10.3.0 by @renovate[bot] in #399
  • Turso: Add a source and destination for local files and Turso Cloud by @hampsterx in #407
  • fix(deps): update dependency pyathena to v4 by @renovate[bot] in #401
  • fix(deps): update dependency google-ads to v34 by @renovate[bot] in #403
  • fix(deps): update dependency snowflake-connector-python to v5 by @renovate[bot] in #404
  • fix(deps): update dependency snowflake-sqlalchemy to v2 by @renovate[bot] in #409
  • fix(deps): update dependency ruff to ==0.17.* by @renovate[bot] in #400
  • Notifications: Send a job report through Apprise when a run ends by @hampsterx in #408
  • fix(deps): update dependency ty to v0.0.90 by @renovate[bot] in #415

Full Changelog: v0.19.0...v0.20.0