Skip to content

v0.8.0

Choose a tag to compare

@amotl amotl released this 26 Jul 22:47

What's Changed

  • Core: Added real SCD2 loading when selecting the scd2 strategy.
    Thanks, @hampsterx.
  • Filesystem: Added file-level incremental loading.
    Thanks, @hampsterx.
  • Filesystem: Migrated local filesystem access to Apache Arrow.
  • Filesystem: Added support for reading from Databricks, Dropbox,
    FTP, Google Drive, HDFS, OCI, OneDrive, OSS, R2, SharePoint,
    SMB, WebDAV, and WebHDFS.
  • Core: Applied a few bits of internal package refactoring.

Details

  • Project: Migrate to src layout by @amotl in #227
  • fix(deps): update dependency ty to v0.0.60 by @renovate[bot] in #228
  • Refactoring: Initialize dlt_filesystem package inline by @amotl in #229
  • Refactoring: Consolidate general filesystem information by @amotl in #230
  • fix(incremental): perform a real SCD2 load for the scd2 strategy by @hampsterx in #235
  • fix(deps): update dependency ty to v0.0.61 by @renovate[bot] in #237
  • Filesystem: Migrate local filesystem access to Apache Arrow by @amotl in #238
  • feat(filesystem): add file-level incremental loading by @hampsterx in #239
  • fix(filesystem): always remove temporary pipeline state by @hampsterx in #243
  • chore(deps): update astral-sh/setup-uv action to v9 by @renovate[bot] in #245
  • fix(deps): update dependency ty to v0.0.62 by @renovate[bot] in #246
  • Documentation: Fix broken links to Google Ads by @amotl in #248
  • fix(deps): update dependency ty to v0.0.63 by @renovate[bot] in #250
  • fix(deps): update dependency testcontainers to >=4.9.1,<4.16 by @renovate[bot] in #252
  • Sandbox: Update to ruff 0.16.0 with old default configuration by @amotl in #253
  • Filesystem integration tests by @amotl in #249
  • Dependencies: Fix security warnings by updating flagged packages by @amotl in #255
  • Filesystem: Add read support for more fsspec-based filesystems by @amotl in #254

Full Changelog: v0.7.0...v0.8.0