v0.8.0
What's Changed
- Core: Added real SCD2 loading when selecting the
scd2strategy.
Thanks, @hampsterx. - Filesystem: Added file-level incremental loading.
Thanks, @hampsterx. - Filesystem: Migrated local filesystem access to Apache Arrow.
- Filesystem: Added support for reading from Databricks, Dropbox,
FTP, Google Drive, HDFS, OCI, OneDrive, OSS, R2, SharePoint,
SMB, WebDAV, and WebHDFS. - Core: Applied a few bits of internal package refactoring.
Details
- Project: Migrate to src layout by @amotl in #227
- fix(deps): update dependency ty to v0.0.60 by @renovate[bot] in #228
- Refactoring: Initialize
dlt_filesystempackage inline by @amotl in #229 - Refactoring: Consolidate general filesystem information by @amotl in #230
- fix(incremental): perform a real SCD2 load for the scd2 strategy by @hampsterx in #235
- fix(deps): update dependency ty to v0.0.61 by @renovate[bot] in #237
- Filesystem: Migrate local filesystem access to Apache Arrow by @amotl in #238
- feat(filesystem): add file-level incremental loading by @hampsterx in #239
- fix(filesystem): always remove temporary pipeline state by @hampsterx in #243
- chore(deps): update astral-sh/setup-uv action to v9 by @renovate[bot] in #245
- fix(deps): update dependency ty to v0.0.62 by @renovate[bot] in #246
- Documentation: Fix broken links to Google Ads by @amotl in #248
- fix(deps): update dependency ty to v0.0.63 by @renovate[bot] in #250
- fix(deps): update dependency testcontainers to >=4.9.1,<4.16 by @renovate[bot] in #252
- Sandbox: Update to ruff 0.16.0 with old default configuration by @amotl in #253
- Filesystem integration tests by @amotl in #249
- Dependencies: Fix security warnings by updating flagged packages by @amotl in #255
- Filesystem: Add read support for more fsspec-based filesystems by @amotl in #254
Full Changelog: v0.7.0...v0.8.0