Repository navigation
v0.18.0
What's Changed
-
The filesystem and blob stack builds as its own distribution,
dlt-filesystem.
It moves topackages/dlt-filesystem/and declares its own dependencies, so a dlt
user can install the readers, the strict lister and the Arrowreadintoshim without
omniload's dependency list: 136 distributions against 250. Nothing an existing install
line resolves to changes.omniload,omniload[filesystem],omniload[iterable]and
omniload[full]each resolve to the same set as before, because the extras now forward
to the new distribution. The install hints a missing long-tail decoder prints name
dlt-filesystem[iterable], which is the extra that supplies it. -
Tests: the memory-store fixtures keep to their own store. Two test modules
subclassMemoryFileSystemto hold their fixtures apart from fsspec's
process-global store, butstoreis a class attribute the subclass inherits
rather than replaces, so theirclear()emptied the store the remote-filesystem
tests write into at import time. Each subclass now carries its own. -
Tests: the Arrow
readintoshim's write-handle test reads the instance. fsspec
2026.9.0 mirrorsreadintoontoArrowFilefor every mode, sohasattrno longer
separates what fsspec provides from what the shim injects. The shim itself is unchanged
and stays, because the supported floor isfsspec>=2024.6. -
Filesystem: Vortex files, read and write.
.vortex(and.vortex.gz) files
read on every filesystem source, and the localfile://destination writes them.
A remote or gzipped file is staged to a local temporary file first, because the
Vortex library opens files by path only. Opt-in through the newvortexextra, which needs Python 3.11 or
newer and is included infull.
Details
- fix(deps): update dependency lakefs-spec to <0.16 by @renovate[bot] in #365
- fix(deps): update dependency ty to v0.0.82 by @renovate[bot] in #366
- fix(deps): update dependency fsspec to >=2024.6,<2026.10 by @renovate[bot] in #367
- Chore: Format code. Satisfy linter. by @amotl in #370
- Tests: Pin the Arrow
readintocontract and isolate the memory-store fixtures by @hampsterx in #372 - chore(deps): update astral-sh/setup-uv action to v10.2.0 by @renovate[bot] in #374
- Filesystem: Build the package as its own distribution, dlt-filesystem by @hampsterx in #375
- Add Vortex file format by @amotl in #373
- fix(deps): update dependency ty to v0.0.83 by @renovate[bot] in #376
Full Changelog: v0.17.0...v0.18.0