v0.7.22
What's Changed
✨ Features
- feat(catalog): support Gravitino Iceberg REST catalogs by @kyo-tom in #7245
- feat(io): add HDFS support via OpenDAL services-hdfs by @Lucas61000 in #7202
- feat(io): export HdfsConfig from daft.io by @Lucas61000 in #7298
- feat(sql): support CREATE TABLE statement by @Lucas61000 in #7186
- feat(functions): add try_divide for PySpark parity by @BABTUNA in #7210
- feat(file): trace media file reads by @everettVT in #7312
- feat(file): trace video frame decoding by @everettVT in #7313
- feat(sql): support read_parquet ignore_corrupt_files by @jackylee-ch in #7133
- feat(sql): resolve read_iceberg io_config from table properties and default config by @jackylee-ch in #7273
- feat(sql): support ignore_corrupt_files in read_csv by @jackylee-ch in #7264
🐛 Bug Fixes
- fix(core): handle all-null list literals by @everySympathy in #7168
- fix(parquet): honor row-group constraints in count_rows() count pushdown by @FANNG1 in #7288
- fix: Gravitino catalog S3 credential key mismatch and _has_table error handling by @jiangxt2 in #7279
- fix(io): decode video with a single PyAV generator so .mkv/.webm yield frames by @AnayGarodia in #7229
- fix(io): support writing to hdfs:// URLs that include a port by @Lucas61000 in #7307
- fix(ai): route OpenAI chat completions parse through beta namespace by @colin-ho in #6995
- fix: truncate default IOConfig fields in explain() output by @shreynath in #7327
♻️ Refactor
- refactor(shuffle): migrate repartition onto shared backend and unify pipeline dispatch by @BABTUNA in #7275
- refactor(io): migrate Delta Lake scans to DataSource by @ihopenre-eng in #7300
- refactor(io): migrate Hudi scans to DataSource by @Lucas61000 in #7324
- refactor: migrate IcebergScanOperator to DataSource by @Lucas61000 in #7311
📖 Documentation
- docs: add motion trimming physical AI example by @ykdojo in #7341
- docs: remove dead HDF5 example notebook link by @everettVT in #7352
✅ Tests
- test(ai): mock beta chat completions parse by @everettVT in #7315
New Contributors
- @jiangxt2 made their first contribution in #7279
- @ihopenre-eng made their first contribution in #7300
- @shreynath made their first contribution in #7327
Full Changelog: v0.7.21...v0.7.22