Skip to content

0.6.0

Latest

Choose a tag to compare

@yihua yihua released this 03 Nov 08:52
· 5 commits to main since this release
5381f85

What's changed

🚀 Features

  • Upgrade to arrow 59, DataFusion 55 and pyo3 0.29 with Lance kept on by default (#698) by @wirybeaver
  • (core) Read any metadata table partition as Arrow (#819) by @alitrack
  • (core) Add support for in and not in file pruning (#800) by @oglego
  • (core) Add append_only writes for append-only tables (#588) by @morningstarxcdcode
  • (core) Native table write support with spark storage parity (#666) by @vinothchandar
  • (core) Return the commit completion time from writes (#877) by @yihua
  • (core) Write composite and non-string record keys (#871) by @yihua
  • (python) Add write APIs to the Python binding (#878) by @yihua
  • (core) Read only the file groups the record index names for key filters (#907) by @yihua
  • (python) Expose commit completion time on write results (#887) by @yihua
  • (benchmark) Add a reproducible upsert and delete benchmark for hudi-rs writes (#880) by @yihua
  • (core) Add merge-on-read and metadata table compaction (#931) by @yihua

🐛 Bug Fixes

  • (reader) Read Avro enum columns and V2 Avro data log blocks (#803) by @yihua
  • (cpp) Catch panics at the FFI boundary so they surface as errors (#766) by @codope
  • (python) Run the DataFusion FFI provider on the binding's tokio runtime (#829) by @yihua
  • (reader) Map Avro local-timestamp logical types instead of panicking (#813) by @yihua
  • (hfile) Count checksum chunks over header + data (#836) by @linliu-code
  • (reader) Apply the logical timestamp repair to parquet log block records (#815) by @yihua
  • (core) Resolve storage options from the environment on table create (#875) by @yihua
  • (core) Accept list and fixed-size list columns on upsert and update (#868) by @yihua
  • (core) Follow-up fixes for native table writes and Java interop (#851) by @linliu-code
  • (python) Register HudiCoreError so callers can catch it (#915) by @yihua

🚜 Refactor

  • (ffi) Rename hudi-jvm-ffi to hudi-ffi (#847) by @yihua
  • Name the table create toggles *_enabled in rust and python (#927) by @yihua
  • (core) Return one WriteResult from every table write (#945) by @yihua
  • (python) Name the result instant requested_time and document the remaining stub items (#939) by @yihua

📚 Documentation

  • (core) Correct stale HFile reasoning in metadata-table reader routing (#723) by @yihua
  • (release) Correct the announce email list to users@hudi.apache.org (#769) by @yihua
  • (python) Add doctests to builder and config APIs, enable --doctest-modules (#623) by @GayathriSrividya
  • Add a python write quick start to the readme (#888) by @yihua
  • (rust) Document the user-facing API and build docs.rs with datafusion (#938) by @yihua
  • Add a feature overview, rust quick start, and doc links to the readme (#940) by @yihua
  • Add a context7.json for coding-agent doc indexes (#941) by @yihua

⚡ Performance

  • (core) Encode record index entries directly as Avro binary (#873) by @yihua
  • (core) Sort hfile entries by reference instead of cloning them (#894) by @yihua
  • (core) Read only the requested keys' record index shards and blocks (#869) by @yihua
  • (core) Write hfile structured keys straight into the block buffer (#912) by @yihua
  • (core) Merge only matched keys in event-time copy-on-write upserts (#882) by @yihua
  • (core) Evaluate string IN filters with a hash set (#901) by @yihua
  • (core) Compute the record index instant once per commit (#892) by @yihua
  • (core) Encode record index shards in parallel (#898) by @yihua
  • (core) Push read filters into the parquet reader (#911) by @yihua
  • (core) Decode table read file slices on runtime worker threads (#890) by @yihua
  • (core) Slice contiguous append chunks instead of copying them (#903) by @yihua
  • (core) Match record index keys in place before building records (#906) by @yihua
  • (core) Build commit sequence numbers without a per-row allocation (#896) by @yihua
  • (core) Merge copy-on-write file groups in parallel (#884) by @yihua
  • (core) Write the base file bloom filter only when asked (#933) by @yihua

🛠️ Build

  • (deps) Bump actions/setup-python from 6 to 7 (#731) by @dependabot[bot]
  • (deps) Update base64 requirement from 0.22 to 0.23 (#744) by @dependabot[bot]
  • (deps) Update serial_test requirement from 3 to 4 (#745) by @dependabot[bot]
  • (deps) Bump actions/checkout from 6 to 7 (#733) by @dependabot[bot]
  • (deps) Bump actions/setup-node from 6 to 7 (#728) by @dependabot[bot]
  • (deps) Bump apache/skywalking-eyes from 0.8.0 to 0.9.0 (#726) by @dependabot[bot]
  • (deps) Update comfy-table requirement from 7 to 8 (#742) by @dependabot[bot]
  • (deps-dev) Bump coverage from 7.16.0 to 7.16.2 in /python (#817) by @dependabot[bot]
  • (deps-dev) Bump mypy from 2.3.1 to 2.4.0 in /python (#818) by @dependabot[bot]
  • (deps) Bump actions/setup-python from 6 to 7 (#770) by @dependabot[bot]
  • (deps) Bump rocksdb to 0.24 for ubuntu-26.04 runners (#837) by @yihua
  • (deps-dev) Bump ruff from 0.15.18 to 0.17.1 in /python (#841) by @dependabot[bot]
  • (deps) Bump docker/setup-buildx-action from 4.3.0 to 4.4.1 (#806) by @dependabot[bot]
  • (deps) Bump swatinem/rust-cache from 2.9.2 to 2.9.3 (#842) by @dependabot[bot]
  • (deps) Bump astral-sh/setup-uv from 10.0.1 to 10.3.0 (#821) by @dependabot[bot]
  • (deps) Upgrade lance to 13 (#845) by @yihua
  • (deps) Zip 8 → 9.0.3 (#850) by @yihua
  • (ffi) Forward the spill-rocksdb feature through hudi-ffi (#849) by @yihua
  • Drop dependency debug info from dev builds (#928) by @yihua

⚙️ Miscellaneous Tasks

New Contributors