Apache paimon-rust release 0.2.0

What's Changed

• Establish rust project layout by @Xuanwo in #1
• docs: Add ASF related files by @Xuanwo in #2
• ci: add asf license check by @QuakeWang in #4
• feat(spec): Add schema types by @Xuanwo in #5
• docs: Fix typo and add contribute section by @Xuanwo in #9
• docs: modify document specifications by @caicancai in #12
• Remove main branch protect by @JingsongLi in #8
• feat(spec): Add Snapshot by @QuakeWang in #11
• chore/docs: add cargo fmt by @caicancai in #13
• chore: Fix spec not exposed by @Xuanwo in #19
• feat(spec): Add DataFileMeta by @crrow in #15
• feat(spec): Add primitive data types by @XuQianJin-Stars in #18
• feat: Add file io by @Xuanwo in #23
• chore: Fix all reference links to tag instead by @Xuanwo in #24
• feat(spec): Add Manifest List by @Asura7969 in #17
• refactor: Refactor DateTypes to make it more easy to use by @Xuanwo in #29
• feat(spec): implement data field by @devillove084 in #21
• chore(deps): update typed-builder requirement from ^0.18 to ^0.19 by @dependabot[bot] in #28
• feat(spec): complete the Display and is_nullable for DataType by @Aitozi in #35
• refactor: Refactor error in DataType by @XuQianJin-Stars in #36
• ut: add test for Snapshot by @QuakeWang in #39
• feat: Implement serde for boolean and array by @Xuanwo in #48
• feat: Implement ser/de for mostly all types by @Xuanwo in #49
• feat(test): add data type fixtures by @devillove084 in #47
• feat: add schema change by @QuakeWang in #27
• test: Use test fixtures to test data type by @Xuanwo in #50
• ci: fix ci error by @QuakeWang in #53
• feat(io): Align FileStatus with the Java API. by @Aitozi in #45
• feat(test): add manifest file meta json fixtures by @devillove084 in #55
• feat(io): Implement base IO Module by @devillove084 in #51
• spec: Implement ManifestList read functionality by @Aitozi in #59
• feat(spec): Add CommitKind in snapshot by @Aitozi in #64
• chore(deps): update opendal requirement from 0.48 to 0.49 by @dependabot[bot] in #67
• docs: improve CONTRIBUTING.md by @XuQianJin-Stars in #65
• feat(file_index): add file index format read and write by @devillove084 in #63
• feat: Implement ser/de for the remaining all by @XuQianJin-Stars in #58
• feat(spec): Impl the ManifestFile read functionality by @Aitozi in #69
• feat: add devcontainer support by @zmlcc in #74
• Implement Index Manifest by @zmlcc in #76
• chore(deps):update rust version be 1.86.0 by @camilesing in #84
• ci: fix clippy warn error to long issue by @luoyuxia in #88
• chore: add issue and pr template by @luoyuxia in #91
• feat: introduce schema by @luoyuxia in #90
• feat: introduce catalog api by @luoyuxia in #92
• Introduce filesystem catalog by @luoyuxia in #93
• Introduce manifest by @luoyuxia in #94
• Introduce split & plan by @luoyuxia in #95
• feat: introduce snapshot manager by @luoyuxia in #103
• fix(io): return correct list_status paths and reuse storage operators by @QuakeWang in #101
• Introduce table scan by @luoyuxia in #96
• ci: introduce spark paimon integration for integration test by @luoyuxia in #107
• ci: introduce deny to check apache compatible licenses by @luoyuxia in #112
• feat: introduce basic deletion vector support by @luoyuxia in #110
• feat: introduce read builder by @luoyuxia in #114
• feat: support read to arrow by @luoyuxia in #116
• feat: plan support to plan deletion vector by @luoyuxia in #118
• feat: introduce c binding to expose ffi for go binding by @luoyuxia in #123
• feat: use bin pack to generate data splits by @luoyuxia in #135
• feat: introduce go binding by @luoyuxia in #132
• feat: add OSS storage support. by @QuakeWang in #134
• feat: implement BinaryRow deserialization for partition bytes by @QuakeWang in #133
• ci: add release pipeline for go binding by @luoyuxia in #138
• feat: support read deletion vector to arrow by @luoyuxia in #141
• feat(spec): implement partition path generation by @QuakeWang in #139
• feat: add REST API client with list_databases support by @discivigour in #144
• feat(io): support S3 file IO by @QuakeWang in #143
• feat: support datafusion integration by @luoyuxia in #150
• feat: support partitioned table in TableScan and TableRead by @QuakeWang in #145
• feat: implement REST API database and table CRUD operations with DLF authentication by @discivigour in #147
• feat(table): support column projection in ReadBuilder by @QuakeWang in #153
• fix: Adjust required_approving_review_count to zero by @JingsongLi in #158
• feat: Introduce docs for paimon-rust by @JingsongLi in #157
• feat(spec): define Predicate data structure and PredicateBuilder by @QuakeWang in #156
• fix: remove invalid gh-pages key from .asf.yaml and add CI validation by @XiaoHongbo-Hope in #159
• feat: introduce projection in go binding and datafusion by @luoyuxia in #162
• ci: trigger go release pipeline on release tag push by @luoyuxia in #166
• feat: implement RESTCatalog with database and table CRUD by @discivigour in #160
• ci: introduce release rust pipeline by @luoyuxia in #171
• feat(datafusion): parallelize paimon split execution by @QuakeWang in #169
• feat(table): support partition predicate pruning in TableScan by @QuakeWang in #167
• feat: introduce catalog factory to unify api for creating catalog by @luoyuxia in #175
• chore(deps): update roaring requirement from 0.10 to 0.11 by @dependabot[bot] in #186
• fix: remove protected_branches in asf.yaml by @JingsongLi in #187
• chore(deps): update snafu requirement from 0.8.3 to 0.9.0 by @dependabot[bot] in #184
• chore(deps): bump apache/skywalking-eyes from 0.6.0 to 0.8.0 by @dependabot[bot] in #178
• chore(deps): bump actions/download-artifact from 4 to 8 by @dependabot[bot] in #179
• chore(deps): bump actions/upload-artifact from 4 to 7 by @dependabot[bot] in #180
• chore(deps): bump actions/checkout from 4 to 6 by @dependabot[bot] in #181
• chore(deps): update serde_avro_fast requirement from 1.1.2 to 2.0.2 by @dependabot[bot] in #182
• feat: support catalog provider in datafusion by @luoyuxia in #176
• feat(io): refactor FileIO to reuse storage operators and improve path handling by @QuakeWang in #106
• feat: support data evolution table mode by @JingsongLi in #193
• feat: add paimon-rust website by @JingsongLi in #194
• feat(datafusion): support partition predicate pushdown by @QuakeWang in #190
• chore(api/auth): fix clippy failure for DLF default signer identifier by @freedom3219 in #191
• feat: support limit push down in datafusion by @luoyuxia in #177
• feat: introduce time travel for data fusion by @JingsongLi in #195
• feat(scan): support deletion vector cardinality by @luoyuxia in #200
• feat: support time travel by tag name and doc it by @JingsongLi in #199
• feat: support schema evolution read with SchemaManager by @JingsongLi in #197
• feat(scan): support data-level stats pruning in TableScan by @QuakeWang in #196
• feat: support complex types by @JingsongLi in #202
• feat: replace apache-avro with serde_avro_fast and parallelize manifest reads by @JingsongLi in #203
• chore: remove obsolete allow(dead_code) annotations by @luoyuxia in #206
• chore(provision): make Spark integration test provisioning idempotent by @freedom3219 in #198
• feat(scan): add bucket pruning, DV/postpone filtering, and DE group pruning by @JingsongLi in #205
• doc: add README for published crates by @luoyuxia in #209
• feat(python): introduce pypaimon core and DataFusion catalog integration by @luoyuxia in #204
• ci: add python release pipeline by @luoyuxia in #212
• feat: push down filters to parquet read path by @QuakeWang in #208
• fix(scan): harden time-travel selector validation by @QuakeWang in #219
• feat: Implement statistics for data fusion scan by @JingsongLi in #217
• feat: Support data evolution row id filter by @littlecoder04 in #222
• feat(go): add filter push-down and predicate API for Go binding by @luoyuxia in #216
• feat: abstract FormatFileReader and introduce ORC & avro readers by @JingsongLi in #225
• feat(btree): Add BTree global index reader with async on-demand block loading by @JingsongLi in #229
• feat(tantivy): Add Tantivy full-text search with on-demand archive reading by @JingsongLi in #231
• feat(table): Add commit pipeline with SnapshotCommit abstraction by @JingsongLi in #233
• add document for how to release and how to verify a release by @luoyuxia in #235
• perf(reader): Improve the performance of parquet reader by @discivigour in #230
• feat(write): add write pipeline with DataFusion INSERT INTO/OVERWRITE support by @JingsongLi in #234
• feat(datafusion): Add DDL support with PRIMARY KEY constraint syntax by @JingsongLi in #237
• feat(datafusion): upgrade to DataFusion 53 and use VERSION AS OF by @JingsongLi in #236
• feat: support hdfs using hdfs native by @SML0127 in #242
• feat(datafusion): Add $options system table by @plusplusjiajia in #240
• fix(docs): use correct crate name for integration tests. by @LinMingQiang in #243
• feat(write): add MERGE & UPDATE with DataEvolutionWriter by @JingsongLi in #241
• chore: update dependency list for release 0.1.0 by @luoyuxia in #244
• refactor(read): split ArrowReader into DataFileReader, DataEvolutionReader and TableRead by @JingsongLi in #246
• chore: bump version to 0.2.0 by @luoyuxia in #247
• feat: Add $schemas system table by @plusplusjiajia in #245
• feat(blob): add BlobType groundwork and preserve DDL semantics by @QuakeWang in #250
• feat: add primary-key table read/write support with sort-merge deduplication by @JingsongLi in #249
• feat(blob): add create-time schema contract checks by @QuakeWang in #251
• feat: add postpone bucket (bucket=-2) write support for primary-key tables by @JingsongLi in #252
• feat: add dynamic bucket assignment for primary-key tables by @JingsongLi in #254
• feat: support reading blob files in data evolution flow by @QuakeWang in #259
• remove stale raw_convertible builder calls by @liang767339 in #261
• feat(datafusion): support fetch contract in PaimonTableScan by @QuakeWang in #224
• feat: Enable exact COUNT(*) pushdown via partition statistics by @JingsongLi in #262
• ci: fix ci to use constant_time_eq 0.4.2 by @JingsongLi in #268
• feat: add Vortex columnar file format support as optional feature by @JingsongLi in #260
• feat(datafusion): add copy-on-write DML support for append-only tables by @JingsongLi in #269
• feat(blob): add blob descriptor write support for append-only tables by @JingsongLi in #270
• feat(datafusion): reject typed columns in PARTITIONED BY clause by @JingsongLi in #271
• fix(datafusion): add error logging to catalog methods that silently swallow errors by @shyjsarah in #273
• feat(datafusion): add session-scoped dynamic options via SET/RESET by @JingsongLi in #274
• fix: replace serde_avro_fast with apache-avro and optimize Avro reading by @luoyuxia in #276
• feat: add custom Avro OCF reader for manifest parsing with filtered decoding by @JingsongLi in #281
• feat(datafusion): Add $snapshots system table by @plusplusjiajia in #264
• feat(datafusion): Add $tags system table by @plusplusjiajia in #283
• feat(datafusion): add Hive-style INSERT OVERWRITE PARTITION support and simplify API by @JingsongLi in #280
• feat(datafusion): Add $manifests system table by @plusplusjiajia in #287
• feat: add Lumina vector index read infrastructure by @XiaoHongbo-Hope in #267
• [doc] Document full_text_search and vector_search for datafusion by @JingsongLi in #289
• feat(datafusion): add TRUNCATE TABLE and DROP PARTITION SQL support by @JingsongLi in #292
• feat(datafusion): add CALL procedure support for tag and rollback operations by @JingsongLi in #291
• feat(datafusion): support branch_manager and system table by @xuzifu666 in #290
• feat(datafusion): rename PaimonSqlHandler to SQLContext with multi-catalog support by @JingsongLi in #293
• feat(datafusion): add catalog-level temporary table support by @JingsongLi in #309
• fix(rest): pass catalog options to FileIO when no data token is vended by @plusplusjiajia in #310
• chore(deps): bump actions/download-artifact from 4 to 8 by @dependabot[bot] in #300
• chore(deps): bump softprops/action-gh-release from 2 to 3 by @dependabot[bot] in #299
• chore(deps): bump actions/cache from 4 to 5 by @dependabot[bot] in #298
• chore(deps): bump pypa/gh-action-pypi-publish from 1.13.0 to 1.14.0 by @dependabot[bot] in #297
• chore(deps): bump actions/upload-artifact from 4 to 7 by @dependabot[bot] in #296
• fix(datafusion): wrap system table scan IO in await_with_runtime to support FFI by @plusplusjiajia in #295
• feat(spec, api, catalog): add Catalog::list_partitions[_paged] for RESTCatalog by @plusplusjiajia in #288
• ci: cherry-pick CI fixes from release-0.1 to main by @luoyuxia in #311
• feat(datafusion): populate partition stats in $manifests system table by @QuakeWang in #308
• feat: support fixed-bucket partial-update merge engine by @QuakeWang in #263
• fix: validate data evolution matched row ids by @QuakeWang in #312
• fix: repair CI after partial-update merge by @QuakeWang in #314
• [fix] register catalog first check default db is exist and support information schema by @jerry-024 in #315
• feat: support dynamic-bucket partial-update by @QuakeWang in #317
• Add explicit read-only permissions to CI workflow by @arpitjain099 in #316
• docs: update releases page for 0.1.0 and upcoming 0.2.0 by @luoyuxia in #320
• ci: trigger CI for release branch pushes and PRs by @luoyuxia in #319
• feat(datafusion): Add $partitions system table by @plusplusjiajia in #294
• feat: add referenced_files_size and physical_files_size table functions by @JingsongLi in #323
• refactor(datafusion): convert files_size from table functions to system tables by @JingsongLi in #325
• test(datafusion): add Vortex SQL E2E test by @liujiwen-up in #321
• feat(datafusion): auto-register built-in table functions on catalog registration by @shyjsarah in #324
• feat(datafusion): Add $table_indexes system table by @jerry-024 in #327
• feat(datafusion): Add $files system table by @jerry-024 in #328
• chore: update dependency list for release 0.2.0 by @XiaoHongbo-Hope in #332
• docs: fix Python package name in RC verification guide by @luoyuxia in #331
• ci: revert pypa/gh-action-pypi-publish to v1.13.0 by @luoyuxia in #338

New Contributors

• @Xuanwo made their first contribution in #1
• @caicancai made their first contribution in #12
• @JingsongLi made their first contribution in #8
• @crrow made their first contribution in #15
• @XuQianJin-Stars made their first contribution in #18
• @Asura7969 made their first contribution in #17
• @devillove084 made their first contribution in #21
• @dependabot[bot] made their first contribution in #28
• @Aitozi made their first contribution in #35
• @zmlcc made their first contribution in #74
• @camilesing made their first contribution in #84
• @discivigour made their first contribution in #144
• @XiaoHongbo-Hope made their first contribution in #159
• @freedom3219 made their first contribution in #191
• @littlecoder04 made their first contribution in #222
• @plusplusjiajia made their first contribution in #240
• @LinMingQiang made their first contribution in #243
• @liang767339 made their first contribution in #261
• @xuzifu666 made their first contribution in #290
• @arpitjain099 made their first contribution in #316

Full Changelog: https://github.com/apache/paimon-rust/commits/v0.2.0