v0.4.0

Source release

Source archive, signature and SHA-512 checksum

KEYS for signature verification

Go Linux runtime requirements

The prebuilt Go Linux amd64 library requires glibc 2.38 or newer and OpenSSL 3. The Go FFI dependency also requires libffi.so.8.

What's Changed

• chore: bump version to 0.4.0 by @XiaoHongbo-Hope in #607
• fix(table): short-circuit empty global index ranges by @QuakeWang in #606
• feat(python): add abort method to TableCommit for write rollback by @hechao-ustc in #612
• perf(table): search global index fields concurrently by @QuakeWang in #610
• feat(vector-search): use data conjuncts only for the PK-vector residual and pre-filter guard by @JunRuiLee in #609
• feat(c): export vector-search projection over the C FFI by @JunRuiLee in #608
• fix(table): validate write batch schema at the core entry by @liwuhen in #602
• feat(table): add primary-key hybrid (vector + full-text) search by @JunRuiLee in #568
• [spec] Support parsing REST catalog Predicate and Transform JSON by @plusplusjiajia in #500
• fix(index): sort search top-k results by relevance when not truncating by @XiaoHongbo-Hope in #614
• fix(datafusion): return vector_search rows in best-first relevance order by @XiaoHongbo-Hope in #613
• perf(arrow): evaluate In/NotIn with a hash set for byte, string, and integer columns by @jordepic in #619
• fix(spec): validate bucket-key against table schema by @jackylee-ch in #620
• feat(datafusion): support alter column by @QuakeWang in #625
• fix(table): compute COUNT(*) for format tables instead of returning 0 by @sundapeng in #624
• fix(spec): accept Java's long memory-size units and reject overflow by @jackylee-ch in #621
• feat: support batch incremental diff reads by @Pandas886 in #510
• feat(c): build a catalog-free Table from a resolved schema JSON by @JunRuiLee in #595
• feat(io): let FileIO wrap a caller-provided filesystem operator by @jordepic in #618
• fix(datafusion): apply dynamic options to vector search by @shyjsarah in #623
• feat(datafusion): support database SQL statements by @shyjsarah in #627
• fix(table): skip unrelated BLOB and vector files in row-range reads by @XiaoHongbo-Hope in #629
• chore: upgrade paimon-vindex-core to 0.3.0 by @JingsongLi in #630
• fix(table): align Data Evolution row-range pruning with Java by @jerry-024 in #628
• perf(parquet): read row groups concurrently by @XiaoHongbo-Hope in #633
• build(deps): bump lz4_flex from 0.11.6 to 0.13.1 by @dependabot[bot] in #657
• build(deps): bump astral-sh/setup-uv from 8.2.0 to 9.0.0 by @dependabot[bot] in #651
• build(deps): bump pypa/gh-action-pypi-publish from 1.13.0 to 1.14.2 by @dependabot[bot] in #650
• build(deps): bump taiki-e/install-action from 2.84.0 to 2.85.4 by @dependabot[bot] in #649
• fix(spec): rewrite field-scoped options on column rename by @u70b3 in #635
• fix(table): honor physical schemas in raw evolution reads by @JingsongLi in #636
• fix(datafusion): normalize Paimon column identifiers consistently by @QuakeWang in #634
• fix(spec): implement Display for BinaryTableStats by @u70b3 in #632
• perf(vindex): read search indexes through positional ranges by @JunRuiLee in #645
• fix(table): keep every split of a format table under a positive LIMIT by @sundapeng in #644
• fix(spec): reject reserved system field names at create and alter time by @jackylee-ch in #641
• docs: correct the local verification commands by @jackylee-ch in #637
• docs: correct three stale entries in the SQL option reference by @jackylee-ch in #639
• docs: correct three wrong default values in CoreOptions doc comments by @jackylee-ch in #640
• fix(datafusion): reduce scan planning peak memory by @XiaoHongbo-Hope in #666
• fix(spec): normalize file-format options to lowercase by @jackylee-ch in #663
• test(vindex): make executor cancellation test deterministic by @QuakeWang in #667
• docs: refresh the getting-started dependency and feature notes by @jackylee-ch in #664
• fix(spec): say precision, not length, in two type-parse errors by @jackylee-ch in #662
• docs: align the global index reference with the supported types and options by @jackylee-ch in #661
• fix(spec): validate final schema consistently by @QuakeWang in #660
• perf(vindex): execute compatible queries through native batch search by @JunRuiLee in #646
• perf(read): feed sorted runs into the PK merge LoserTree by @JunRuiLee in #668
• fix(spec): validate sequence.field against table schema by @jackylee-ch in #615
• perf(vector-search): execute PK-vector leaves concurrently by @JunRuiLee in #647
• referenced_files: merge scope results incrementally by @shyjsarah in #680
• feat(datafusion): allow configuring SQLContext runtime resources by @shyjsarah in #675
• feat(python): expose data evolution row ranges by @XiaoHongbo-Hope in #674
• fix(spec): reject changelog-producer on tables without primary keys by @jackylee-ch in #670
• test(vindex): make PK-vector concurrency tests deterministic by @QuakeWang in #685
• fix(spec): reject table-read.sequence-number.enabled without primary keys by @jackylee-ch in #684
• fix(python): declare abort and default_database in the type stub by @jackylee-ch in #683
• fix(spec): parse commit duration options and align their defaults with Java by @jackylee-ch in #671
• docs: document the $files and $branches system tables by @jackylee-ch in #669
• fix(scan): preserve partition group order by @XiaoHongbo-Hope in #692
• fix(arrow): list every writable format in the write-format error by @jackylee-ch in #690
• fix(scan): respect family-specific global-index search modes by @XiaoHongbo-Hope in #693
• fix(datafusion): show an unreported partition statistic as NULL, not as -1 by @sundapeng in #697
• fix(storage): retry temporary OSS failures by @XiaoHongbo-Hope in #695
• fix(arrow): enforce exact Mosaic predicate filtering by @QuakeWang in #694
• docs: fix Go predicate usage comments and document with_row_ranges by @jackylee-ch in #689
• test(spec): cover untested partial-update and blob-view error paths by @jackylee-ch in #688
• feat(table): support scan.watermark batch time travel by @u70b3 in #677
• fix(predicate): resolve _ROW_ID by name, not by its placeholder leaf index by @plusplusjiajia in #642
• [core] Reduce global index planning IO by @XiaoHongbo-Hope in #699
• fix(scan): prune primary-key files with key stats by @QuakeWang in #698
• perf(scan): prune index manifest entries by @QuakeWang in #703
• docs: correct five system-table column descriptions by @jackylee-ch in #701
• fix(table): list the supported formats when a format table is rejected by @jackylee-ch in #702
• perf(parquet): prune IN predicates with page stats by @QuakeWang in #704
• Support deletion vector merge-on-read by @JingsongLi in #706
• feat(vindex): build indexes from streamed data splits by @jerry-024 in #707
• fix(vindex): avoid full index reads during refine by @jerry-024 in #708
• fix(auth): Fail closed on query-auth tables outside the read boundary by @plusplusjiajia in #691
• fix(datafusion): normalize index_type in global index procedures by @jackylee-ch in #715
• fix(dlf): refresh expiring credentials by @XiaoHongbo-Hope in #714
• feat(io): support in-memory local cache by @jerry-024 in #710
• perf(arrow): prune IN predicates with row-group stats by @QuakeWang in #705
• fix(table): reject row ranges for format tables at read construction by @jackylee-ch in #700
• feat: rust examples for creating and querying Paimon tables by @ganesh-skumar in #648
• feat(write): add fixed-bucket write primitives for postpone tables by @XiaoHongbo-Hope in #659
• fix(python): Release GIL during catalog I/O by @XiaoHongbo-Hope in #716
• fix: avoid OSS multipart limit in Parquet writes by @jerry-024 in #713
• fix(file_index): align file index format with Java V1 by @QuakeWang in #719
• fix(scan): preserve Data Evolution file order in row-id groups by @XiaoHongbo-Hope in #717
• feat(go): add table write bindings by @XiaoHongbo-Hope in #658
• fix(data-evolution): require row tracking and row IDs by @XiaoHongbo-Hope in #718
• perf(vindex): size native batches by active indexes by @jerry-024 in #709
• feat: add vector search timing and I/O diagnostics by @jerry-024 in #711
• feat(vindex): add DiskANN and IVF-SQ/RQ support by @jerry-024 in #726
• perf(vindex): decouple range-read limits and remove chunk barrier by @jerry-024 in #720
• fix: serialize unbounded varchar as string by @JingsongLi in #730
• feat: add Java-compatible array predicate pushdown by @JingsongLi in #732
• [core] Support multivalue global index by @JingsongLi in #731
• fix(datafusion): surface tag create-time and retention in $tags by @jackylee-ch in #728
• fix(avro): read TIME, BLOB, MULTISET and non-string-key map columns by @jackylee-ch in #724
• perf(vindex): split build timing logs by phase by @jerry-024 in #723
• feat(go): add postpone fixed-bucket write bindings by @XiaoHongbo-Hope in #722
• feat(file_index): add predicate evaluation foundation by @QuakeWang in #721
• perf: vectorize raw vector search by @JingsongLi in #734
• fix(datafusion): make table definitions optional for queries by @jerry-024 in #735
• feat: route tables by declared TableType to pluggable engines by @plusplusjiajia in #733
• feat: expose point-query primitives in paimon by @JingsongLi in #739
• Upgrade paimon-vindex-core to 0.4.0 by @shyjsarah in #737
• fix: align manifest entry schema with Java Paimon by @shyjsarah in #742
• fix: write manifest _FILE as an Avro record by @shyjsarah in #743
• fix(table): accept DataSplit version 9 and mirror _WRITE_COLS_SEQUENCES by @JunRuiLee in #745
• feat(file_index): add Java-compatible Bloom filter index by @QuakeWang in #744
• fix: reject Paimon reads and writes on iceberg/object/lance tables by @plusplusjiajia in #740
• fix: satisfy Rust 1.98 chunks lint by @shyjsarah in #750
• perf(vindex): parallelize Parquet reads for index builds by @jerry-024 in #736
• feat(file_index): add Java-compatible Bitmap reader by @QuakeWang in #754
• fix(spec): widen index-manifest row count to i64 by @JunRuiLee in #751
• feat(table): decode the BucketVectorSearchSplit byte form by @JunRuiLee in #746
• fix(lumina): drop unfilled search slots and rank NaN scores last by @jackylee-ch in #725
• feat: add native object table read support by @shyjsarah in #749
• feat(blob): add standalone BlobDescriptor reader by @XiaoHongbo-Hope in #762
• feat(file_index): add Java-compatible Bitmap V2 writer by @QuakeWang in #759
• chore(deps): refresh locked arrow and opendal by @plusplusjiajia in #756
• fix(datafusion): keep metadata queries available without table engines by @shyjsarah in #747
• feat(blob): add seekable descriptor streams by @XiaoHongbo-Hope in #763
• feat(c): expose FileIO cache callbacks by @JingsongLi in #765
• feat(go): add BlobDescriptor batch and stream readers by @XiaoHongbo-Hope in #761
• feat: add Java-compatible FM global index by @JingsongLi in #766
• refactor: split global index implementations by @JingsongLi in #767
• feat(file_index): add indexer contracts and outer-format composition by @QuakeWang in #764
• vector-search: support scalar pre-filter before Top-K by @shyjsarah in #760
• refactor(vector-search): remove extra prefilter guards by @JingsongLi in #769
• fix(table): resolve index files by external path and bucket layout by @JunRuiLee in #752
• feat(table): plan a PK-vector search from a decoded bucket split by @JunRuiLee in #757
• datafusion: cast lateral vector search rows to provider schema by @shyjsarah in #781
• chore(deps): bump apache/skywalking-eyes from 0.8.0 to 0.9.0 by @dependabot[bot] in #772
• chore(deps): bump taiki-e/install-action from 2.85.4 to 2.87.1 by @dependabot[bot] in #778
• chore(deps): bump astral-sh/setup-uv from 9.0.0 to 10.0.1 by @dependabot[bot] in #779
• fix(spec): enforce the writer's layout in the string array decoder by @JunRuiLee in #770
• feat(blob): support MAP<K, BLOB> reads in Go by @XiaoHongbo-Hope in #784
• build: raise MSRV to Rust 1.94 by @XiaoHongbo-Hope in #788
• feat(datafusion): support the $consumers system table by @jerry-024 in #782
• build: prepare the 0.4.0 release by @XiaoHongbo-Hope in #790
• fix(spec): reserve 16 bytes for non-compact decimal in BinaryRow by @jackylee-ch in #791
• feat(file_index): integrate pruning into raw data-file reads by @QuakeWang in #783
• fix(variant): sort object keys by UTF-8 bytes like Java by @jackylee-ch in #789
• fix(spec): align index manifest field order with Java by @JingsongLi in #797
• fix(lumina): decide on the index size before parsing the metric by @JunRuiLee in #803
• fix(vindex): bound a deletion-vector position by its own source file by @JunRuiLee in #802
• fix(table): check batch arity instead of asserting it in debug builds by @JunRuiLee in #801
• fix(spec): accept the legacy first_not_null_value aggregate name by @jackylee-ch in #794
• fix(scan): prune NOT IN predicates using file stats by @jackylee-ch in #795
• fix(read): null-fill nested fields a data file predates by @JunRuiLee in #805
• fix(datafusion): keep format table partition columns out of decoder filters by @sundapeng in #811
• feat(rest): add partition registration, lookup and statistics APIs by @sundapeng in #812
• fix(table): skip staging files and list format table partitions concurrently by @sundapeng in #813
• fix(c): align append vector filter with core behavior by @QuakeWang in #793
• fix(file_index): skip pruning for narrowing integer schema changes by @QuakeWang in #806
• fix: resolve time-travel clauses instead of answering with current rows by @plusplusjiajia in #753
• feat(table): read the registered partitions of a catalog-managed format table by @sundapeng in #814
• feat(datafusion): add SHOW, ADD and DROP PARTITION for catalog-managed format tables by @sundapeng in #816
• feat(table): execute a primary-key vector search over engine-planned splits by @JunRuiLee in #771
• feat(datafusion): add MSCK REPAIR TABLE for catalog-managed format tables by @sundapeng in #817
• fix(rest): sign the query string the client actually sends by @plusplusjiajia in #821
• refactor(table): unify vector search scan and read APIs by @JingsongLi in #822
• feat(datafusion): support ANALYZE TABLE on catalog-managed format tables by @sundapeng in #815
• feat(rest): add permission management APIs by @plusplusjiajia in #820
• feat(catalog): add tag lifecycle APIs by @XiaoHongbo-Hope in #824
• feat(go): expose snapshot and tag APIs by @XiaoHongbo-Hope in #828
• fix(python): align native scan planning with PyPaimon by @JingsongLi in #831
• fix(python): rebuild the fallback runtime after fork by @JingsongLi in #832
• feat(rest): add policy management APIs by @plusplusjiajia in #829
• fix(core): align timestamp partition paths with Java by @JingsongLi in #833
• fix(core): preserve local timezone semantics for TIMESTAMP_LTZ schemas by @JingsongLi in #834
• feat(datafusion): support the $audit_log system table by @jerry-024 in #787
• fix(core): preserve partition-bucket order in scan planning by @JingsongLi in #838
• fix(core): align evolution stats and aggregation flush with Java by @JingsongLi in #839
• vindex: use automatic IVF search width by default by @shyjsarah in #840
• [core] Align incremental scans and version selection with Java by @JingsongLi in #841
• [core] Align dynamic PK events and first-row scans with Java by @JingsongLi in #845
• feat(io): support URI-aware FileIO providers by @JingsongLi in #846
• feat: expand native planning across resolved tables and scan modes by @JingsongLi in #847
• fix(go): prevent native crashes from concurrent record reader access by @XiaoHongbo-Hope in #844
• feat(file-index): read range bitmap indexes by @jianguotian in #849
• fix(spec): preserve manifest extra files by @jianguotian in #848
• fix(spec): decode _GLOBAL_INDEX fields by name, not by position by @jackylee-ch in #843
• fix(fm_index): store blocks in Java's codec envelope, not the SST one by @jackylee-ch in #826
• feat(go): support case-insensitive column names on reads by @jackylee-ch in #796
• fix(spec): encode TIME columns in the batch BinaryRow path by @jackylee-ch in #823
• fix(table): support all Java-comparable types in min/max aggregators by @jackylee-ch in #827
• fix(rest-server): use Java's uppercase resourceType in error responses by @jackylee-ch in #830
• fix(io): flush disk cache blocks before publishing by @QuakeWang in #836
• feat(scan): read bucket metadata for manifest pruning by @jianguotian in #850
• fix(parquet): reject unknown file.compression instead of ignoring it by @jackylee-ch in #853
• fix(write): avoid double separators in bucket paths by @shyjsarah in #860
• [Feature] Add manifest sidecar support by @JingsongLi in #861
• perf(mosaic): prefetch row groups with a Mosaic-owned budget by @jianguotian in #851
• perf(btree): optimize sparse IN queries by @JingsongLi in #868
• feat(btree): add optional bloom filters by @JingsongLi in #869
• fix(table): refuse row-level DML on a format table by @sundapeng in #866
• [python] Expose streaming native table reader by @JingsongLi in #870
• perf(spec): deserialize DataType by keyword instead of untagged trial by @sundapeng in #867
• fix(datafusion): truncate format tables instead of reporting success by @sundapeng in #863
• feat(file_index): generate indexes for append writes by @QuakeWang in #819
• fix(deletion_vector): read Java bitmap64 deletion vectors by @jackylee-ch in #785
• fix(python): render field types as SQL strings, not Rust Debug by @jackylee-ch in #786
• feat(python): configure native BLOB read parallelism by @JingsongLi in #871
• fix(arrow): map BLOB to LargeBinary by @JingsongLi in #874
• feat(mosaic): read row-group prefetch depth and byte budget from table options by @sundapeng in #864
• [rust] Add parquet.filter.columnindex.enabled option to control Page Index pruning by @XiaoHongbo-Hope in #875
• [rust] Coalesce inline BLOB reads with bounded amplification by @JingsongLi in #880
• Upgrade paimon-vindex-core to 0.5.0 by @jerry-024 in #879
• [rust] Bound sparse BlobDescriptor range reads by @JingsongLi in #883
• [rust] Map precision-zero timestamps to Arrow seconds by @JingsongLi in #884
• fix: make Python record batch reader cancellable by @JingsongLi in #885
• feat: expand Python native reads with nested projection and row kinds by @JingsongLi in #887
• [rust] Optimize scalar index reads and support BTree V2 by @JingsongLi in #888
• [python] Deserialize cross-language splits for native reads by @JingsongLi in #889
• feat(python): plan chunk-shuffled reads with IndexedSplit by @JingsongLi in #890
• perf(vindex): optimize IVF index build reads by @jerry-024 in #800
• Support append row-position selection in native scans by @JingsongLi in #893
• Expose combined changelog scans to Python by @JingsongLi in #894
• fix(datafusion): hand off the worker before blocking in catalog callbacks by @sundapeng in #891
• refactor(vindex): trim granule planning overhead by @jerry-024 in #892
• feat(read): bound data evolution BLOB reads by LIMIT by @JingsongLi in #896
• fix: remove manifest minor compaction by @XiaoHongbo-Hope in #899
• feat(read): preserve physical row IDs with data predicates by @JingsongLi in #900
• feat(read): resolve BLOB views lazily under LIMIT by @JingsongLi in #902
• feat(read): avoid unselected managed BLOB payloads under LIMIT by @JingsongLi in #901
• fix(table): reject duplicate ADD and DELETE file entries by @jerry-024 in #904
• Support PK partial-update and aggregation reads with deletion vectors by @JingsongLi in #905
• fix(read): stop BLOB predicate payload reads at LIMIT by @JingsongLi in #903
• Support BLOB and row-id predicates in Python native reads by @JingsongLi in #906
• Align native read schema evolution, ROW decoding, and merge semantics by @JingsongLi in #907
• fix(global-index): stop reading index keys with a comparator that can panic by @jackylee-ch in #909
• fix(lumina): reproduce Java's table, index, query option order by @jackylee-ch in #908
• fix(lumina): reject non-finite vector elements when building an index by @jackylee-ch in #898
• fix(lumina): store the canonical native metric name in index metadata by @jackylee-ch in #895
• fix(scan): keep the predicate when a partition set cannot express it by @jackylee-ch in #886
• fix(vortex): reject columns the writer cannot convert instead of panicking by @jackylee-ch in #878
• feat(file_index): support range bitmap append writes by @QuakeWang in #877
• test(file_index): verify Rust reads of Java-generated indexed tables by @liwuhen in #876
• feat(datafusion): add REST management procedures by @plusplusjiajia in #837
• perf(datafusion): push down partition-grouped COUNT(*) to manifests by @jerry-024 in #858
• fix(commit): align TableCommit validation and retries with Java by @JingsongLi in #913
• Add Java-compatible CommitMessage serialization and Python commit API by @JingsongLi in #912
• fix(commit): close remaining TableCommit parity gaps by @JingsongLi in #915
• [python] Preserve external batch commit identity by @JingsongLi in #916
• feat: Support Java-compatible scan.timestamp time travel by @JingsongLi in #919
• feat: integrate reader memory reservations with DataFusion by @JingsongLi in #914
• [rust] Account writer memory with ResourceContext by @JingsongLi in #928
• [rust] Reduce I/O requests for MAP BLOB descriptor reads by @XiaoHongbo-Hope in #920
• fix(spec): don't prune files that may hold NaN on NOT(= float) by @jackylee-ch in #926
• fix(datafusion): reject unknown arguments for tag and index procedures by @jackylee-ch in #917
• fix(python): declare read-scan methods in the datafusion type stub by @jackylee-ch in #918
• fix(variant): emit valid JSON for non-finite floats in to_json by @jackylee-ch in #927
• fix(arrow): scale INTEGER/BIGINT epoch seconds evolved to TIMESTAMP by @jackylee-ch in #924
• fix(c): guard predicate AND/OR/NOT against null inputs by @jackylee-ch in #923
• build: align ASF configuration with Paimon by @JingsongLi in #930
• fix(table): keep PK sequence numbers advancing across TableWrite reuse by @jackylee-ch in #925
• feat(c): expose reader memory reservations through ResourceContext by @QuakeWang in #931
• feat(c): attach ResourceContext to table writers by @QuakeWang in #933
• Honor data-file prefix in Python native writes by @JingsongLi in #932
• Give postpone writers distinct Java-style write IDs by @JingsongLi in #935
• [python] Construct native tables from resolved REST responses by @XiaoHongbo-Hope in #922
• Support more scalar types in incremental Diff by @JingsongLi in #940
• fix: write value stats for primary-key parquet files by @JingsongLi in #938
• Support nested values and deletion vectors in batch Diff by @JingsongLi in #944
• Support BSI FileIndex and indexed primary-key and data-evolution writes by @JingsongLi in #947
• Support native Format Table append and overwrite writes by @JingsongLi in #948
• Support Avro writes, defaults, and custom partition locations for native Format Tables by @JingsongLi in #950
• Support Mosaic writes for native Paimon tables by @JingsongLi in #952
• Support managed BLOBs in native primary-key Parquet tables by @JingsongLi in #954
• feat(go): expose shared resource reservations by @QuakeWang in #949
• test(file_index): verify Rust reads of Java-generated range-bitmap tables by @liwuhen in #953
• fix(variant): truncate decimals and floats in variant_get int casts by @jackylee-ch in #945
• docs(python-binding): auto incremental mode follows changelog-producer by @jackylee-ch in #943
• feat(datafusion): support the $aggregation_fields system table by @jackylee-ch in #937
• Align merge engines and all field aggregators with Java by @JingsongLi in #955
• Align native table writes and merge ordering with Java by @JingsongLi in #958
• Support nested ROW data evolution in native table reads and writes by @JingsongLi in #960
• Extend Format Table support to ORC, CSV, JSON, TEXT and Mosaic by @JingsongLi in #959
• Expose core table upsert and native updates to PyPaimon by @JingsongLi in #961
• fix(core): complete native update partition and deletion vector paths by @JingsongLi in #969
• fix(core): close gaps for PyPaimon native Parquet writers by @JingsongLi in #970
• [core] Complete nested updates and partition-aware temporal upserts by @JingsongLi in #973
• [core] Support configured data directories throughout table IO by @JingsongLi in #974
• [auth] Support local-file DLF token loading by @XiaoHongbo-Hope in #936
• [rust] Cache decoded BLOB indexes across readers by @XiaoHongbo-Hope in #897
• [rust] Cache parsed Parquet metadata across readers by @XiaoHongbo-Hope in #929
• [core] Honor external data paths across table writers by @JingsongLi in #981
• [core] Honor scan.manifest.parallelism during planning by @XiaoHongbo-Hope in #988
• [core] Reduce OSS requests for dense manifest sidecar selections by @JingsongLi in #990
• [core] Align dedicated Blob writes and URI resolution with Java by @JingsongLi in #989
• [rust] Reuse BTree data blocks across index lookups by @XiaoHongbo-Hope in #995
• [python] Add batch numeric extraction for Variant fields by @XiaoHongbo-Hope in #997
• feat(storage): support Jindo for OSS scan planning by @XiaoHongbo-Hope in #696
• feat(catalog): support tag creation with retention by @QuakeWang in #1000
• datafusion: support SQL execution options by @shyjsarah in #996
• [core] Unify the REST User-Agent format by @sundapeng in #992
• [core] Unify the object storage User-Agent format by @sundapeng in #991
• fix(variant): store shredded timestamps as microseconds by @jackylee-ch in #983
• feat(spec): support the Avro deflate codec by @jackylee-ch in #982
• fix(arrow): read CSV boolean columns like Java's CsvParser by @jackylee-ch in #980
• fix(arrow): convert dictionary and fixed-size-binary Arrow types by @jackylee-ch in #978
• feat(datafusion): support the $buckets system table by @jackylee-ch in #934
• feat(auth): authorize query-auth reads and carry the grant on the split by @plusplusjiajia in #758
• [io] Reduce BLOB footer and index reads with tail prefetch by @XiaoHongbo-Hope in #1010
• [jindo] Fix unsafe fallback after process fork by @XiaoHongbo-Hope in #999
• [core] Align FileType classification and mutability with Java by @sundapeng in #1013
• [rust] Parallelize OR global-index planning across files by @XiaoHongbo-Hope in #1011
• perf(btree): load null bitmap on demand by @QuakeWang in #1008
• fix(datafusion): map column-less MERGE INSERT VALUES by position by @jackylee-ch in #984
• fix(datafusion): guard ALTER TABLE DROP PARTITION against time travel by @jackylee-ch in #979
• fix(rest): preserve column errors through the REST catalog round trip by @jackylee-ch in #977
• fix(spec): reject non-primitive primary-key and partition-key types by @jackylee-ch in #976
• feat(btree): support composite global indexes by @JingsongLi in #1015
• [rust] Replace BLOB index entry limit with a memory budget by @XiaoHongbo-Hope in #998
• Align dynamic bucket writes and HASH index recovery with Java by @JingsongLi in #1016
• Extend native cross-partition and postpone writes by @JingsongLi in #1017
• Align postpone retract validation and managed Blob writes with Java by @JingsongLi in #1020
• Align MAP writes and Variant shredding factories with Java by @JingsongLi in #1027
• feat(io): add optional OSS C++ SDK read backend by @XiaoHongbo-Hope in #1028
• feat(c,go): list databases and tables through the C and Go bindings by @jackylee-ch in #1024
• feat(datafusion): add CALL sys.rename_tag by @jackylee-ch in #1023
• feat(go): expose string and range predicates through the Go binding by @jackylee-ch in #1022
• [rust] Parallelize compound AND global-index evaluation by @QuakeWang in #1012
• [python] Push batched Variant extraction into native reads by @XiaoHongbo-Hope in #1009
• fix(spec): match Java whitespace for CHAR/VARCHAR partition values by @jackylee-ch in #985
• fix(arrow): read ragged CSV format-table rows permissively by @jackylee-ch in #975
• fix(spec): reject a non-positive or non-integer bucket number by @jackylee-ch in #956
• feat(go): support a read-builder limit hint for split pruning by @jackylee-ch in #946
• feat(datafusion): support the rename_branch procedure by @jackylee-ch in #942
• Align row-tracking commit metadata and conflicts with Java by @JingsongLi in #1029
• Support Java-aligned Paimon branch writes by @JingsongLi in #1033
• Align inline Blob updates and historical layouts with Java by @JingsongLi in #1039
• Drive nested, MAP and VARIANT projections from structured read types by @JingsongLi in #1041
• [core] Write BLOB collections and read packed video files by @JingsongLi in #1044
• [core] Align commit failure cleanup with Java by @JingsongLi in #1045
• [core] Support Java Data Evolution ROW sidecars by @JingsongLi in #1047
• [core] Support row-tracking sequence projections and predicates by @JingsongLi in #1051
• [python] Discover an optionally packaged OSS C++ bridge by @XiaoHongbo-Hope in #1043
• [parquet] Support per-column dictionary encoding options by @XiaoHongbo-Hope in #1034
• [core] Apply primary-key BLOB filters and limits before payload resolution by @JingsongLi in #1053
• [core] Support column-subset append writes and upserts by @JingsongLi in #1054
• feat(table): build and drop full-text global indexes by @zhuxiangyi in #963
• perf(table): skip redundant BTree not-null queries by @QuakeWang in #1038
• [core] Support precomputed bucket writes and snapshot restoration by @JingsongLi in #1056
• [core] Support Java worker sharding in native scans by @JingsongLi in #1059
• [core] Preserve supplied fields across key upsert batches by @JingsongLi in #1061
• perf(arrow): evaluate integer IN with a bitmap or binary search by @sundapeng in #1058
• perf(arrow): read Variant metadata as a dictionary for field extraction by @sundapeng in #1057
• fix(arrow): map NULL to false in decimal residual leaf masks by @plusplusjiajia in #1030
• feat(predicate): read Java's string temporal and decimal literals in REST predicate JSON by @plusplusjiajia in #1018
• Support BlobConsumer for native append writes by @JingsongLi in #1063

New Contributors

• @jordepic made their first contribution in #619
• @jackylee-ch made their first contribution in #620
• @u70b3 made their first contribution in #635
• @ganesh-skumar made their first contribution in #648
• @jianguotian made their first contribution in #849
• @zhuxiangyi made their first contribution in #963

Full Changelog: v0.3.0...v0.4.0