v0.6.3 · lakehq sail · Discussion #1953 · GitHub

linhr
May 21, 2026
Maintainer

Overview

You can find the release summary in the documentation.

The PySail Python package is available on PyPI.

What's Changed

perf: add SparkYear UDF with preimage for date-bucket filter pushdown by @davidlghellin in perf: add SparkYear UDF with preimage for date-bucket filter pushdown #1845
fix: truncate large gRPC error messages to prevent metadata size overflow by @pomykalakyle in fix: truncate large gRPC error messages to prevent metadata size overflow #1859
perf: implement preimage hook on floor for Parquet filter pushdown by @davidlghellin in perf: implement preimage hook on floor for Parquet filter pushdown #1843
perf: implement simplify hook on ceil and floor for integer identity and idempotence by @davidlghellin in perf: implement simplify hook on ceil and floor for integer identity and idempotence #1844
docs: add dbt integration guide by @pomykalakyle in docs: add dbt integration guide #1863
feat: support Variant type for Delta Lake and Iceberg & Iceberg V3 specs by @lonless9 in feat: support Variant type for Delta Lake and Iceberg & Iceberg V3 specs #1836
fix: assign unique iceberg field ids to nested columns by @Copilot in fix: assign unique iceberg field ids to nested columns #1871
fix: use correct timestamp datatype for to_timestamp functions by @Copilot in fix: use correct timestamp datatype for to_timestamp functions #1873
refactor: consolidate options resolution and move listing table modules by @codex in refactor: consolidate options resolution and move listing table modules #1869
refactor: split listing format read and write traits by @codex in refactor: split listing format read and write traits #1874
fix: resolve arrays_zip parity with Spark behavior by @Copilot in fix: resolve arrays_zip parity with Spark behavior #1866
fix: handle from_json/from_csv display names and schema parsing by @Copilot in fix: handle from_json/from_csv display names and schema parsing #1830
fix: correct Delta Lake partition handling with schema evolution and metadata updates by @lonless9 in fix: correct Delta Lake partition handling with schema evolution and metadata updates #1878
feat: improve row-level metrics for Delta Lake by @lonless9 in feat: improve row-level metrics for Delta Lake #1870
test: add snapshots for ClickBench query plans by @linhr in test: add snapshots for ClickBench query plans #1885
feat: support type widening for Delta Lake by @lonless9 in feat: support type widening for Delta Lake #1854
fix: implement try_encode_udwf / try_decode_udwf for SparkNtile in codec by @davidlghellin in fix: implement try_encode_udwf / try_decode_udwf for SparkNtile in codec #1884
deps(cargo): bump the all group with 11 updates by @dependabot[bot] in deps(cargo): bump the all group with 11 updates #1889
deps(npm): bump the all group with 7 updates by @dependabot[bot] in deps(npm): bump the all group with 7 updates #1888
fix: improve Spark string function parity by @Copilot in fix: improve Spark string function parity #1881
fix: type casting for ClickBench data by @linhr in fix: type casting for ClickBench data #1894
test: fix catalog pytest by @lonless9 in test: fix catalog pytest #1812
feat: add HMS table metadata parity by @santosh-d3vpl3x in feat: add HMS table metadata parity #1865
fix: sort transform parsing for Iceberg REST catalog by @Copilot in fix: sort transform parsing for Iceberg REST catalog #1867
feat: implement a provider-agnostic caching layer for catalog listings by @aminebizid in feat: implement a provider-agnostic caching layer for catalog listings #1886
fix: allow generated columns for datasource target & introduce snapshot/commit context by @lonless9 in fix: allow generated columns for datasource target & introduce snapshot/commit context #1855
feat: implement ALTER TABLE SET/UNSET TBLPROPERTIES for Iceberg by @lonless9 in feat: implement ALTER TABLE SET/UNSET TBLPROPERTIES for Iceberg #1822
feat: add variant_explode and variant_explode_outer functions by @twsl in feat: add variant_explode and variant_explode_outer functions #1829
fix: concat() with empty array and timestamp args by @davidlghellin in fix: concat() with empty array and timestamp args #1802
chore: update pyarrow version constraints in pyproject.toml by @Copilot in chore: update pyarrow version constraints in pyproject.toml #1902
feat: implement coalesce function with Spark-compatible coercion for mixed string and temporal arguments by @Copilot in feat: implement coalesce function with Spark-compatible coercion for mixed string and temporal arguments #1898
fix: support namespace aliases for database SQL by @Copilot in fix: support namespace aliases for database SQL #1904
feat: enhance coalesce function to support ANSI mode for string and date types by @lonless9 in feat: enhance coalesce function to support ANSI mode for string and date types #1906
test: fix issue with query result display by @linhr in test: fix issue with query result display #1907
refactor: unify try_to_number into to_number with safe mode flag by @davidlghellin in refactor: unify try_to_number into to_number with safe mode flag #1879
chore: remove prost recursion limit by @linhr in chore: remove prost recursion limit #1835
docs: remove partial notation from CREATE TABLE feature status by @Copilot in docs: remove partial notation from CREATE TABLE feature status #1912
feat: align abs UDF with Spark by @davidlghellin in feat: align abs UDF with Spark #1769
feat: implement json_tuple Spark function by @anarefolio in feat: implement json_tuple Spark function #1897
fix: interpret input in local time zone for to_timestamp_ntz by @linhr in fix: interpret input in local time zone for to_timestamp_ntz #1915
feat: support noop write format by @CoderHariswar in feat: support noop write format #1918
fix: Improve missing JDBC data source error by @alexy in fix: Improve missing JDBC data source error #1928
fix: fixing create() table for Nessie, adding test for it by @alexy in fix: fixing create() table for Nessie, adding test for it #1924
feat: remove Kubernetes worker pods on session end by @linhr in feat: remove Kubernetes worker pods on session end #1933
deps(cargo): bump the all group with 2 updates by @dependabot[bot] in deps(cargo): bump the all group with 2 updates #1931
docs: update supported features for Delta and Iceberg by @lonless9 in docs: update supported features for Delta and Iceberg #1926
feat: from_json builder approach by @kentkr in feat: from_json builder approach #1913
feat: time zone conversion by @linhr in feat: time zone conversion #1935
deps(npm): bump the all group with 8 updates by @dependabot[bot] in deps(npm): bump the all group with 8 updates #1930
fix: preserve managed catalog table type by @Copilot in fix: preserve managed catalog table type #1905
feat: improve Iceberg REST catalog support by @shehabgamin in feat: improve Iceberg REST catalog support #1920
fix: correct monotonic_id partitioning by @lonless9 in fix: correct monotonic_id partitioning #1936
fix: withMetadata should override metadata instead of merging by @Copilot in fix: withMetadata should override metadata instead of merging #1816
fix: minor catalog provider changes by @linhr in fix: minor catalog provider changes #1939
fix: cast output record batch to match schema in map partitions UDF by @shehabgamin in fix: cast output record batch to match schema in map partitions UDF #1949
chore: improve test setup in multi-platform build by @linhr in chore: improve test setup in multi-platform build #1951
chore: prepare v0.6.3 by @linhr in chore: prepare v0.6.3 #1940

New Contributors

@codex made their first contribution in refactor: consolidate options resolution and move listing table modules #1869
@aminebizid made their first contribution in feat: implement a provider-agnostic caching layer for catalog listings #1886
@anarefolio made their first contribution in feat: implement json_tuple Spark function #1897
@CoderHariswar made their first contribution in feat: support noop write format #1918
@alexy made their first contribution in fix: Improve missing JDBC data source error #1928

Full Changelog: v0.6.2...v0.6.3

This discussion was created from the release v0.6.3.

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment