v0.5.3

Overview

You can find the release summary in the documentation.

The PySail Python package is available on PyPI.

What's Changed

• deps(npm): bump the all group with 2 updates by @dependabot[bot] in #1453
• deps(cargo): bump the all group with 5 updates by @dependabot[bot] in #1454
• fix: expand wildcard to column references for COUNT(DISTINCT *) by @davidlghellin in #1394
• chore: upgrade rand, rand_chacha, and rand_distr by @Copilot in #1457
• chore: upgrade kube and k8s-openapi by @Copilot in #1458
• chore: bdd style testing guidence for Copilot agent by @lonless9 in #1463
• chore: bump aws-lc-sys from 0.37.0 to 0.37.1 in the cargo group across 1 directory by @dependabot[bot] in #1467
• feat: emit execution metrics on a configured interval by @Copilot in #1465
• feat: support serialization for CatalogCommand by @Copilot in #1466
• feat: add the system table to expose AppConfig by @Copilot in #1464
• feat: manage functions in catalog manager by @linhr in #1471
• fix: distributed planner for SortPreservingMergeExec by @linhr in #1473
• chore: bump svgo from 4.0.0 to 4.0.1 in the npm_and_yarn group across 1 directory by @dependabot[bot] in #1475
• fix: update schema reference in pushdown filter for Delta Lake by @lonless9 in #1477
• fix: using SQL with Python data sources by @shehabgamin in #1469
• chore: bump MSRV to 1.91.0 by @Copilot in #1476
• refactor: align the log scan subtree with table provider scan for Delta Lake by @lonless9 in #1346
• fix: correct SQL syntax for CREATE DATABASE by @linhr in #1480
• feat: move catalog commands to physical plans by @Copilot in #1479
• fix: namespace handling for catalog providers by @linhr in #1484
• fix: truncate gRPC error messages to prevent RESOURCE_EXHAUSTED by @davidlghellin in #1488
• fix: support non-literal expressions in window RANGE frame boundaries by @davidlghellin in #1482
• fix: ensure CollectLeft HashJoinExec build side has single partition by @davidlghellin in #1483
• feat: resolve default table location using database location by @Copilot in #1492
• deps(npm): bump the all group with 6 updates by @dependabot[bot] in #1497
• deps(cargo): bump the all group with 10 updates by @dependabot[bot] in #1498
• refactor: move BDD steps to pysail.testing and remove DataFrame BDD tests by @Copilot in #1496
• refactor: restructure compatibility check utilities and tests by @Copilot in #1495
• refactor: internalization for Delta Lake by @lonless9 in #1461
• chore: store syrupy snapshots in snapshots directories instead of __snapshots__ by @Copilot in #1501
• chore: bump quinn-proto from 0.11.13 to 0.11.14 in the cargo group across 1 directory by @dependabot[bot] in #1503
• refactor: reorganize snapshot testing files by @Copilot in #1502
• fix: update snapshot file locations by @Copilot in #1505
• refactor: move test utilities to pysail.testing.spark.utils by @Copilot in #1506
• fix: move delta metadata pruning to a post-replay stats pipeline by @Copilot in #1504
• fix: use stable Delta log file schema for JSON-only log reads by @Copilot in #1500
• chore: cache mtime by @lonless9 in #1513
• feat: set up bdd testing & implement partition transforms for Iceberg by @lonless9 in #1183
• chore: update dependencies by @lonless9 in #1515
• chore: revert cache mtime by @lonless9 in #1516
• chore: bump lz4_flex from 0.12.0 to 0.12.1 in the cargo group across 1 directory by @dependabot[bot] in #1529
• deps(cargo): bump the all group with 4 updates by @dependabot[bot] in #1527
• feat: version checksum file write support for Delta Lake by @lonless9 in #1530
• fix: saveAsTable with mode="append" creates table when it does not exist by @Copilot in #1525
• feat: version checksum file read support for Delta Lake by @lonless9 in #1532
• fix: support df.write.format(...).option("path", path).save() by @Copilot in #1533
• fix: soundex Utf8View support, LargeUtf8 handling, and edge cases by @davidlghellin in #1517
• feat: add Spark 4.1 functions for TIME type by @tamirkifle in #1470
• feat: add CLI command to run PySpark script by @linhr in #1535
• test: add TPC-H and TPC-DS parity tests by @linhr in #1509
• fix: use structured partition fields for Iceberg by @lonless9 in #1512
• fix: array_min/max NULL input and float-to-Decimal128 overflow by @davidlghellin in #1536
• feat: streamline log segment resolution for Delta Lake by @lonless9 in #1534
• docs: agent skills and function support by @linhr in #1544
• chore: prepare v0.5.3 by @linhr in #1542
• chore: prepare v0.5.3 by @linhr in #1546
• chore: prepare v0.5.3 by @linhr in #1547

Full Changelog: v0.5.2...v0.5.3