python-v1.6.4

What's Changed

• fix: speed up full vacuum listing on multi-level partitions (#4634) by @andrei-ionescu in #4635
• fix: make skip_stats snapshot updates replay safe by @ethan-tyler in #4660
• feat: add num_retries metric to write operation metrics by @vsmanish1772 in #3848
• fix: when merging on non-null field withe mergeSchema enabled, the pr… by @MedHeikelBouzayene in #4617
• refactor: use Snapshot for file discovery by @ethan-tyler in #4661
• feat: added function to retrieve a tuple of all table versions with commit_infos by @Nordalf in #3302
• feat: true DNF filters and SQL predicates for the file listing APIs [wip] by @FrankPortman in #4585
• feat(core)!: delete PartitionFilter in favor of kernel predicates by @FrankPortman in #4665
• chore(core): tighten the filter literal internals by @FrankPortman in #4670
• chore: address a number of latent deprecations by @rtyler in #4668
• fix(writer): implement MergeSchema for partitioned tables with RecordBatchWriter by @rtyler in #4669
• chore: regression test for partition pruning with type mismatches by @rtyler in #4671
• fix: enforce non-null map values and list elements in DataValidationExec by @raphaelroshan in #4561
• refactor: harden datafusion execution plan contracts by @ethan-tyler in #4674
• chore: demonstrate conditional put/conflict checker behavior with PySpark by @rtyler in #4673
• feat(python): add DeltaTable.scan for lazy engine-native reads by @FrankPortman in #4675
• refactor(python): drop file_pruning_predicate from the dataframe read methods by @FrankPortman in #4666
• feat(writer): optionally roll files on row-group boundaries by @djouallah in #4677
• chore: fix failing doc test and run doc tests in CI by @adamreeve in #4691
• refactor: use Snapshot in filesystem check, restore, and vacuum planning by @ethan-tyler in #4684
• fix(datafusion): mark num_rows inexact when deletion vectors are present by @Nordalf in #4681
• chore: modify log statements that might produce excess logs on large tables by @rtyler in #4698
• chore: add regression test for percent encoding as part of #4684 by @rtyler in #4697
• chore(deps): update brotli-decompressor requirement from =5.0.1 to =6.0.0 by @dependabot[bot] in #4699
• refactor: consolidate parquet read and write paths behind traits by @corwinjoy in #4568
• fix: preserve row order in deletion vector scans by @ethan-tyler in #4692
• fix(python): align DataFusion FFI integration with version 55 by @ethan-tyler in #4702
• fix: truncate timestamp min and max to millisecond precision by @adamreeve in #4690
• fix: reintroduce zero copy parquet uploads by @ion-elgreco in #4705
• chore: slimming some dependencies ahead of release by @rtyler in #4707
• feat: add column mapping create support by @ion-elgreco in #4505
• fix: keep non-skippable columns out of the kernel scan predicate by @adampolomski in #4693
• chore: add regression test for #3636 by @rtyler in #3638
• fix: include null constraint violations in invalid data preview by @mahesh-desu in #4721
• fix: pin aws-smithy-types to 1.6.3 to unblock builds by @mahesh-desu in #4728
• test: cover void type support in schema conversion and reads by @mahesh-desu in #4723
• perf: improve efficiency of batch partition splitting and preserve input order by @adamreeve in #4715
• feat(python): support RemoveAction in create_write_transaction by @seokjin0414 in #4713
• feat(python): allow passing datafusion session config in query builder by @ArulJerald in #4725
• docs: replace from_arrow in docstring by @jx2lee in #4733
• fix: chunk metadata-only scan batches within session batch size by @mahesh-desu in #4730
• chore: upgrade to the released buoyant_kernel 0.28 by @rtyler in #4712
• chore: bump python version by @ion-elgreco in #4741
• fix: unbounded upload memory by @ion-elgreco in #4708

New Contributors

• @MedHeikelBouzayene made their first contribution in #4617
• @raphaelroshan made their first contribution in #4561
• @djouallah made their first contribution in #4677
• @mahesh-desu made their first contribution in #4721
• @ArulJerald made their first contribution in #4725

Full Changelog: python-v1.6.3...python-v1.6.4