1 Release published by 1 person

• v4.0.0 Delta Lake 4.0.0

  published Jun 9, 2025

25 Pull requests merged by 15 people

• [Kernel] [Refactor] Refactor TestUtils to provide reading APIs using both legacy (Snapshot) and new (ResolvedTable) APIs

  #4676 merged Jun 14, 2025

• [Kernel] [CatalogManaged] Add the catalog managed preview table feature (read) support

  #4686 merged Jun 14, 2025

• [4.0][INFRA] Update integration tests to clear all possible ivy caches + minor update to S3LogStoreUtilIntegrationTest

  #4767 merged Jun 14, 2025

• [Delta] Migrate DeltaLogMinorCompactionSuite to CatalogOwned

  #4751 merged Jun 13, 2025

• [Docs] Cherry-pick 4.0 docs changes to master branch

  #4749 merged Jun 13, 2025

• [INFRA] Remove DSMP CI job as it no longer works (Spark 4.0 is no longer compatible with spark master)

  #4747 merged Jun 13, 2025

• [INFRA] Update integration tests to clear all possible ivy caches + minor update to S3LogStoreUtilIntegrationTest

  #3258 merged Jun 13, 2025

• [4.0] Update the docs for the scala version upgrade + missing Flink/Standalone projects

  #4748 merged Jun 13, 2025

• [Infra] Cherry-pick Add an additional back up repo to avoid hitting rate limits for maven deps (#4660) for master

  #4750 merged Jun 13, 2025

• [Delta] Refactor & Migrate DeltaLog related suites to CatalogOwned

  #4746 merged Jun 13, 2025

• [Spark] Refactor convertSnapshot to isolate snapshot expiry logic

  #4758 merged Jun 13, 2025

• [BUILD] Don't build javadoc for iceberg jars.

  #4721 merged Jun 13, 2025

• [Kernel][#1] Implement table features and tests for iceberg writer compat v3

  #4733 merged Jun 13, 2025

• refactor iceberg compat to prepare for newer versions

  #4736 merged Jun 12, 2025

• [Kernel][#3][test-only] IcebergCompatV3 - refactor IcebergCompatMetadataValidatorAndUpdaterSuites

  #4739 merged Jun 12, 2025

• [Spark] Rename DeltaTransaction to TransactionHelper

  #4741 merged Jun 12, 2025

• [Spark] Make DeltaDMLTestUtils agnostic to name/path-based access

  #4738 merged Jun 12, 2025

• Revert "[Delta] Support absolute path in Clone"

  #4740 merged Jun 12, 2025

• [Spark] Some refactoring to clean up OptimisticTransaction

  #4735 merged Jun 11, 2025

• [Kernel][#2] IcebergCompatV3 - move IcebergCompatCheck to base class

  #4732 merged Jun 11, 2025

• [Kernel][#1] IcebergCompatV3 - add icebergCompatV3 table feature

  #4731 merged Jun 10, 2025

• [Spark] Add more Unity Catalog integration tests

  #4626 merged Jun 10, 2025

• [Delta] Support absolute path in Clone

  #4729 merged Jun 9, 2025

• [Spark] Fixes options-based time travel with timestamps

  #4706 merged Jun 9, 2025

• Fix import format

  #4720 merged Jun 9, 2025

19 Pull requests opened by 11 people

• [Protocol Change Request] Geospatial types

  #4726 opened Jun 9, 2025

• [POC] [Kernel] Support for V3 deletion vectors

  #4727 opened Jun 9, 2025

• Introduce WriterCompatV3

  #4730 opened Jun 9, 2025

• [Kernel][#2] Refactor WriterCompat metadata validator and updater

  #4734 opened Jun 10, 2025

• [Kernel] Update max id to use SchemaIterable

  #4742 opened Jun 12, 2025

• [Kernel] use Schema iterable for assigning IDs

  #4743 opened Jun 12, 2025

• [WIP] [Kernel] [CatalogManaged] InMemoryCatalogManagedClient and Test Setup

  #4745 opened Jun 12, 2025

• [Kernel] Use schema iterable for table feature checks

  #4752 opened Jun 12, 2025

• [Kernel] Update validators to use SchemaIterable

  #4753 opened Jun 12, 2025

• [Kernel] Remove filterRecursively

  #4754 opened Jun 12, 2025

• [SPARK] Disable flaky test.

  #4756 opened Jun 12, 2025

• [Feature Request][Spark] Add test coverage for clustering on generate…

  #4757 opened Jun 12, 2025

• Fix wrong schema evolution inside structs in MERGE

  #4759 opened Jun 13, 2025

• [Spark] Update the comparison as it could be same but not not greater

  #4760 opened Jun 13, 2025

• [WIP] [Kernel] [CatalogManaged] Simple E2E read suite with real table and real staged commits

  #4761 opened Jun 13, 2025

• prepare to sync delta commit version with iceberg snapshot sequence number

  #4762 opened Jun 13, 2025

• [Spark][ICT] Bug Fix: Avoid accidentally dropping enablement info during REPLACE on ICT tables

  #4766 opened Jun 13, 2025

• [Kernel] [CatalogManaged] Update LogSegment construction logic to prioritize catalog commits over published deltas

  #4768 opened Jun 14, 2025

• [WIP][Delta][Spark] Additional cases to validate Schema Evolution

  #4769 opened Jun 15, 2025

2 Issues closed by 1 person

• Support for pyspark4

  #4631 closed Jun 9, 2025

• Version 4.0.0 to maven central

  #4692 closed Jun 9, 2025

8 Issues opened by 6 people

• [Kernel] [CatalogManaged] Update ResolvedTableBuilder to accept other types of `ParsedLogData`s, not only Staged Ratified Commits

  #4765 opened Jun 13, 2025

• [Kernel] [CatalogManaged] Update catalogOwned-preview test to use catalog-managed writer APIs

  #4764 opened Jun 13, 2025

• [Kernel] [CatalogManaged] Update table feature support to include `catalogManaged` when RFC accepted

  #4763 opened Jun 13, 2025

• [BUG] `delta.exceptions` module is missing in 4.0.0

  #4755 opened Jun 12, 2025

• [Feature Request][Kernel] Consolidate recursive code to SchemaIterable where appropriate.

  #4744 opened Jun 12, 2025

• [Feature Request] Expose full partition metadata list programmatically in DeltaTable

  #4737 opened Jun 11, 2025

• [PROTOCOL RFC] Clarify non-nullable field requirements for writers

  #4728 opened Jun 9, 2025

• [PROTOCOL RFC] Geospatial support

  #4725 opened Jun 9, 2025

20 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [Kernel] Introduce IcebergCompatV3

  #4595 commented on Jun 14, 2025 • 29 new comments

• [Kernel] Expose the clustering columns in txn report returned by the txn commit status

  #4688 commented on Jun 13, 2025 • 7 new comments

• [WIP][Kernel] Support enabling RowTracking

  #4582 commented on Jun 13, 2025 • 5 new comments

• [Kernel] [CatalogManaged] More Builder validation and tests

  #4664 commented on Jun 14, 2025 • 4 new comments

• [PROTOCOL RFC][FOLLOW-UP] Variant Shredding

  #4714 commented on Jun 13, 2025 • 1 new comment

• [Kernel] Introduce PeekableIterator class

  #4667 commented on Jun 13, 2025 • 1 new comment

• [Kernel] Add isNested method to DataType class

  #4722 commented on Jun 13, 2025 • 0 new comments

• [WIP] [Kernel] [CatalogManaged] ResolvedTable Scan implementation + tests

  #4663 commented on Jun 13, 2025 • 0 new comments

• [VARIANT] Add automaticallyUpdateProtocolOfExistingTables to variantType features

  #4600 commented on Jun 10, 2025 • 0 new comments

• [SPARK] Allow providing schema when creating a source

  #4550 commented on Jun 9, 2025 • 0 new comments

• [Spark] Make MERGE source checkpoint materialization a SQL operation

  #4390 commented on Jun 12, 2025 • 0 new comments

• [WIP][Kernel] add dvs as a part of DataFileStatus and then use it during append actions

  #4376 commented on Jun 9, 2025 • 0 new comments

• [SPARK] Include notMatchedBySource conditions for filtering files for merge

  #4236 commented on Jun 13, 2025 • 0 new comments

• [Spark] Fix metadata cleanup by retaining files required for log reconstruction

  #4146 commented on Jun 11, 2025 • 0 new comments

• [Kernel] Implement catalogManaged table feature in Kernel

  #4573 commented on Jun 14, 2025 • 0 new comments

• [BUG][Spark] Syntax Error with Convert to Delta with a Partitioned By on Timestamp Column, works on date

  #4441 commented on Jun 13, 2025 • 0 new comments

• [Feature Request][Spark] Add test coverage for clustering on generated columns

  #3248 commented on Jun 11, 2025 • 0 new comments

• [BUG] Third delta write failing for partition column position

  #3738 commented on Jun 10, 2025 • 0 new comments

• [BUG] `Table {name} does not support truncate in batch mode` in pyspark 3.5.6 when append after overwrite

  #4671 commented on Jun 10, 2025 • 0 new comments

• [Feature Request] [Spark] Upgrade Delta Spark to DSv2, built on Kernel, with deep CatalogManaged table support

  #4718 commented on Jun 10, 2025 • 0 new comments