-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
Feature request
Which Delta project/connector is this regarding?
- Spark
- Standalone
- Flink
- Kernel
- Other (fill in here)
Overview
Delta Lake’s Python API currently surfaces partition column names via:
from delta.tables import DeltaTable
dt = DeltaTable.forName(spark, "tbl_sales_bronze")
dt.detail().show()
However, it lacks a method to list actual partition values (e.g. year=2025/month=06/day=11) programmatically via the API—instead relying on SQL (SHOW PARTITIONS) or a Spark DataFrame distinct query.
A full partition metadata listing ([{ 'year':'2025', 'month':'06', 'day':'11' }, …]) would be highly valuable.
Motivation
Use case in Microsoft Fabric Notebooks / Lakehouses: Fabric does not support SHOW PARTITIONS on OneLake-based Delta tables
Users need a reliable API method (e.g. DeltaTable.partitions()) to introspect partition metadata without executing Spark SQL queries.
Supports tooling and audit workflows that need to iterate through all partition combinations stored in a Delta Lake.
Reduces complexity and bridging between Python-only and SQL hybrid approaches.
Further details
The delta-rs Python API which is not supported by MS Fabric already implements:
dt = DeltaTable(...)
dt.partitions() # -> list[dict[str, str]]
which returns a list of partition-value dictionaries (e.g., [{ 'year': '2020', 'month': '1', 'day': '1' }, …])
Willingness to contribute
The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?
- Yes. I can contribute this feature independently.
- Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
- No. I cannot contribute this feature at this time.