Skip to content

[Feature Request] Expose full partition metadata list programmatically in DeltaTable #4737

@amtaftp

Description

@amtaftp

Feature request

Which Delta project/connector is this regarding?

  • Spark
  • Standalone
  • Flink
  • Kernel
  • Other (fill in here)

Overview

Delta Lake’s Python API currently surfaces partition column names via:

from delta.tables import DeltaTable
dt = DeltaTable.forName(spark, "tbl_sales_bronze")
dt.detail().show()

However, it lacks a method to list actual partition values (e.g. year=2025/month=06/day=11) programmatically via the API—instead relying on SQL (SHOW PARTITIONS) or a Spark DataFrame distinct query.

A full partition metadata listing ([{ 'year':'2025', 'month':'06', 'day':'11' }, …]) would be highly valuable.

Motivation

Use case in Microsoft Fabric Notebooks / Lakehouses: Fabric does not support SHOW PARTITIONS on OneLake-based Delta tables

Users need a reliable API method (e.g. DeltaTable.partitions()) to introspect partition metadata without executing Spark SQL queries.

Supports tooling and audit workflows that need to iterate through all partition combinations stored in a Delta Lake.

Reduces complexity and bridging between Python-only and SQL hybrid approaches.

Further details

The delta-rs Python API which is not supported by MS Fabric already implements:

dt = DeltaTable(...)
dt.partitions()  # -> list[dict[str, str]]
which returns a list of partition-value dictionaries (e.g., [{ 'year': '2020', 'month': '1', 'day': '1' }, …])

Willingness to contribute

The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?

  • Yes. I can contribute this feature independently.
  • Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
  • No. I cannot contribute this feature at this time.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions