Skip to content

Remove partitioned table read restriction in TableRead #129

@luoyuxia

Description

@luoyuxia

Parent Issue

Part of #124 (support partitioned table)
Depends on #128 (fix TableScan partition paths)

Background

TableRead::to_arrow() explicitly rejects partitioned tables:

// read_builder.rs:88-95
if !self.table.schema.partition_keys().is_empty() {
    return Err(Error::Unsupported {
        message: format!(
            "Reading partitioned tables is not yet supported. Partition keys: {:?}",
            self.table.schema.partition_keys()
        ),
    });
}

Once TableScan generates correct partition paths (#128), the ArrowReader will be able to read data files under partition directories without additional changes (since it reads from the bucket_path in DataSplit).

What needs to be done

  1. Remove the partition key check in TableRead::to_arrow() (lines 88-95 of read_builder.rs)
  2. Verify ArrowReader works with the corrected paths — it reads {bucket_path}/{file_name}, so as long as bucket_path is correct, no ArrowReader changes should be needed

Affected files

  • crates/paimon/src/table/read_builder.rsTableRead::to_arrow() method

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions