Skip to content

[Bug] OBS HadoopCompliantFileIO does not override listFilesIterative #8959

Description

@thswlsqls

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version
master @ 526b91f (2.1-SNAPSHOT)

Compute Engine
Engine-agnostic (tables stored on Huawei Cloud OBS)

Minimal reproduce step
Read an object table (type = object-table) whose location is on an obs:// path, so org.apache.paimon.obs.HadoopCompliantFileIO (paimon-filesystems/paimon-obs-impl) serves the listing. It does not override FileIO.listFilesIterative(Path, boolean), so the default in paimon-common fs/FileIO.java (line 161) runs, walking the tree with one listStatus call per directory.

What doesn't meet your expectations?
The sibling implementations already override this method and delegate to FileSystem.listFiles(path, recursive): oss, s3, jindo (commit 0cc45257a), and core HadoopFileIO. OBS never uses the bulk listing Hadoop OBSA provides. (gs, azure and cosn have the same gap; this issue covers OBS only.)

Anything else?
The override keeps the returned set identical (files only). Two differences match core/oss/s3/jindo: iteration order becomes flat instead of BFS (unspecified by the contract), and FileNotFoundException for a missing path surfaces eagerly, not on first hasNext().

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions