Search before asking
Paimon version
master @ 526b91f (2.1-SNAPSHOT)
Compute Engine
Engine-agnostic (tables stored on Huawei Cloud OBS)
Minimal reproduce step
Read an object table (type = object-table) whose location is on an obs:// path, so org.apache.paimon.obs.HadoopCompliantFileIO (paimon-filesystems/paimon-obs-impl) serves the listing. It does not override FileIO.listFilesIterative(Path, boolean), so the default in paimon-common fs/FileIO.java (line 161) runs, walking the tree with one listStatus call per directory.
What doesn't meet your expectations?
The sibling implementations already override this method and delegate to FileSystem.listFiles(path, recursive): oss, s3, jindo (commit 0cc45257a), and core HadoopFileIO. OBS never uses the bulk listing Hadoop OBSA provides. (gs, azure and cosn have the same gap; this issue covers OBS only.)
Anything else?
The override keeps the returned set identical (files only). Two differences match core/oss/s3/jindo: iteration order becomes flat instead of BFS (unspecified by the contract), and FileNotFoundException for a missing path surfaces eagerly, not on first hasNext().
Are you willing to submit a PR?
Search before asking
Paimon version
master @ 526b91f (2.1-SNAPSHOT)
Compute Engine
Engine-agnostic (tables stored on Huawei Cloud OBS)
Minimal reproduce step
Read an object table (
type = object-table) whose location is on anobs://path, soorg.apache.paimon.obs.HadoopCompliantFileIO(paimon-filesystems/paimon-obs-impl) serves the listing. It does not overrideFileIO.listFilesIterative(Path, boolean), so the default in paimon-commonfs/FileIO.java(line 161) runs, walking the tree with onelistStatuscall per directory.What doesn't meet your expectations?
The sibling implementations already override this method and delegate to
FileSystem.listFiles(path, recursive):oss,s3,jindo(commit0cc45257a), and coreHadoopFileIO. OBS never uses the bulk listing Hadoop OBSA provides. (gs,azureandcosnhave the same gap; this issue covers OBS only.)Anything else?
The override keeps the returned set identical (files only). Two differences match core/oss/s3/jindo: iteration order becomes flat instead of BFS (unspecified by the contract), and
FileNotFoundExceptionfor a missing path surfaces eagerly, not on firsthasNext().Are you willing to submit a PR?