Skip to content

feat(python/sedonadb): Support specifying layer name and accessing sub-paths inside archives when working with pyogrio sources.#778

Merged
paleolimbot merged 4 commits intoapache:mainfrom
pratheekrebala:passthrough-pyogrio-opts
Apr 23, 2026
Merged

feat(python/sedonadb): Support specifying layer name and accessing sub-paths inside archives when working with pyogrio sources.#778
paleolimbot merged 4 commits intoapache:mainfrom
pratheekrebala:passthrough-pyogrio-opts

Conversation

@pratheekrebala
Copy link
Copy Markdown
Contributor

Summary

  • Enables passing pyogrio keyword arguments (e.g., layer, where, sql, max_features) through read_pyogrio(options=...) by switching from the internal pyogrio.raw.ogr_open_arrow() to the public pyogrio.raw.open_arrow() API, which accepts these as keyword arguments directly including specifying the layer= flag to handle multi-layer files (python/sedonadb: No mechanism to pass layer parameter to read_pyogrio #772)

  • Adds a path_suffix option to append a subpath to the resolved GDAL source, enabling reading of formats like FileGeoDatabase (.gdb) stored inside .zip archives (e.g., options={"path_suffix": 'subdir/test.gdb')). This is equivalent to running ogrinfo /vsizip/test.zip/subdir/test.gdb.

Motivation

Previously, read_pyogrio() had no way to pass options through to pyogrio, making it impossible to select a specific layer in a multi-layer file or use GDAL driver-specific open options. For example, reading a named layer from a GeoPackage now works:

con.read_pyogrio("file.gpkg", options={"layer": "my_layer"})

For formats like FileGeoDatabase distributed inside .zip archives, GDAL expects a path of the form /vsizip/archive.zip/data.gdb. Because the file listing infrastructure resolves real filesystem paths before the Python reader runs, passing a full /vsizip/... path directly fails. The path_suffix option bridges this gap:

con.read_pyogrio("path/to/file.zip", options={"path_suffix": "data.gdb"})

Copy link
Copy Markdown
Member

@paleolimbot paleolimbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

In CI I see:

        # column comparison
>       assert_index_equal(
            left.columns, right.columns, exact=check_column_type, obj="GeoDataFrame.columns"
        )
E       AssertionError: GeoDataFrame.columns are different
E       
E       GeoDataFrame.columns values are different (50.0 %)
E       [left]:  Index(['val', 'geom'], dtype='str')
E       [right]: Index(['val', 'wkb_geometry'], dtype='str')

and left a few suggestions for how to fix the test. I'm never quite sure what the name of a geometry column will be when I write or read a GeoDataFrame (everybody chooses something different for the "no geometry column name" case).

Comment thread python/sedonadb/tests/io/test_pyogrio.py Outdated
Comment thread python/sedonadb/tests/io/test_pyogrio.py Outdated
Co-authored-by: Dewey Dunnington <dewey@dunnington.ca>
Copy link
Copy Markdown
Member

@paleolimbot paleolimbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@paleolimbot paleolimbot merged commit a25a370 into apache:main Apr 23, 2026
5 checks passed
pwrliang pushed a commit to wherobots/sedona-db-gpu-tester that referenced this pull request Apr 24, 2026
…b-paths inside archives when working with pyogrio sources. (apache#778)

Co-authored-by: Dewey Dunnington <dewey@dunnington.ca>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants