Skip to content

GH-44900: [Python] Support fsspec filesystem URIs #46851

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

AlenkaF
Copy link
Member

@AlenkaF AlenkaF commented Jun 18, 2025

Rationale for this change

It would be convenient to read parquet files from fsspec filesystems using schemes available in the fsspec registry using URIs, without explicitly specifying the filesystem object.

What changes are included in this PR?

This changes are based on #45089. In this PR _resolve_filesystem_and_path() can recognize the scheme used in the URI together with the fsspec prefix. For example:

>>> table = pa.table({"a": range(10)})
>>> pq.write_table(table, "fsspec+memory://example.parquet")
>>> pq.read_table("fsspec+memory://example.parquet")
pyarrow.Table
a: int64
----
a: [[0,1,2,3,4,5,6,7,8,9]]

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant