Apache Iceberg version
Latest main (also affects all released versions)
Please describe the bug
On Windows, passing a local absolute path (e.g. C:\Users\data\warehouse) to any PyIceberg IO operation crashes with:
from pyiceberg.io.pyarrow import PyArrowFileIO
io = PyArrowFileIO()
f = io.new_input(r"C:\Users\me\warehouse\table\metadata.json")
# ValueError: Unrecognized filesystem type in URI: c
an example from #1005 is:
from pyiceberg.catalog import load_catalog
catalog = load_catalog(
"my_catalog",
**{
"type": "sql",
"uri": "sqlite:///catalog.db",
"warehouse": r"C:\Users\me\iceberg_warehouse",
},
)
# UserWarning: No preferred file implementation for scheme: c
The root cause: Python's urlparse treats Windows drive letters as URI schemes (C:\... scheme='c'). Since 'c' is not in PyIceberg's scheme-to-FileIO mapping, all local file operations fail.
This affects three independent parse sites:
_infer_file_io_from_scheme() in pyiceberg/io/__init__.py
PyArrowFileIO.parse_location() in pyiceberg/io/pyarrow.py
FsspecFileIO._get_fs_from_uri() in pyiceberg/io/fsspec.py
Related issues
Proposed fix
A platform-guarded predicate that detects when urlparse has misidentified a Windows drive letter as a URI scheme, and remaps it to 'file':
import sys
def _is_windows_drive_letter(scheme: str) -> bool:
return sys.platform == 'win32' and len(scheme) == 1 and scheme.isalpha()
Applied at all three parse sites. Zero behavior change on non-Windows (the platform check short-circuits). Includes platform-conditional tests.
Apache Iceberg version
Latest main (also affects all released versions)
Please describe the bug
On Windows, passing a local absolute path (e.g.
C:\Users\data\warehouse) to any PyIceberg IO operation crashes with:from pyiceberg.io.pyarrow import PyArrowFileIO
an example from #1005 is:
The root cause: Python's
urlparsetreats Windows drive letters as URI schemes (C:\...scheme='c'). Since'c'is not in PyIceberg's scheme-to-FileIO mapping, all local file operations fail.This affects three independent parse sites:
_infer_file_io_from_scheme()inpyiceberg/io/__init__.pyPyArrowFileIO.parse_location()inpyiceberg/io/pyarrow.pyFsspecFileIO._get_fs_from_uri()inpyiceberg/io/fsspec.pyRelated issues
Proposed fix
A platform-guarded predicate that detects when
urlparsehas misidentified a Windows drive letter as a URI scheme, and remaps it to'file':Applied at all three parse sites. Zero behavior change on non-Windows (the platform check short-circuits). Includes platform-conditional tests.