Skip to content

IO: Fix Windows drive letters misidentified as URI schemes - #3721

Open
qzyu999 wants to merge 1 commit into
apache:mainfrom
qzyu999:fix-windows-ci
Open

IO: Fix Windows drive letters misidentified as URI schemes#3721
qzyu999 wants to merge 1 commit into
apache:mainfrom
qzyu999:fix-windows-ci

Conversation

@qzyu999

@qzyu999 qzyu999 commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Rationale

On Windows, urlparse('C:\Users\...') produces scheme='c', causing:

ValueError: Unrecognized filesystem type in URI: c

This breaks all local file operations for Windows users.

The Fix

A single platform-guarded predicate shared across all three affected parse sites:

def _is_windows_drive_letter(scheme: str) -> bool:
    return sys.platform == 'win32' and len(scheme) == 1 and scheme.isalpha()

When detected, the scheme is remapped to 'file', which routes to PyArrow/fsspec's local filesystem which already handles Windows paths natively.

Changes

File Change
pyiceberg/io/__init__.py Add predicate + normalize scheme in _infer_file_io_from_scheme
pyiceberg/io/pyarrow.py Guard in parse_location
pyiceberg/io/fsspec.py Guard in _get_fs_from_uri
tests/io/test_io.py Predicate + integration tests
tests/io/test_pyarrow.py parse_location Windows test

Why all three sites?

They execute at different lifecycle stages and are reachable through independent code paths. Fixing only one (as PR #3161 did) leaves the others broken.

Regression safety

The sys.platform == 'win32' guard means this is literally invisible on Linux/macOS the predicate short-circuits to False without inspecting the scheme. Existing CI on ubuntu-latest cannot observe any change.

Are these changes tested?

Yes. 11 new test cases covering:

  • Predicate correctness (real schemes False, drive letters on Windows True)
  • Integration (Windows paths resolve to PyArrowFileIO)
  • Platform-conditional via @pytest.mark.skipif

All existing tests continue to pass.

Closes

Closes #3720
Related: #2477, #1005

On Windows, Python's urlparse treats paths like 'C:\Users\...' as having scheme='c', which causes 'Unrecognized filesystem type in URI: c' errors. This adds a platform-guarded predicate that detects single-character alphabetic schemes on Windows and remaps them to 'file', enabling correct local filesystem routing.

Fixes all three parse sites (_infer_file_io_from_scheme, PyArrowFileIO.parse_location, FsspecFileIO._get_fs_from_uri) and includes platform-conditional tests.

Related: apache#2477, apache#1005
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

IO: Windows paths crash with 'Unrecognized filesystem type in URI: c'

1 participant