Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Override path formatting method for data-URIs #169

Merged
merged 5 commits into from Feb 10, 2024

Conversation

joouha
Copy link
Contributor

@joouha joouha commented Jan 2, 2024

This PR fixes an issue where UPath.stat is broken for data-URIs:

>>> from upath import UPath
>>> UPath('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVQI12PYeuECAASTAlbqXbfWAAAAAElFTkSuQmCC').stat()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/josiah/.local/share/hatch/env/virtual/euporie/afiNbXov/euporie/lib/python3.11/site-packages/upath/core.py", line 339, in stat
    return self._accessor.stat(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/josiah/.local/share/hatch/env/virtual/euporie/afiNbXov/euporie/lib/python3.11/site-packages/upath/core.py", line 66, in stat
    return self._fs.stat(self._format_path(path), **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/josiah/.local/share/hatch/env/virtual/euporie/afiNbXov/euporie/lib/python3.11/site-packages/fsspec/spec.py", line 1514, in stat
    return self.info(path, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/josiah/.local/share/hatch/env/virtual/euporie/afiNbXov/euporie/lib/python3.11/site-packages/fsspec/implementations/data.py", line 34, in info
    mime = pref.split(":", 1)[1].split(";", 1)[0]
           ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

This occurs because fsspec.implementations.data.DataFileSystem.info expects the full URI including the scheme as a parameter, while currently only the path component is passed.

joouha and others added 5 commits January 2, 2024 11:25
This fixes an issue with `UPath.stat()` for data URIs, where fsspec was
expecting the full URI to be passed to `fs.info` instead of just the
URI path as was previously implemented.
@ap--
Copy link
Collaborator

ap-- commented Feb 10, 2024

Thank you for your contribution ❤️

I made a few changes and ported it to the soon to be released UPath version.

@ap-- ap-- merged commit f6e9adc into fsspec:main Feb 10, 2024
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants