Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Pathlib-like protocol in type_utils.py #2562

Closed
Conchylicultor opened this issue Oct 12, 2020 · 1 comment
Closed

Add a Pathlib-like protocol in type_utils.py #2562

Conchylicultor opened this issue Oct 12, 2020 · 1 comment
Labels
enhancement New feature or request

Comments

@Conchylicultor
Copy link
Member

https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/core/utils/type_utils.py define Python typing annotations.

Currently it's not easy to annotate function which accept multiple types of pathlib-like object:

def f(path: Union[pathlib.Path, importlib.resources.abc.Transversal, gfile.GPath, githubpath.GithubPath]):
  p = path.iterdir()
  ...

All pathlib.Path,... have the same interface, but are not based on the same class.
We should have a typing annotation based on protocol (https://www.python.org/dev/peps/pep-0544/) to annotate those functions:

def f(path: type_utils.ReadOnlyPath):
  p = path.iterdir()
  ...
class ReadOnlyPath(typing.Protocol):
  def iterdir() -> Iterable['ReadOnlyPath']:
  def isdir() -> bool:
  ...

It will be very helpful when a Gfile based path-like interface exists.

@Conchylicultor
Copy link
Member Author

Conchylicultor commented Oct 13, 2020

Could be inspired from importlib_resources.abc.Traversable:

class Traversable(typing.Protocol):
    """
    An object with a subset of pathlib.Path methods suitable for
    traversing directories and opening files.
    """

    @abc.abstractmethod
    def iterdir(self):
        """
        Yield Traversable objects in self
        """

    @abc.abstractmethod
    def read_bytes(self):
        """
        Read contents of self as bytes
        """

    @abc.abstractmethod
    def read_text(self, encoding=None):
        """
        Read contents of self as bytes
        """

    @abc.abstractmethod
    def is_dir(self):
        """
        Return True if self is a dir
        """

    @abc.abstractmethod
    def is_file(self):
        """
        Return True if self is a file
        """

    @abc.abstractmethod
    def joinpath(self, child):
        """
        Return Traversable child in self
        """

    @abc.abstractmethod
    def __truediv__(self, child):
        """
        Return Traversable child in self
        """

    @abc.abstractmethod
    def open(self, mode='r', *args, **kwargs):
        """
        mode may be 'r' or 'rb' to open as text or binary. Return a handle
        suitable for reading (same as pathlib.Path.open).

        When opening as text, accepts encoding parameters such as those
        accepted by io.TextIOWrapper.
        """

    @abc.abstractproperty
    def name(self):
        # type: () -> str
        """
        The base name of this object without any parent references.
        """

But with parent property, and typing annotation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant