Skip to content

Commit

Permalink
docs
Browse files Browse the repository at this point in the history
  • Loading branch information
willmcgugan committed Nov 19, 2016
1 parent 7193cd4 commit ab15ba7
Show file tree
Hide file tree
Showing 15 changed files with 218 additions and 176 deletions.
57 changes: 29 additions & 28 deletions docs/source/guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,33 +57,33 @@ The opener system is particularly useful when you want to store the physical loc
If you don't specify the protocol in the FS URL, then PyFilesystem will assume you want a OSFS relative from the current working directory. So the following would be an equivalent way of opening your home directory::

>>> from fs import open_fs
>>> home_fs = open_fs('~/')
>>> home_fs = open_fs('.')
>>> home_fs.listdir('/')
['world domination.doc', 'paella-recipe.txt', 'jokes.txt', 'projects']

Tree
%%%%
Tree Printing
~~~~~~~~~~~~~

Calling :meth:`fs.base.FS.tree` on a FS object will print an ascii tree view of your filesystem. Here's an example::

>>> from fs import open_fs
>>> my_fs = open_fs('.')
>>> my_fs.tree()
├── locale
── readme.txt
── readme.txt
├── logic
│ ├── content.xml
│ ├── data.xml
│ ├── mountpoints.xml
── readme.txt
── readme.txt
├── lib.ini
── readme.txt
── readme.txt

This can be a useful debugging aid!


Closing Filesystems
~~~~~~~~~~~~~~~~~~~
Closing
~~~~~~~

FS objects have a :meth:`fs.base.FS.close` methd which will perform any required clean-up actions. For many filesystems (notably :class:`fs.osfs.OSFS`), the ``close`` method does very little. Other filesystems may only finalize files or release resources once ``close()`` is called.

Expand Down Expand Up @@ -124,11 +124,11 @@ Info objects have a number of advantages over just a filename. For instance you
Additionally, FS objects have a :meth:`fs.base.FS.filterdir` method which extends ``scandir`` with the ability to filter directory contents by wildcard(s). Here's how you might find all the Python files in a directory:

>>> code_fs = OSFS('~/projects/src')
>>> directory = list(code_fs.filterdir('/', wildcards=['*.py']))
>>> directory = list(code_fs.filterdir('/', files=['*.py']))

By default, the resource information objects returned by ``scandir`` and ``listdir`` will contain only the file name and the ``is_dir`` flag. You can request additional information with the ``namespaces`` parameter. Here's how you can request additional details (such as file size and file modified times)::

>>> directory = code_fs.filterdir('/', wildcards=['*.py'], namespaces=['details'])
>>> directory = code_fs.filterdir('/', files=['*.py'], namespaces=['details'])

This will add a ``size`` and ``modified`` property (and others) to the resource info objects. Which makes code such as this work::

Expand Down Expand Up @@ -164,22 +164,6 @@ The :class:`fs.base.FS.makedir` and :class:`fs.base.FS.makedirs` methods also re

Working with ``SubFS`` objects means that you can generally avoid writing much path manipulation code, which tends to be error prone.

Walking
~~~~~~~

Often you will need to scan the files in a given directory, and any sub-directories. This is known as *walking* the filesystem.

Here's how you would print the paths to all your Python files in your home directory::

>>> from fs import open_fs
>>> home_fs = open_fs('~/')
>>> for path in home_fs.walk.files(wildcards=['*.py']):
... print(path)

The ``walk`` attribute on FS objects is instance of a :class:`fs.walk.BoundWalker`, which should be able to handle most directory walking requirements.

See :ref:`walking` for more information on walking directories.

Working with Files
~~~~~~~~~~~~~~~~~~

Expand All @@ -192,10 +176,26 @@ You can open a file from a FS object with :meth:`fs.base.FS.open`, which is very

In the case of a ``OSFS``, a standard file-like object will be returned. Other filesystems may return a different object supporting the same methods. For instance, :class:`fs.memoryfs.MemoryFS` will return a ``io.BytesIO`` object.

PyFilesystem also offers a number of shortcuts for common file related operations. For example, :meth:`fs.base.FS.getbytes` will return the file contents as a bytes, and :meth:`fs.base.FS.gettext` will read unicode text. Using these methods is generally preferable to explicitly opening files, as the FS object may have an optimized implementation.
PyFilesystem also offers a number of shortcuts for common file related operations. For instance, :meth:`fs.base.FS.getbytes` will return the file contents as a bytes, and :meth:`fs.base.FS.gettext` will read unicode text. These methods is generally preferable to explicitly opening files, as the FS object may have an optimized implementation.

Other *shortcut* methods are :meth:`fs.base.FS.setbin`, :meth:`fs.base.FS.setbytes`, :meth:`fs.base.FS.settext`.

Walking
~~~~~~~

Often you will need to scan the files in a given directory, and any sub-directories. This is known as *walking* the filesystem.

Here's how you would print the paths to all your Python files in your home directory::

>>> from fs import open_fs
>>> home_fs = open_fs('~/')
>>> for path in home_fs.walk.files(filter=['*.py']):
... print(path)

The ``walk`` attribute on FS objects is instance of a :class:`fs.walk.BoundWalker`, which should be able to handle most directory walking requirements.

See :ref:`walking` for more information on walking directories.

Moving and Copying
~~~~~~~~~~~~~~~~~~

Expand All @@ -219,5 +219,6 @@ The :func:`fs.copy.copy_fs` and :func:`fs.copy.copy_dir` functions also accept a

>>> from fs.copy import copy_fs
>>> from fs.walk import Walker
>>> copy_fs('~/projects', 'zip://projects.zip', walker=Walker(wildcards=['*.py']))
>>> copy_fs('~/projects', 'zip://projects.zip', walker=Walker(files=['*.py']))


1 change: 1 addition & 0 deletions docs/source/reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,4 @@ Reference
reference/path.rst
reference/tree.rst
reference/walk.rst
reference/wildcard.rst
2 changes: 1 addition & 1 deletion docs/source/reference/tree.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
fs.tree
=======

Render ASCII (or unicode) directory trees.
Render a text tree view, with optional color in terminals.

.. automodule:: fs.tree
:members:
82 changes: 45 additions & 37 deletions fs/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -219,20 +219,23 @@ def close(self):
"""
self._closed = True

def copy(self, src_path, dst_path):
def copy(self, src_path, dst_path, overwrite=False):
"""
Copy file contents from ``src_path`` to ``dst_path``.
:param src_path: Path of source file.
:type src_path: str
:param dst_path: Path to destination file.
:type dst_path: str
If the path specified by ``dst_path`` exists, and is a file,
it will first be truncated.
:raises `fs.errors.DestinationExists`: If ``dst_path`` exists,
and overwrite == ``False``.
:raises `fs.errors.ResourceNotFound`: If a parent directory of
``dst_path`` does not exist.
"""
with self._lock:
if not overwrite and self.exists(dst_path):
raise errors.DestinationExists(dst_path)
with closing(self.open(src_path, 'rb')) as read_file:
self.setbin(dst_path, read_file)

Expand Down Expand Up @@ -319,36 +322,36 @@ def filterdir(self,
path,
exclude_dirs=False,
exclude_files=False,
wildcards=None,
dir_wildcards=None,
files=None,
dirs=None,
namespaces=None,
page=None):
"""
Get an iterator of resource info, filtered by wildcards.
This method enhances the :meth:`fs.base.FS.scandir` method with
additional filtering functionality.
:param str path: A path to a directory on the filesystem.
:param bool exclude_dirs: Exclude directories.
:param bool exclude_files: Exclude files.
:param wildcards: A list of unix shell-style wildcards to filter
file names.
:param files: A list of unix shell-style patterns to filter
file names, e.g. ``['*.py']``.
:type wildcards: list or None
:param dir_wildcards: A list of unix shell-style wildcards to
:param dirs: A list of unix shell-style wildcards to
filter directory names.
:type dir_wildcards: list or None
:type dirs: list or None
:param namespaces: A list of info namespaces to include in
results.
:type namespaces: list or None
:param page: May be a tuple of (start, end) indexes to return an
iterator of a subset of the resource info, or ``None`` to
iterator the entire directory. Paging a directory scan may
be necessary for very large directories.
:param page: May be a tuple of ``(<start>, <end>)`` indexes to
return an iterator of a subset of the resource info, or
``None`` to iterator the entire directory. Paging a
directory scan may be necessary for very large directories.
:type page: tuple or None
:return: An iterator of :class:`fs.info.Info` objects.
:rtype: iterator
This method enhances the :meth:`fs.base.FS.scandir` method with
additional filtering functionality.
"""

case_sensitive = self.getmeta().get('case_sensitive', True)
Expand All @@ -369,24 +372,24 @@ def filterdir(self,
if info.is_dir
)

if wildcards is not None:
if isinstance(wildcards, six.text_type):
if files is not None:
if isinstance(files, six.text_type):
raise ValueError(
'wildcards must be a sequence, not a string'
)
match = wildcard.get_matcher(wildcards, case_sensitive)
match = wildcard.get_matcher(files, case_sensitive)
resources = (
info
for info in resources
if info.is_dir or match(info.name)
)

if dir_wildcards is not None:
if isinstance(dir_wildcards, six.text_type):
if dirs is not None:
if isinstance(dirs, six.text_type):
raise ValueError(
'dir_wildcards must be a sequence, not a string'
)
match = wildcard.get_matcher(dir_wildcards, case_sensitive)
match = wildcard.get_matcher(dirs, case_sensitive)
resources = (
info
for info in resources
Expand Down Expand Up @@ -577,8 +580,8 @@ def geturl(self, path, purpose='download'):
:param str purpose: A short string that indicates which URL to
retrieve for the given path (if there is more than one). The
default is `'download'`, which should return a URL that
serves the file. See the filesystem documentation for
information on what other URLs may be generated.
serves the file. Other filesystems may support other values
for ``purpose``.
:returns: A URL.
:rtype: str
:raises `fs.errors.NoURL`: If the path does not map to a URL.
Expand Down Expand Up @@ -712,9 +715,8 @@ def makedirs(self, path, permissions=None, recreate=False):
:raises `fs.errors.DirectoryExists`: if the path is already
a directory, and ``recreate`` is False.
:raises `fs.errors.NotADirectory`: if one of the ancestors in
the path isn't a directory.
:raises `fs.errors.ResourceNotFound`: if the path is not found.
:raises `fs.errors.DirectoryExpected`: if one of the ancestors
in the path isn't a directory.
"""
self.check()
Expand Down Expand Up @@ -742,6 +744,10 @@ def move(self,
file will be written to.
:param bool overwrite: If `True` destination path will be
overwritten if it exists.
:raises `fs.errors.DestinationExists`: If ``dst_path`` exists,
and overwrite == ``False``.
:raises `fs.errors.ResourceNotFound`: If a parent directory of
``dst_path`` does not exist.
"""

Expand Down Expand Up @@ -811,6 +817,8 @@ def opendir(self, path):
:param str path: Path to a directory on the filesystem.
:returns: A filesystem object representing a sub-directory.
:rtype: :class:`fs.subfs.SubFS`
:raises `fs.errors.DirectoryExpected`: If ``dst_path`` does not
exist or is not a directory.
"""
from .subfs import SubFS
Expand Down Expand Up @@ -850,10 +858,10 @@ def scandir(self, path, namespaces=None, page=None):
:param str path: A path on the filesystem
:param list namespaces: A sequence of info namespaces.
:param page: May be a tuple of (start, end) indexes to return an
iterator of a subset of the resource info, or ``None`` to
iterator the entire directory. Paging a directory scan may
be necessary for very large directories.
:param page: May be a tuple of ``(<start>, <end>)`` indexes to
return an iterator of a subset of the resource info, or
``None`` to iterator the entire directory. Paging a
directory scan may be necessary for very large directories.
:type page: tuple or None
:rtype: iterator
Expand Down Expand Up @@ -1128,11 +1136,11 @@ def check(self):
if self.isclosed():
raise errors.FilesystemClosed()

def match(self, wildcards, name):
def match(self, patterns, name):
"""
Check if a name matches any of a list of wildcards.
:param list wildcards: A list of wildcards, e.g. ``['*.py']``
:param list patterns: A list of patterns, e.g. ``['*.py']``
:param str name: A file or directory name (not a path)
:rtype: bool
Expand All @@ -1147,14 +1155,14 @@ def match(self, wildcards, name):
>>> home_fs.match(['*.jpg', '*.png'], 'foo.gif')
False
If ``wildcards`` is ``None``, or (``['*']``), then this method
If ``patterns`` is ``None``, or (``['*']``), then this method
will always return True.
"""
if wildcards is None:
if patterns is None:
return True
case_sensitive = self.getmeta().get('case_sensitive', True)
matcher = wildcard.get_matcher(wildcards, case_sensitive)
matcher = wildcard.get_matcher(patterns, case_sensitive)
return matcher(name)

def tree(self, **kwargs):
Expand Down
7 changes: 5 additions & 2 deletions fs/copy.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ def copy_file(src_fs, src_path, dst_fs, dst_path):
"""
Copy a file from one filesystem to another.
If the destination exists, and is a file, it will be first
truncated.
:param src_fs: Source filesystem.
:type src_fs: FS URL or instance
:param src_path: Path to a file on ``src_fs``.
Expand All @@ -52,7 +55,7 @@ def copy_file(src_fs, src_path, dst_fs, dst_path):
with manage_fs(dst_fs, create=True) as dst_fs:
if src_fs is dst_fs:
# Same filesystem, so we can do a potentially optimized copy
src_fs.copy(src_path, dst_path)
src_fs.copy(src_path, dst_path, overwrite=True)
else:
# Standard copy
with src_fs.lock(), dst_fs.lock():
Expand Down Expand Up @@ -93,7 +96,7 @@ def copy_dir(src_fs, src_path, dst_fs, dst_path, walker=None):
:type src_path: str
:param dst_fs: Destination filesystem.
:type dst_fs: FS URL or instance
:param dst_path: A path to a directory on ``dst_fs``.
:param str dst_path: A path to a directory on ``dst_fs``.
:param walker: A walker object that will be used to scan for files
in ``src_fs``. Set this if you only want to consider a sub-set
of the resources in ``src_fs``.
Expand Down
4 changes: 3 additions & 1 deletion fs/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,11 @@
'CreateFailed',
'DestinationExists',
'DirectoryExists',
'DirectoryExpected',
'DirectoryNotEmpty',
'FilesystemClosed',
'FileExists',
'FileExpected',
'FilesystemClosed',
'FSError',
'IllegalBackReference',
'InsufficientStorage',
Expand Down
2 changes: 1 addition & 1 deletion fs/move.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ def move_dir(src_fs, src_path, dst_fs, dst_path):
:type src_path: str
:param dst_fs: Destination filesystem.
:type dst_fs: FS URL or instance
:param dst_path: A path to a directory on ``dst_fs``.
:param str dst_path: A path to a directory on ``dst_fs``.
"""
with manage_fs(src_fs) as src_fs:
Expand Down

0 comments on commit ab15ba7

Please sign in to comment.