Skip to content

Commit

Permalink
Merge a620194 into 65139a5
Browse files Browse the repository at this point in the history
  • Loading branch information
juarezr committed Jun 26, 2020
2 parents 65139a5 + a620194 commit c3a305d
Show file tree
Hide file tree
Showing 15 changed files with 440 additions and 582 deletions.
16 changes: 16 additions & 0 deletions docs/changes.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,22 @@
Changes
=======

Version 1.6.0
-------------

* Added class :class:`petl.io.remotes.RemoteSource` using package **fsspec**
for reading and writing files in remote servers by using the protocol in the
url for selecting the implementation.
By :user:`juarezr`, :issue:`494`.

* Removed classes :class:`petl.io.source.s3.S3Source` as it's handled by fsspec
By :user:`juarezr`, :issue:`494`.

* Removed classes :class:`petl.io.codec.xz.XZCodec`, :class:`petl.io.codec.xz.LZ4Codec`
and :class:`petl.io.codec.zstd.ZstandardCodec` as it's handled by fsspec.
By :user:`juarezr`, :issue:`494`.


Version 1.5.0
-------------

Expand Down
35 changes: 13 additions & 22 deletions docs/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -359,13 +359,8 @@ The behaviour of each source can usually be configured by passing arguments
to the constructor, see the source code of the :mod:`petl.io.sources` module
for full details.

.. autoclass:: petl.io.sources.FileSource
.. autoclass:: petl.io.sources.GzipSource
.. autoclass:: petl.io.sources.BZ2Source
.. autoclass:: petl.io.sources.ZipSource
.. autoclass:: petl.io.sources.StdinSource
.. autoclass:: petl.io.sources.StdoutSource
.. autoclass:: petl.io.sources.URLSource
.. autoclass:: petl.io.sources.MemorySource
.. autoclass:: petl.io.sources.PopenSource

Expand All @@ -383,38 +378,34 @@ in :ref:`Extract <io_extract>` and :ref:`Load <io_load>`.
It's possible to read and write just by prefixing the protocol (e.g: `s3://`)
in the source path of the file.

.. autoclass:: petl.io.source.s3.S3Source
.. autoclass:: petl.io.source.smb.SMBSource
.. autoclass:: petl.io.remotes.RemoteSource
.. autoclass:: petl.io.remotes.SMBSource

.. _io_codecs:
.. _io_deprecated:

Compression I/O helper classes
------------------------------
Deprecated I/O helper classes
-----------------------------

The following classes are helpers for decompressing (``from...()``) and
compressing (``to...()``) in functions transparently as a file-like source.
The following helpers are deprecated and will be removed in a future version.

There are no need to instantiate them. They are used in the mecanism described
in :ref:`Extract <io_extract_codec>` and :ref:`Load <io_load_codec>`.
It's functionality was replaced by helpers in :ref:`Remote helpers <io_remotes>`.

It's possible to compress and decompress just by specifying the file extension
(e.g: `.csv.xz`) in end of the source filename.

.. autoclass:: petl.io.codec.xz.XZCodec
.. autoclass:: petl.io.codec.zstd.ZstandardCodec
.. autoclass:: petl.io.codec.lz4.LZ4Codec
.. autoclass:: petl.io.sources.FileSource
.. autoclass:: petl.io.sources.GzipSource
.. autoclass:: petl.io.sources.BZ2Source
.. autoclass:: petl.io.sources.ZipSource
.. autoclass:: petl.io.sources.URLSource

.. _io_custom_helpers:

Custom I/O helper classes
------------------------------

For creating custom helpers for :ref:`remote I/O <io_remotes>` or
:ref:`compression <io_codecs>` use the following functions:
`compression` use the following functions:

.. autofunction:: petl.io.sources.register_reader
.. autofunction:: petl.io.sources.register_writer
.. autofunction:: petl.io.sources.register_codec

See the source code of the classes in :mod:`petl.io.sources` module for
more details.
4 changes: 4 additions & 0 deletions petl/io/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,7 @@
from petl.io.avro import fromavro, toavro, appendavro

from petl.io.sources import register_codec, register_reader, register_writer

from petl.io.remotes import RemoteSource

from petl.io.remotes import SMBSource
7 changes: 0 additions & 7 deletions petl/io/codec/__init__.py

This file was deleted.

50 changes: 0 additions & 50 deletions petl/io/codec/lz4.py

This file was deleted.

38 changes: 0 additions & 38 deletions petl/io/codec/xz.py

This file was deleted.

58 changes: 0 additions & 58 deletions petl/io/codec/zstd.py

This file was deleted.

Loading

0 comments on commit c3a305d

Please sign in to comment.