From ccd694c7b1240773c802084b57b8502630f054ce Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Wed, 31 Mar 2021 10:58:20 +0200 Subject: [PATCH 1/8] Start on documentation with installation instruction --- README.rst | 55 +++---------------------------- docs/conf.py | 8 +++-- docs/includes/Installation.rst | 53 +++++++++++++++++++++++++++++ docs/{ => includes}/igzip.rst | 0 docs/{ => includes}/isal_zlib.rst | 0 docs/index.rst | 12 ++----- requirements-docs.txt | 1 + 7 files changed, 68 insertions(+), 61 deletions(-) create mode 100644 docs/includes/Installation.rst rename docs/{ => includes}/igzip.rst (100%) rename docs/{ => includes}/isal_zlib.rst (100%) diff --git a/README.rst b/README.rst index 55b7ba79..1790cb15 100644 --- a/README.rst +++ b/README.rst @@ -72,57 +72,12 @@ application (just like ``python -m gzip``). Installation ------------ -Installation with pip -..................... +- with pip: ``pip install isal`` +- with conda: ``conda install python-isal`` -:: - - pip install isal - -Installation is supported on Linux, MacOS and Windows. On x86-64 (amd64) -platforms wheels are provided, so installation should be almost instantaneous. -The installation will include a staticallly linked version of ISA-L. -If a wheel is not provided for your system the -installation will build ISA-L first in a temporary directory. Please check the -`ISA-L homepage `_ for the build requirements. - -The latest development version of python-isal can be installed with:: - - pip install git+https://github.com/rhpvorderman/python-isal.git - -This requires having the build requirements installed. -If you wish to link -dynamically against a version of libisal installed on your system use:: - - PYTHON_ISAL_LINK_DYNAMIC=true pip install isal --no-binary isal - -ISA-L is available in numerous Linux distro's as well as on conda via the -conda-forge channel. Checkout the `ports documentation -`_ on the ISA-L project wiki -to find out how to install it. It is important that the development headers -are also installed. - -On Debian and Ubuntu the ISA-L libraries (including the development headers) -can be installed with:: - - sudo apt install libisal-dev - -Installation via conda -.................................. -Python-isal can be installed via conda, for example using -the `miniconda `_ installer -with a properly setup `conda-forge -`_ -channel. When used with bioinformatics tools setting up `bioconda -`_ -provides a clear set of installation instructions for conda. - -python-isal is available on conda-forge and can be installed with:: - - conda install python-isal - -This will automatically install the ISA-L library dependency as well, since -it is available on conda-forge. +Installation is supported on Linux, Windows and MacOS. For more advanced +installation options check the `documentation +`_. Differences with zlib and gzip modules -------------------------------------- diff --git a/docs/conf.py b/docs/conf.py index 82a627eb..4592e7cc 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -40,7 +40,8 @@ # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This pattern also affects html_static_path and html_extra_path. -exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] +# includes/* prevents double indexing +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'includes/*'] # -- Options for HTML output ------------------------------------------------- @@ -48,7 +49,10 @@ # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # -html_theme = 'alabaster' +html_theme = 'sphinx_rtd_theme' +html_theme_options = dict( + display_version=True, +) # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, diff --git a/docs/includes/Installation.rst b/docs/includes/Installation.rst new file mode 100644 index 00000000..a0b9f89d --- /dev/null +++ b/docs/includes/Installation.rst @@ -0,0 +1,53 @@ +Installation +------------ +Installation with pip +..................... + +:: + + pip install isal + +Installation is supported on Linux, MacOS and Windows. On most platforms +wheels are provided. +The installation will include a staticallly linked version of ISA-L. +If a wheel is not provided for your system the +installation will build ISA-L first in a temporary directory. Please check the +`ISA-L homepage `_ for the build requirements. + +The latest development version of python-isal can be installed with:: + + pip install git+https://github.com/rhpvorderman/python-isal.git + +This requires having the build requirements installed. +If you wish to link +dynamically against a version of libisal installed on your system use:: + + PYTHON_ISAL_LINK_DYNAMIC=true pip install isal --no-binary isal + +ISA-L is available in numerous Linux distro's as well as on conda via the +conda-forge channel. Checkout the `ports documentation +`_ on the ISA-L project wiki +to find out how to install it. It is important that the development headers +are also installed. + +On Debian and Ubuntu the ISA-L libraries (including the development headers) +can be installed with:: + + sudo apt install libisal-dev + +Installation via conda +.................................. +Python-isal can be installed via conda, for example using +the `miniconda `_ installer +with a properly setup `conda-forge +`_ +channel. When used with bioinformatics tools setting up `bioconda +`_ +provides a clear set of installation instructions for conda. + +python-isal is available on conda-forge and can be installed with:: + + conda install python-isal + +This will automatically install the ISA-L library dependency as well, since +it is available on conda-forge. diff --git a/docs/igzip.rst b/docs/includes/igzip.rst similarity index 100% rename from docs/igzip.rst rename to docs/includes/igzip.rst diff --git a/docs/isal_zlib.rst b/docs/includes/isal_zlib.rst similarity index 100% rename from docs/isal_zlib.rst rename to docs/includes/isal_zlib.rst diff --git a/docs/index.rst b/docs/index.rst index 2f6ff485..e02352cb 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -10,14 +10,8 @@ Welcome to python-isal's documentation! :maxdepth: 2 :caption: Contents: - isal_zlib - igzip +.. include:: includes/Installation.rst +.. include:: includes/isal_zlib.rst - -Indices and tables -================== - -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search` +.. include:: includes/igzip.rst diff --git a/requirements-docs.txt b/requirements-docs.txt index eda5b8f9..5fb69a4e 100644 --- a/requirements-docs.txt +++ b/requirements-docs.txt @@ -1,2 +1,3 @@ sphinx setuptools +sphinx-rtd-theme \ No newline at end of file From 1d0a81e725d5c0b9d5bf366d4b6ba75853937595 Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Wed, 31 Mar 2021 11:07:36 +0200 Subject: [PATCH 2/8] Add changelog to documentation --- docs/includes/CHANGELOG.rst | 1 + docs/includes/Installation.rst | 7 ++++--- docs/includes/igzip.rst | 1 + docs/includes/isal_zlib.rst | 1 + docs/index.rst | 3 +++ 5 files changed, 10 insertions(+), 3 deletions(-) create mode 120000 docs/includes/CHANGELOG.rst diff --git a/docs/includes/CHANGELOG.rst b/docs/includes/CHANGELOG.rst new file mode 120000 index 00000000..bfa394db --- /dev/null +++ b/docs/includes/CHANGELOG.rst @@ -0,0 +1 @@ +../../CHANGELOG.rst \ No newline at end of file diff --git a/docs/includes/Installation.rst b/docs/includes/Installation.rst index a0b9f89d..a9150e29 100644 --- a/docs/includes/Installation.rst +++ b/docs/includes/Installation.rst @@ -1,7 +1,8 @@ +============ Installation ------------- +============ Installation with pip -..................... +--------------------- :: @@ -36,7 +37,7 @@ can be installed with:: sudo apt install libisal-dev Installation via conda -.................................. +---------------------- Python-isal can be installed via conda, for example using the `miniconda `_ installer with a properly setup `conda-forge diff --git a/docs/includes/igzip.rst b/docs/includes/igzip.rst index 192665e8..65b6df52 100644 --- a/docs/includes/igzip.rst +++ b/docs/includes/igzip.rst @@ -1,3 +1,4 @@ +======================== API-documentation: igzip ======================== diff --git a/docs/includes/isal_zlib.rst b/docs/includes/isal_zlib.rst index 117cc55f..f9c9b6a6 100644 --- a/docs/includes/isal_zlib.rst +++ b/docs/includes/isal_zlib.rst @@ -1,3 +1,4 @@ +============================ API Documentation: isal_zlib ============================ diff --git a/docs/index.rst b/docs/index.rst index e02352cb..5c0091d6 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -3,6 +3,7 @@ You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. +======================================= Welcome to python-isal's documentation! ======================================= @@ -15,3 +16,5 @@ Welcome to python-isal's documentation! .. include:: includes/isal_zlib.rst .. include:: includes/igzip.rst + +.. include:: includes/CHANGELOG.rst From 4b751c395f4d139b54bfefbee84edf9b77ba7767 Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Wed, 31 Mar 2021 11:35:10 +0200 Subject: [PATCH 3/8] Add some extra chapters --- README.rst | 22 ++++++++++++++++--- docs/includes/README.rst | 1 + docs/index.rst | 47 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 67 insertions(+), 3 deletions(-) create mode 120000 docs/includes/README.rst diff --git a/README.rst b/README.rst index 1790cb15..68bcde98 100644 --- a/README.rst +++ b/README.rst @@ -34,6 +34,8 @@ python-isal =========== +.. introduction start + Faster zlib and gzip compatible compression and decompression by providing Python bindings for the ISA-L library. @@ -47,8 +49,12 @@ a variety of functions to provide zlib/gzip-compatible compression. ``igzip`` module which are usable as drop-in replacements for the ``zlib`` and ``gzip`` modules from the stdlib (with some minor exceptions, see below). -Usage ------ +.. introduction end + +Quickstart +---------- + +.. usage start Python-isal has faster versions of the stdlib's ``zlib`` and ``gzip`` module these are called ``isal_zlib`` and ``igzip`` respectively. @@ -70,6 +76,8 @@ A full API documentation can be found on `our readthedocs page ``python -m isal.igzip`` implements a simple gzip-like command line application (just like ``python -m gzip``). +.. usage end + Installation ------------ - with pip: ``pip install isal`` @@ -77,11 +85,13 @@ Installation Installation is supported on Linux, Windows and MacOS. For more advanced installation options check the `documentation -`_. +`_. Differences with zlib and gzip modules -------------------------------------- +.. differences start + + Compression level 0 in ``zlib`` and ``gzip`` means **no compression**, while in ``isal_zlib`` and ``igzip`` this is the **lowest compression level**. This is a design choice that was inherited from the ISA-L library. @@ -106,8 +116,14 @@ Differences with zlib and gzip modules to reflect this. ``igzip.GzipFile`` does exist as an alias of ``igzip.IGzipFile`` for compatibility reasons. +.. differences end + Contributing ------------ +.. contributing start + Please make a PR or issue if you feel anything can be improved. Bug reports are also very welcome. Please report them on the `github issue tracker `_. + +.. contributing end diff --git a/docs/includes/README.rst b/docs/includes/README.rst new file mode 120000 index 00000000..c768ff7d --- /dev/null +++ b/docs/includes/README.rst @@ -0,0 +1 @@ +../../README.rst \ No newline at end of file diff --git a/docs/index.rst b/docs/index.rst index 5c0091d6..c56f3db2 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -11,10 +11,57 @@ Welcome to python-isal's documentation! :maxdepth: 2 :caption: Contents: +============ +Introduction +============ + +.. include:: includes/README.rst + :start-after: .. introduction start + :end-before: .. introduction end + +========== +Quickstart +========== + +Python-isal has faster versions of the stdlib's ``zlib`` and ``gzip`` module +these are called ``isal_zlib`` and ``igzip`` respectively. + +They can be imported as follows + +.. code-block:: python + + from isal import isal_zlib + from isal import igzip + +``isal_zlib`` and ``igzip`` are meant to be used as drop in replacements so +their api and functions are the same as the stdlib's modules. Except where +ISA-L does not support the same calls as zlib (See differences below). + +A full API documentation can be found below. + +``python -m isal.igzip`` implements a simple gzip-like command line +application (just like ``python -m gzip``). + + .. include:: includes/Installation.rst +====================================== +Differences with zlib and gzip modules +====================================== + +.. include:: includes/README.rst + :start-after: .. differences start + :end-before: .. differences end + .. include:: includes/isal_zlib.rst .. include:: includes/igzip.rst +============ +Contributing +============ +.. include:: includes/README.rst + :start-after: .. contributing start + :end-before: .. contributing end + .. include:: includes/CHANGELOG.rst From dbdbed53a027a02fba4693bd01b17b31fb62108c Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Wed, 31 Mar 2021 11:46:08 +0200 Subject: [PATCH 4/8] Add python -m isal.igzip usage --- docs/conf.py | 3 +-- docs/includes/isal.igzip_usage.rst | 8 ++++++++ docs/index.rst | 2 ++ requirements-docs.txt | 3 ++- src/isal/igzip.py | 12 +++++++++--- 5 files changed, 22 insertions(+), 6 deletions(-) create mode 100644 docs/includes/isal.igzip_usage.rst diff --git a/docs/conf.py b/docs/conf.py index 4592e7cc..1f6b7297 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -31,8 +31,7 @@ # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions = ["sphinx.ext.autodoc" -] +extensions = ["sphinx.ext.autodoc", 'sphinxarg.ext'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] diff --git a/docs/includes/isal.igzip_usage.rst b/docs/includes/isal.igzip_usage.rst new file mode 100644 index 00000000..dc5b602b --- /dev/null +++ b/docs/includes/isal.igzip_usage.rst @@ -0,0 +1,8 @@ +========================== +python -m isal.igzip usage +========================== + +.. argparse:: + :module: isal.igzip + :func: _argument_parser + :prog: python -m isal.igzip diff --git a/docs/index.rst b/docs/index.rst index c56f3db2..f1be683f 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -57,6 +57,8 @@ Differences with zlib and gzip modules .. include:: includes/igzip.rst +.. include:: includes/isal.igzip_usage.rst + ============ Contributing ============ diff --git a/requirements-docs.txt b/requirements-docs.txt index 5fb69a4e..051c278d 100644 --- a/requirements-docs.txt +++ b/requirements-docs.txt @@ -1,3 +1,4 @@ sphinx setuptools -sphinx-rtd-theme \ No newline at end of file +sphinx-rtd-theme +sphinx-argparse \ No newline at end of file diff --git a/src/isal/igzip.py b/src/isal/igzip.py index eb2c617f..0813248d 100644 --- a/src/isal/igzip.py +++ b/src/isal/igzip.py @@ -348,7 +348,7 @@ def decompress(data): return b"".join(all_blocks) -def main(): +def _argument_parser(): parser = argparse.ArgumentParser() parser.description = ( "A simple command line interface for the igzip module. " @@ -371,9 +371,11 @@ def main(): "-3", "--best", action="store_const", dest="compresslevel", const=_COMPRESS_LEVEL_BEST, help="use compression level 3 (best)") + compress_group.set_defaults(compress=True) compress_group.add_argument( - "-d", "--decompress", action="store_false", + "-d", "--decompress", action="store_const", dest="compress", + const=False, help="Decompress the file instead of compressing.") parser.add_argument("-c", "--stdout", action="store_true", help="write on standard output") @@ -383,7 +385,11 @@ def main(): parser.add_argument("-b", "--buffer-size", default=32 * 1024, type=int, help=argparse.SUPPRESS) - args = parser.parse_args() + return parser + + +def main(): + args = _argument_parser().parse_args() compresslevel = args.compresslevel or _COMPRESS_LEVEL_TRADEOFF From e1a07cecb32ffa25400b13ea82d56dd47adcb6f9 Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Fri, 2 Apr 2021 11:04:23 +0200 Subject: [PATCH 5/8] Add acknowledgements chapter --- README.rst | 33 +++++++++++++++++++++++++++++++++ docs/index.rst | 7 +++++++ 2 files changed, 40 insertions(+) diff --git a/README.rst b/README.rst index 68bcde98..2e5f9687 100644 --- a/README.rst +++ b/README.rst @@ -127,3 +127,36 @@ are also very welcome. Please report them on the `github issue tracker `_. .. contributing end + +Acknowledgements +---------------- + +.. acknowledgements start + +This project builds upon the software and experience of many. Many thanks to: + ++ The `ISA-L contributors + `_ for making ISA-L. ++ The `Cython contributors + `_ for making it easy + to create an extension and helping a novice get start with pointer addresses. ++ The `CPython contributors + `_. + Python-isal mimicks ``zlibmodule.c`` and ``igzip.py`` to make it easier for + python users to adopt it. ++ `@marcelm `_ for taking a chance on this project + and make it a dependency for his `xopen + `_ and by extension `cutadapt + `_ projects. This gave python-isal its + first users who used python-isal in production. ++ The `github actions team `_ for + creating the actions CI service that enables building and testing on all + three major operating systems. ++ `@animalize `_ for explaining how to test and + build python-isal for ARM 64-bit platforms. ++ And last but not least: everyone who submitted a bug report or a feature + request. These make the project better! + +Python-isal would not have been possible without you! + +.. acknowledgements end \ No newline at end of file diff --git a/docs/index.rst b/docs/index.rst index f1be683f..29801a94 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -66,4 +66,11 @@ Contributing :start-after: .. contributing start :end-before: .. contributing end +================ +Acknowledgements +================ +.. include:: includes/README.rst + :start-after: .. acknowledgements start + :end-before: .. acknowledgements end + .. include:: includes/CHANGELOG.rst From 685d1851efd910a030b2de1ebf4840a2654794a8 Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Fri, 2 Apr 2021 11:09:53 +0200 Subject: [PATCH 6/8] Refer to usage documentation --- README.rst | 4 +++- docs/index.rst | 3 ++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/README.rst b/README.rst index 2e5f9687..4a2d0905 100644 --- a/README.rst +++ b/README.rst @@ -74,7 +74,9 @@ A full API documentation can be found on `our readthedocs page `_. ``python -m isal.igzip`` implements a simple gzip-like command line -application (just like ``python -m gzip``). +application (just like ``python -m gzip``). Full usage documentation can be +found on `our readthedocs page `_. + .. usage end diff --git a/docs/index.rst b/docs/index.rst index 29801a94..3b4a869a 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -40,7 +40,8 @@ ISA-L does not support the same calls as zlib (See differences below). A full API documentation can be found below. ``python -m isal.igzip`` implements a simple gzip-like command line -application (just like ``python -m gzip``). +application (just like ``python -m gzip``). Full usage documentation can be +found below. .. include:: includes/Installation.rst From f9cd3f809bf50580251b0876549ef8e702d0dc5c Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Fri, 2 Apr 2021 11:19:04 +0200 Subject: [PATCH 7/8] Update changelog --- CHANGELOG.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CHANGELOG.rst b/CHANGELOG.rst index f6554898..06834b2d 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -7,6 +7,10 @@ Changelog .. This document is user facing. Please word the changes in such a way .. that users understand how the changes affect the new version. +version 0.10.0-dev +------------------ ++ Added more extensive documentation. + version 0.9.0 ----------------- + Fix a bug where a AttributeError was triggered when zlib.Z_RLE or From dcbbe95a0cabf18b144e1c36fad11ca250782b3d Mon Sep 17 00:00:00 2001 From: Ruben Vorderman Date: Fri, 2 Apr 2021 11:19:32 +0200 Subject: [PATCH 8/8] Doc update if needed checkbox in PR template --- .github/PULL_REQUEST_TEMPLATE.md | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index fd6a943e..3e4afab3 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,3 +1,4 @@ ### Checklist - [ ] Pull request details were added to CHANGELOG.rst +- [ ] Documentation was updated (if needed)