Skip to content

Commit

Permalink
split doc index file
Browse files Browse the repository at this point in the history
  • Loading branch information
@ssato
ssato committed Aug 6, 2015
1 parent ab7e5ec commit 4622346
Show file tree
Hide file tree
Showing 2 changed files with 273 additions and 258 deletions.
259 changes: 1 addition & 258 deletions docs/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,263 +10,6 @@ python-anyconfig
design
cli

Introduction
=============

python-anyconfig [#]_ is a `MIT licensed <http://opensource.org/licenses/MIT>`_
python library provides generic access to configuration files in various
formats with configuration merge along with config template and schema
validation/generation support.

python-anyconfig provides very simple and unified APIs for various
configuration files:

- anyconfig.load() to load configuration files and it will return a dict-like object represents configuration loaded
- anyconfig.loads() to load a configuration string and ...
- anyconfig.dump() to dump a configuration file from a dict or dict-like object represents some configurations
- anyconfig.dumps() to dump a configuration string from ...
- anyconfig.validate() to validate configuration files with JSON schema [#]_ . Both configuration files and schema files can be written in any formats anyconfig supports.
- anyconfig.gen_schema() to generate a object represents JSON schema for given configuration file[s] to validate it/them later.

python-anyconfig enables to load configuration files in various formats in the
same manner without taking care of each file format in some cases like the
followings:

.. code-block:: python
import anyconfig
# Config type (format) is automatically detected by filename (file
# extension) in some cases.
conf1 = anyconfig.load("/path/to/foo/conf.d/a.yml")
# Loaded config data is a dict-like object, for example:
#
# conf1["a"] => 1
# conf1["b"]["b1"] => "xyz"
# conf1["c"]["c1"]["c13"] => [1, 2, 3]
# Or you can specify the format (config type) explicitly if automatic
# detection may not work.
conf2 = anyconfig.load("/path/to/foo/conf.d/b.conf", "yaml")
# Specify multiple config files by the list of paths. Configurations of each
# files are merged.
conf3 = anyconfig.load(["/etc/foo.d/a.json", "/etc/foo.d/b.json"])
# Similar to the above but all or one of config file[s] is/are missing:
conf4 = anyconfig.load(["/etc/foo.d/a.json", "/etc/foo.d/b.json"],
ignore_missing=True)
# Specify config files by glob path pattern:
conf5 = anyconfig.load("/etc/foo.d/*.json")
# Similar to the above, but parameters in the former config file will be simply
# overwritten by the later ones:
conf6 = anyconfig.load("/etc/foo.d/*.json", merge=anyconfig.MS_REPLACE)
Also, python-anyconfig can process configuration files which are actually
`jinja2-based template <http://jinja.pocoo.org>`_ files:

- Can generate an actual configuration from prepared half-baked config files template with given context to render the template later.
- It's possible to 'include' config files from given config files your applications with using jinja2's 'include' directive.

.. code-block:: console
In [1]: import anyconfig
In [2]: open("/tmp/a.yml", 'w').write("a: {{ a|default('aaa') }}\n")
In [3]: anyconfig.load("/tmp/a.yml", ac_template=True)
Out[3]: {'a': 'aaa'}
In [4]: anyconfig.load("/tmp/a.yml", ac_template=True, ac_context=dict(a='bbb'))
Out[4]: {'a': 'bbb'}
In [5]: open("/tmp/b.yml", 'w').write("{% include 'a.yml' %}\n") # 'include'
In [6]: anyconfig.load("/tmp/b.yml", ac_template=True, ac_context=dict(a='ccc'))
Out[6]: {'a': 'ccc'}
And python-anyconfig also enables to validate configuration files in various
format with using JSON schema like the followings:

.. code-block:: python
# Validate a JSON config file (conf.json) with JSON schema (schema.yaml).
# If validatation suceeds, `rc` -> True, `err` -> ''.
conf1 = anyconfig.load("/path/to/conf.json")
schema1 = anyconfig.load("/path/to/schema.yaml")
(rc, err) = anyconfig.validate(conf1, schema1) # err should be empty if success (rc == 0)
# Validate a config file (conf.yml) with JSON schema (schema.yml) while
# loading the config file.
conf2 = anyconfig.load("/a/b/c/conf.yml", ac_schema="/c/d/e/schema.yml")
# Validate config loaded from multiple config files with JSON schema
# (schema.json) while loading them.
conf3 = anyconfig.load("conf.d/*.yml", ac_schema="/c/d/e/schema.json")
# Generate jsonschema object from config files loaded.
conf4 = anyconfig.load("conf.d/*.yml")
scm4 = anyconfig.gen_schema(conf4)
scm4_s = anyconfig.dumps(scm4, "json")
And in the last place, python-anyconfig provides a CLI tool called
anyconfig_cli to process configuration files and:

- Convert a/multiple configuration file[s] to another configuration files in different formats
- Get configuration value in a/multiple configuration file[s]
- Validate configuration file[s] with JSON schema
- Generate JSON schema from given configuration file[s]

.. [#] This name took an example from the 'anydbm' python standard library.
.. [#] http://json-schema.org
Supported configuration formats
--------------------------------

python-anyconfig supports various (configuration) file formats if the required
module is available and the corresponding backend is ready to use:

.. csv-table:: Supported formats
:header: "Format", "Type", "Required", "Notes"
:widths: 10, 10, 30, 40

JSON, json, ``json`` (standard lib) or ``simplejson`` [#]_, Enabled by default.
Ini-like, ini, ``configparser`` (standard lib), Enabled by default.
YAML, yaml, ``PyYAML`` [#]_, Enabled automatically if the requirement is satisfied.
XML, xml, ``lxml`` [#]_ or ``ElementTree`` (experimental), Likewise.
ConifgObj, configobj, ``configobj`` [#]_, Likewise.
MessagePack, msgpack, ``msgpack-python`` [#]_, Likewise.

The supported formats (types) of python-anyconfig on your system is able to be
listed by 'anyconfig_cli -L' like this:

.. code-block:: console
$ anyconfig_cli -L
Supported config types: configobj, ini, json, xml, yaml
$
or with the API 'anyconfig.list_types()' will show them:

.. code-block:: console
In [8]: anyconfig.list_types()
Out[8]: ['configobj', 'ini', 'json', 'xml', 'yaml']
In [9]:
python-anyconfig utilizes plugin mechanism provided by setuptools [#]_ and
other formats may be supported by corresponding pluggale backends (see the next
sub section also) like Java properties format.

- Java properties file w/ pyjavaproperties [#]_ (experimental):

- https://github.com/ssato/python-anyconfig-pyjavaproperties-backend

.. [#] https://pypi.python.org/pypi/simplejson
.. [#] https://pypi.python.org/pypi/PyYAML
.. [#] https://pypi.python.org/pypi/lxml
.. [#] https://pypi.python.org/pypi/configobj
.. [#] https://pypi.python.org/pypi/msgpack-python
.. [#] http://peak.telecommunity.com/DevCenter/setuptools#dynamic-discovery-of-services-and-plugins
.. [#] https://pypi.python.org/pypi/pyjavaproperties
Installation
-------------

Requirements
^^^^^^^^^^^^^^

Many runtime dependencies are resolved dynamically and python-anyconfig just
disables specific features if required dependencies are not satisfied.
Therefore, only python standard library is required to install and use
python-anyconfig at minimum.

The following packages need to be installed along with python-anycofig to
enable the features.

.. csv-table::
:header: "Feature", "Requirements", "Notes"
:widths: 20, 10, 25

YAML load/dump, PyYAML, none
ConifgObj load/dump, configobj, none
MessagePack load/dump, msgpack-python, none
Template config, Jinja2, none
Validation with JSON schema, jsonschema [#]_ , Not required to generate JSON schema.

.. [#] https://pypi.python.org/pypi/jsonschema
How to install
^^^^^^^^^^^^^^^^

There is a couple of ways to install python-anyconfig:

- Binary RPMs:

If you're Fedora or Red Hat Enterprise Linux user, you can install
RPMs from the copr repository,
http://copr.fedoraproject.org/coprs/ssato/python-anyconfig/.

- PyPI: You can install python-anyconfig from PyPI with using pip:

.. code-block:: console
$ pip install anyconfig
- Build RPMs from source: It's easy to build python-anyconfig with using rpm-build and mock:

.. code-block:: console
$ python setup.py srpm && mock dist/python-anyconfig-<ver_dist>.src.rpm
or:

.. code-block:: console
$ python setup.py rpm
and install built RPMs.

- Build from source: Of course you can build and/or install python modules in usual way such like 'python setup.py bdist', 'pip install git+https://github.com/ssato/python-anyconfig/' and so on.

Hack
=======

Help wanted!
--------------

These areas are still insufficient, I guess.

- Make python-anyconfig robust for invalid inputs
- Documentation:

- Especially API docs need more love! CLI doc is non-fulfilling.
- English is not my native lang and there are many wrong and hard-to-understand expressions.

Any feedbacks, helps, suggestions are welcome! Please open issues on github.com
site if you have any problems on anyconfig!

How to write backend plugin modules
-------------------------------------

Backend class must inherit anyconfig.backend.Parser and need some member
variables and method ('load_impl' and 'dumps_impl' at minimum) implementations.

JSON and YAML backend modules (anyconfig.backend.{json,yaml}_) should be good
examples to write backend modules, I think.

Also, please take a look at some example backend plugin modules mentioned in
the `Supported configuration formats`_ section.

How to test
-------------

Try to run '[WITH_COVERAGE=1] ./pkg/runtest.sh [path_to_python_code]' or 'tox'.

About test-time requirements, please take a look at pkg/test_requirements.txt.
.. include:: introduction.rst

.. vim:sw=2:ts=2:et:
Loading

0 comments on commit 4622346

Please sign in to comment.