Skip to content
YAML config loader with environment variables interpolation for Python and command line
Python Makefile
Branch: master
Clone or download
Latest commit 0d9a602 Jun 27, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Add feautere request template (#20) Jun 17, 2019
docs Documentation updated (#34) Jun 27, 2019
piny Piny CLI tool added (#35) Jun 26, 2019
tests Piny CLI tool added (#35) Jun 26, 2019
.gitignore Initial commit Jun 7, 2019
.pyup.yml First alpha version Jun 7, 2019
.readthedocs.yml Fix Sphinx config (#28) Jun 18, 2019
.travis.yml CI config secrets fixes, README updated Jun 7, 2019
CHANGELOG.rst Documentation updated (#34) Jun 27, 2019
CONTRIBUTING.rst
LICENSE First alpha version Jun 7, 2019
MANIFEST.in First alpha version Jun 7, 2019
Makefile Docs and artwork added (#12, #6) Jun 18, 2019
README.rst Piny CLI tool added (#35) Jun 26, 2019
requirements.txt Piny CLI tool added (#35) Jun 26, 2019
setup.cfg First alpha version Jun 7, 2019
setup.py
version.py Fix Sphinx release variable (#30) Jun 18, 2019

README.rst

Piny

Piny logo

Build Codecov Black Formatter PyPI - Python Version MIT License

Piny is YAML config loader with environment variables interpolation for Python.

Keep your app's configuration in a YAML file. Mark up sensitive data in the config as environment variables. Set environment variables on application deployment. Now let the Piny load your config and substitute environment variables in it with their values.

Piny is developed with Docker and Kubernetes in mind, though it's not limited to any deployment system.

Rationale

Piny combines readability and versioning you get when using config files, and security that environment variables provide. Read more about this approach in the blog post.

Help

See documentation for more details.

Installation

Just run:

pip install -U piny

Usage

Set your environment variables, add them to your YAML configuration file:

db:
  login: user
  password: ${DB_PASSWORD}
mail:
  login: user
  password: ${MAIL_PASSWORD:-my_default_password}
sentry:
  dsn: ${VAR_NOT_SET}

Then load your config:

from piny import YamlLoader

config = YamlLoader(path="config.yaml").load()
print(config)
# {'db': {'login': 'user', 'password': 'my_db_password'},
# 'mail': {'login': 'user', 'password': 'my_default_password'},
# 'sentry': {'dsn': None}}

You may want to discourage Bash-style envs with defaults in your configs. In such case, use a StrictMatcher:

from piny import YamlLoader, StrictMatcher

config = YamlLoader(path="config.yaml", matcher=StrictMatcher).load()

Both strict and default matchers produce None value if environment variable matched is not set in the system (and no default syntax used in the case of default matcher).

Piny also comes with command line utility that works both with files and standard input and output:

$ export PASSWORD=mySecretPassword
$ echo "db: \${PASSWORD}" | piny
db: mySecretPassword

Validation

Piny supports optional data validation using third-party libraries: Marshmallow, Pydantic, Trafaret.

import marshmallow as ma
from piny import MarshmallowValidator, StrictMatcher, YamlLoader

class DBSchema(ma.Schema):
    login = ma.fields.String(required=True)
    password = ma.fields.String()

class ConfigSchema(ma.Schema):
    db = ma.fields.Nested(DBSchema)

config = YamlLoader(
    path="database.yaml",
    matcher=StrictMatcher,
    validator=MarshmallowValidator,
    schema=ConfigSchema,
    strict=True
).load(many=False)

Exceptions

LoadingError is thrown when something goes wrong with reading or parsing YAML-file. ValidationError is a wrapper for exceptions raised by the libraries for optional data validation. Original exception can be accessed by origin attribute. It comes in handy when you need more than just an original exception message (e.g. a dictionary of validation errors).

Both exceptions inherit from the ConfigError.

Best practices

  • Maintain a healthy security/convenience balance for your config
  • Mark up entity as an environment variable in your YAML if and only if it really is a secret (login/passwords, private API keys, crypto keys, certificates, or maybe DB hostname too? You decide)
  • When loading config file, validate your data. Piny supports a few popular data validation tools.
  • Store your config files in the version control system along with your app’s code.
  • Environment variables are set by whoever is responsible for the deployment. Modern orchestration systems like Kubernetes make it easier to keep envs secure (see Kubernetes Secrets).

Fun facts

Piny is a recursive acronym for Piny Is Not YAML. Not only it's a library name, but also a name for YAML marked up with environment variables.

Contributing

See CONTRIBUTING.rst.

You can’t perform that action at this time.