A template for PostgreSQL High Availability with ZooKeeper, etcd, or Consul
Python Gherkin Shell
Latest commit 8aa173e Jan 18, 2017 @alexeyklyukin alexeyklyukin committed on GitHub Bump up to 1.2.3 (#386)
Permalink
Failed to load latest commit information.
docker Volume could be passed to the docker only with absolute path Jul 1, 2016
docs Release notes for version 1.2 Jan 5, 2017
extras Add haproxy confd templates Jun 17, 2016
features Scheduled failovers in 1 second don't work reliably with loop_wait=2 Jan 13, 2017
patroni Bump up to 1.2.3 (#386) Jan 18, 2017
tests Don't resolve url hostnames manualy but mokey patch urllib3 (#385) Jan 18, 2017
.gitignore Make patroni documentation available on patroni.readthedocs.io. (#373) Dec 20, 2016
.travis.yml Switch to trusty and run acceptance tests with postgres 9.6 Jan 13, 2017
.zappr.yaml Add .zappr.yaml Jul 18, 2016
Dockerfile Add postgres-9.6 support (#357) Nov 25, 2016
LICENSE Rename LICENCE to LICENSE Feb 17, 2016
MAINTAINERS Improve quality of code by resolving issues found by quantifiedcode a… Feb 12, 2016
MANIFEST.in Refactoring to enable package building and succesfull installation of… Sep 3, 2015
README.rst Mention readthedocs Dec 20, 2016
TODO.md Remove the done items May 30, 2016
docker-compose.yml Set the standard name for the docker-compose file. Jul 5, 2016
haproxy.cfg Handle haproxy requests. Improve failover status code. Nov 12, 2015
mkbinary.sh Try to get rid from hardcoded names when building binary Sep 5, 2016
patroni.py Add patroni.py executable file Sep 7, 2015
patroni.spec Try to get rid from hardcoded names when building binary Sep 5, 2016
patronictl.py Improve quality of code by resolving issues found by quantifiedcode a… Feb 12, 2016
postgres0.yml Introduce starting state and master start timeout. (#295) Dec 8, 2016
postgres1.yml Add post_init configuration parameter on bootstrap (#296) Sep 28, 2016
postgres2.yml Make sure that we are running and testing latest versions of everythi… Sep 19, 2016
release.sh Tag on github is prefixed with v. Sep 8, 2015
requirements.txt Don't resolve url hostnames manualy but mokey patch urllib3 (#385) Jan 18, 2017
setup.py Fix flake8 warnings, release 1.2 (#364) Dec 13, 2016
tox.ini Make sure that we are running and testing latest versions of everythi… Sep 19, 2016

README.rst

Build Status Coverage Status

Patroni: A Template for PostgreSQL HA with ZooKeeper, etcd or Consul

You can find a version of this documentation that is searchable and also easier to navigagate at patroni.readthedocs.io.

There are many ways to run high availability with PostgreSQL; for a list, see the PostgreSQL Documentation.

Patroni is a template for you to create your own customized, high-availability solution using Python and - for maximum accessibility - a distributed configuration store like ZooKeeper, etcd or Consul. Database engineers, DBAs, DevOps engineers, and SREs who are looking to quickly deploy HA PostgreSQL in the datacenter-or anywhere else-will hopefully find it useful.

We call Patroni a "template" because it is far from being a one-size-fits-all or plug-and-play replication system. It will have its own caveats. Use wisely.

Note to Kubernetes users: We're currently developing Patroni to be as useful as possible for teams running Kubernetes on top of Google Compute Engine; Patroni can be the HA solution for Postgres in such an environment. To this end, we've created a Helm Chart that enables you to deploy a five-node Patroni cluster using a Kubernetes PetSet.

How Patroni Works

Patroni originated as a fork of Governor, the project from Compose. It includes plenty of new features.

For an example of a Docker-based deployment with Patroni, see Spilo, currently in use at Zalando.

For additional background info, see:

Development Status

Patroni is in active development and accepts contributions. See our Contributing section below for more details.

We report new releases information here.

Technical Requirements/Installation

For Mac

To install requirements on a Mac, run the following:

brew install postgresql etcd haproxy libyaml python
pip install psycopg2 pyyaml

Running and Configuring

To get started, do the following from different terminals:

> etcd --data-dir=data/etcd
> ./patroni.py postgres0.yml
> ./patroni.py postgres1.yml

You will then see a high-availability cluster start up. Test different settings in the YAML files to see how the cluster's behavior changes. Kill some of the components to see how the system behaves.

Add more postgres*.yml files to create an even larger cluster.

Patroni provides an HAProxy configuration, which will give your application a single endpoint for connecting to the cluster's leader. To configure, run:

> haproxy -f haproxy.cfg
> psql --host 127.0.0.1 --port 5000 postgres

YAML Configuration

Go here for comprehensive information about settings for etcd, consul, and ZooKeeper. And for an example, see postgres0.yml.

Environment Configuration

Go here for comprehensive information about configuring(overriding) settings via environment variables.

Replication Choices

Patroni uses Postgres' streaming replication, which is asynchronous by default. Patroni's asynchronous replication configuration allows for maximum_lag_on_failover settings. This setting ensures failover will not occur if a follower is more than a certain number of bytes behind the leader. This setting should be increased or decreased based on business requirements. It's also possible to use synchronous replication for better durability guarantees. See replication modes documentation <https://github.com/zalando/patroni/blob/master/docs/replication_modes.rst> for details.

Applications Should Not Use Superusers

When connecting from an application, always use a non-superuser. Patroni requires access to the database to function properly. By using a superuser from an application, you can potentially use the entire connection pool, including the connections reserved for superusers, with the superuser_reserved_connections setting. If Patroni cannot access the Primary because the connection pool is full, behavior will be undesirable.