Improve documentation thanks to user feedback. (#909)

README.md

@@ -1,6 +1,5 @@
 # pg_auto_failover
 
-[![Slack Status](http://slack.citusdata.com/badge.svg)](https://slack.citusdata.com)
 [![Documentation Status](https://readthedocs.org/projects/pg-auto-failover/badge/?version=master)](https://pg-auto-failover.readthedocs.io/en/master/?badge=master)
 
 pg_auto_failover is an extension and service for PostgreSQL that monitors
@@ -64,6 +63,17 @@ the packages from there.
 
 ### Ubuntu or Debian:
 
+Binary packages for debian and derivatives (ubuntu) are available from
+`apt.postgresql.org`__ repository, install by following the linked
+documentation and then::
+
+```bash
+$ sudo apt-get install pg-auto-failover-cli
+$ sudo apt-get install postgresql-14-auto-failover
+```
+
+__ https://wiki.postgresql.org/wiki/Apt
+
 When using debian, two packages are provided for pg_auto_failover: the
 monitor Postgres extension is packaged separately and depends on the
 Postgres version you want to run for the monitor itself. The monitor's
@@ -74,15 +84,17 @@ Then another package is prepared that contains the `pg_autoctl` command, and
 the name of the package is `pg-auto-failover-cli`. That's the package that
 is needed for the Postgres nodes.
 
-```bash
-# Add the repository to your system
-curl https://install.citusdata.com/community/deb.sh | sudo bash
+To avoid debian creating a default Postgres data directory and service,
+follow these steps before installing the previous packages.
 
-# Install pg_auto_failover
-sudo apt-get install postgresql-11-auto-failover
+```bash
+$ curl https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
+$ echo "deb http://apt.postgresql.org/pub/repos/apt buster-pgdg main" > /etc/apt/sources.list.d/pgdg.list
 
-# Confirm installation
-/usr/bin/pg_autoctl --version
+# bypass initdb of a "main" cluster
+$ echo 'create_main_cluster = false' | sudo tee -a /etc/postgresql-common/createcluster.conf
+$ apt-get update
+$ apt-get install -y --no-install-recommends postgresql-14
 ```
 
 ### Fedora, CentOS, or Red Hat:
@@ -193,6 +205,8 @@ Once the building and installation is done, follow those steps:
   4. See the state of the new system:
 
      ~~~ bash
+     $ export PGDATA=./monitor
+     $ export PGPORT=5000
 	 $ pg_autoctl show state
 	   Name |  Node |      Host:Port |       LSN | Reachable |       Current State |      Assigned State
      -------+-------+----------------+-----------+-----------+---------------------+--------------------

docs/failover-state-machine.rst

@@ -1,3 +1,5 @@
+.. _failover_state_machine:
+
 Failover State Machine
 ======================
 

docs/faq.rst

@@ -96,6 +96,33 @@ When things are not obvious, the next step is to go read the logs. Both the
 output of the ``pg_autoctl`` command and the Postgres logs are relevant. See
 the :ref:`logs` question for details.
 
+Impossible / unresolveable state after crash - How to recover?
+--------------------------------------------------------------
+
+The pg_auto_failover :ref:`failover_state_machine` is great to simplify node
+management and orchestrate multi-nodes operations such as a switchover or a
+failover. That said, it might happen that the FSM is unable to proceed in
+some cases, usually after a hard crash of some components of the system, and
+mostly due to bugs.
+
+Even if we have an extensive test suite to prevent such bugs from happening,
+you might have to deal with a situation that the monitor doesn't know how to
+solve.
+
+The FSM has been designed with a last resort operation mode. It is always
+possible to unregister a node from the monitor with the
+:ref:`pg_autoctl_drop_node` command. This helps the FSM getting back to a
+simpler situation, the simplest possible one being when only one node is
+left registered in a given formation and group (state is then SINGLE).
+
+When the monitor is back on its feet again, then you may add your nodes
+again with the :ref:`pg_autoctl_create_postgres` command. The command
+understands that a Postgres service is running and will recover from where
+you left.
+
+In some cases you might have to also delete the local pg_autoctl state file,
+error messages will instruct you about the situation.
+
 The monitor is a SPOF in pg_auto_failover design, how should we handle that?
 ----------------------------------------------------------------------------
 

docs/index.rst

@@ -6,6 +6,40 @@
 Welcome to pg_auto_failover's documentation!
 ============================================
 
+The pg_auto_failover project is an Open Source Software project. The
+development happens at `https://github.com/citusdata/pg_auto_failover`__ and
+is public: everyone is welcome to participate by opening issues or pull
+requests, giving feedback, etc.
+
+Remember that the first steps are to actually play with the pg_autoctl
+command, then read the entire available documentation (after all, I took the
+time to write it), and then to address the community in a kind and polite
+way — the same way you would expect people to use when addressing you.
+
+__ https://github.com/citusdata
+
+.. note::
+
+   The development of pg_auto_failover has been driven by Citus Data, since
+   then a team at Microsoft. The Citus Data team at Microsoft generously
+   maintains the pg_auto_failover Open Source Software so that its users may
+   continue using it in production.
+
+   The pg_auto_failover Open Source project does not include support for the
+   Citus extension. Citus support in pg_auto_failover has not been Open
+   Sourced at this time.
+
+   For enhancements, improvements, and new features, consider contributing
+   to the project. Pull Requests are reviewed as part of the offered
+   maintenance.
+
+.. note::
+
+   Assistance is provided as usual with Open Source projects, on a voluntary
+   basis. If you need help to cook a patch, enhance the documentation, or
+   even to use the software, you're welcome to ask questions and expect some
+   level of free guidance.
+
 .. toctree::
    :maxdepth: 4
    :caption: Contents:

docs/install.rst

@@ -3,7 +3,8 @@
 Installing pg_auto_failover
 ===========================
 
-We provide native system packages for pg_auto_failover on most popular Linux distributions.
+We provide native system packages for pg_auto_failover on most popular Linux
+distributions.
 
 Use the steps below to install pg_auto_failover on PostgreSQL 11. At the
 current time pg_auto_failover is compatible with both PostgreSQL 10 and
@@ -12,50 +13,48 @@ PostgreSQL 11.
 Ubuntu or Debian
 ----------------
 
-Quick install
-~~~~~~~~~~~~~
-
-The following installation method downloads a bash script that automates
-several steps. The full script is available for review at our `package cloud
-installation instructions`__ page.
-
-__ https://packagecloud.io/citusdata/community/install#bash
-
-.. code-block:: bash
-
-  # add the required packages to your system
-  curl https://install.citusdata.com/community/deb.sh | sudo bash
-
-  # install pg_auto_failover
-  sudo apt-get install postgresql-11-auto-failover
-
-  # confirm installation
-  /usr/bin/pg_autoctl --version
+Postgres apt repository
+~~~~~~~~~~~~~~~~~~~~~~~
 
-Manual Installation
-~~~~~~~~~~~~~~~~~~~
+Binary packages for debian and derivatives (ubuntu) are available from
+`apt.postgresql.org`__ repository, install by following the linked
+documentation and then::
 
-If you'd prefer to install your repo on your system manually, follow the
-instructions from `package cloud manual installation`__ page. This page will
-guide you with the specific details to achieve the 3 steps:
+  $ sudo apt-get install pg-auto-failover-cli
+  $ sudo apt-get install postgresql-14-auto-failover
 
-__ https://packagecloud.io/citusdata/community/install#manual
+__ https://wiki.postgresql.org/wiki/Apt
 
-  1. install CitusData GnuPG key for its package repository
-  2. install a new apt source for CitusData packages
-  3. update your available package list
+The Postgres extension named "pgautofailover" is only necessary on the
+monitor node. To install that extension, you can install the
+``postgresql-14-auto-failover`` package when using Postgres 14. It's
+available for other Postgres versions too.
 
-Then when that's done, you can proceed with installing pg_auto_failover
-itself as in the previous case:
+Avoiding the default Postgres service
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-.. code-block:: bash
+When installing the debian Postgres package, the installation script will
+initialize a Postgres data directory automatically, and register it to the
+systemd services. When using pg_auto_failover, it is best to avoid that step.
 
-  # install pg_auto_failover
-  sudo apt-get install postgresql-11-auto-failover
+To avoid automated creation of a Postgres data directory when installing the
+debian package, follow those steps:
 
-  # confirm installation
-  /usr/bin/pg_autoctl --version
+::
+   
+  $ curl https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
+  $ echo "deb http://apt.postgresql.org/pub/repos/apt buster-pgdg main" > /etc/apt/sources.list.d/pgdg.list
+  
+  # bypass initdb of a "main" cluster
+  $ echo 'create_main_cluster = false' | sudo tee -a /etc/postgresql-common/createcluster.conf
+  $ apt-get update
+  $ apt-get install -y --no-install-recommends postgresql-14
 
+That way when it's time to :ref:`pg_autoctl_create_monitor` or
+:ref:`pg_autoctl_create_postgres` there is no confusion about how to handle
+the default Postgres service created by debian: it has not been created at
+all.
+
 Fedora, CentOS, or Red Hat
 --------------------------
 

docs/operations.rst

@@ -19,6 +19,23 @@ pg_auto_failover monitor. The ``pg_autoctl create`` command honors the
 running. If Postgres is detected, the new node is registered in SINGLE mode,
 bypassing the monitor's role assignment policy.
 
+Postgres configuration management
+---------------------------------
+
+The :ref:`pg_autoctl_create_postgres` command edits the default Postgres
+configuration file (``postgresql.conf``) to include pg_auto_failover
+settings.
+
+The include directive is placed on the top of the ``postgresql.conf`` file
+in a way that you may override any setting by editing it later in the file.
+
+Unless using the ``--skip-pg-hba`` option then pg_autoctl edits a minimal
+set of HBA rules for you, in order for the pg_auto_failover nodes to be able
+to connect to each other. The HBA rules that are needed for your application
+to connect to your Postgres nodes still need to be added. As pg_autoctl
+knows nothing about your applications, then you are responsible for editing
+the HBA file.
+
 Upgrading pg_auto_failover, from versions 1.4 onward
 -----------------------------------------------------
 

docs/ref/pg_autoctl.rst

@@ -140,3 +140,13 @@ A typical JSON output would be::
 This is for version 1.4.2 of pg_auto_failover. This particular version of
 the pg_autoctl client tool has been compiled using ``libpq`` for PostgreSQL
 12.3 and is compatible with Postgres 10, 11, 12, and 13.
+
+Environment
+-----------
+
+PG_AUTOCTL_DEBUG
+
+  When this environment variable is set (to anything) then ``pg_autoctl``
+  allows more commands. Use with care, this opens abilities to destroy your
+  production clusters.
+

docs/ref/pg_autoctl_config_check.rst

@@ -30,6 +30,35 @@ Options
 
   Output JSON formatted data.
 
+
+Environment
+-----------
+
+PGDATA
+
+  Postgres directory location. Can be used instead of the ``--pgdata``
+  option.
+
+PG_AUTOCTL_MONITOR
+
+  Postgres URI to connect to the monitor node, can be used instead of the
+  ``--monitor`` option.
+
+XDG_CONFIG_HOME
+
+  The pg_autoctl command stores its configuration files in the standard
+  place XDG_CONFIG_HOME. See the `XDG Base Directory Specification`__.
+
+  __ https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
+
+XDG_DATA_HOME
+
+  The pg_autoctl command stores its internal states files in the standard
+  place XDG_DATA_HOME, which defaults to ``~/.local/share``. See the `XDG
+  Base Directory Specification`__.
+
+  __ https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
+
 Examples
 --------
 

docs/ref/pg_autoctl_config_get.rst

@@ -28,6 +28,34 @@ Options
 
   Output JSON formatted data.
 
+Environment
+-----------
+
+PGDATA
+
+  Postgres directory location. Can be used instead of the ``--pgdata``
+  option.
+
+PG_AUTOCTL_MONITOR
+
+  Postgres URI to connect to the monitor node, can be used instead of the
+  ``--monitor`` option.
+
+XDG_CONFIG_HOME
+
+  The pg_autoctl command stores its configuration files in the standard
+  place XDG_CONFIG_HOME. See the `XDG Base Directory Specification`__.
+
+  __ https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
+
+XDG_DATA_HOME
+
+  The pg_autoctl command stores its internal states files in the standard
+  place XDG_DATA_HOME, which defaults to ``~/.local/share``. See the `XDG
+  Base Directory Specification`__.
+
+  __ https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
+
 Description
 -----------
 

docs/ref/pg_autoctl_config_set.rst

@@ -28,6 +28,34 @@ Options
 
   Output JSON formatted data.
 
+Environment
+-----------
+
+PGDATA
+
+  Postgres directory location. Can be used instead of the ``--pgdata``
+  option.
+
+PG_AUTOCTL_MONITOR
+
+  Postgres URI to connect to the monitor node, can be used instead of the
+  ``--monitor`` option.
+
+XDG_CONFIG_HOME
+
+  The pg_autoctl command stores its configuration files in the standard
+  place XDG_CONFIG_HOME. See the `XDG Base Directory Specification`__.
+
+  __ https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
+
+XDG_DATA_HOME
+
+  The pg_autoctl command stores its internal states files in the standard
+  place XDG_DATA_HOME, which defaults to ``~/.local/share``. See the `XDG
+  Base Directory Specification`__.
+
+  __ https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
+
 Description
 -----------