Added Flask Daemon to handyrep. Debugged a auto-failover extensively.

CONTRIBUTORS.txt

@@ -2,18 +2,18 @@ HandyRep has been made possible through the contributions of:
 
 The PostgreSQL Experts Team, including:
 
-Joshua Berkus
-Christophe Pettus
-Jeff Frost
-Quinn Weaver
+* Joshua Berkus
+* Christophe Pettus
+* Jeff Frost
+* Quinn Weaver
 
 Other Contributors include:
 
-
+* Zonar Systems
 
 A Special Thanks to the following people, who helped make HandyRep possible:
 
-The helpful folks on the #python channel on irc.freenode.net
-Jeff Forcier of the Fabric project
-The contributors to the PostgreSQL Project
-The Vagrant and VirtualBox teams for awesome testing software
+* The helpful folks on the #python channel on irc.freenode.net
+* Jeff Forcier of the Fabric project
+* The contributors to the PostgreSQL Project
+* The Vagrant and VirtualBox teams for awesome testing software

docs/api.rst

@@ -1,12 +1,23 @@
 HandyRep Library API
 ====================
 
+Web vs. Library Access
+----------------------
+
+The API below is the full library API for importing HandyRep as a class.
+The vast majority of these functions are also available via the web API, with
+identical parameters as described.  A few functions are marked as
+"Not available in web API".
+
 General
 =======
 
 Initialization
 --------------
 
+If calling HandyRep as a library, it needs to be initialized.  In the
+web services version, this is the reason for the daemonfunctions.py module.
+
 ::
     HandyRep
         config_file filename default 'handyrep.conf'
@@ -26,7 +37,8 @@ is malformatted.
 Return Types
 ------------
 
-All APIs return dictionaries, for easy conversion to JSON and other formats.  Most of the
+All APIs return dictionaries, for easy conversion to JSON and other formats.  In the
+web API, these are returned as JSON but have identical layout. Most of the
 time, this return is in the format of the "return dictionary", or RD:
 
 ::
@@ -214,6 +226,48 @@ numlines
     how many lines of the log to retrieve
 
 
+get_setting
+-----------
+
+Retrieves a single configuration setting.
+
+::
+
+    get_setting
+        setting_name string OR list
+
+setting_name
+    can be either a string for a single setting, or a list which is the full path
+    to that setting in conf.  If just a string, the setting is assumed to be in the
+    "handyrep" section.
+
+Note: the Web API has a slightly different version of this function, which can't return
+settings from nested sections:
+
+::
+
+    get_setting
+        category string default "handyrep"
+        setting string
+
+category
+    section of the config
+
+setting
+    the individual setting name
+
+In either case, get_setting will refuse to return anything in the "passwords" section.
+
+set_verbose
+-----------
+
+Toggles verbose logging.
+
+::
+
+    set_verbose
+        verbose boolean default True
+
 
 Availability API
 ================
@@ -237,6 +291,9 @@ Wraps most of the other availability functions.  Updates the status
 dictionary.  Performs an auto_failover if a failover is required,
 and if auto_failover is configured.
 
+This function is not available through the Web API, is it is invoked only
+by the Periodic portion of that daemon.
+
 ::
 
     failover_check
@@ -264,7 +321,8 @@ handyrep.conf.  Generally one runs with verify=False more frequently (the poll_i
 poll_master
 -----------
 
-Uses the configured polling method to check the master for availability.  Updates the status dictionary in the process.  Can only determine up/down,
+Uses the configured polling method to check the master for availability.
+Updates the status dictionary in the process.  Can only determine up/down,
 and cannot determine if the master has issues; as a result, will not
 change "warning" to "healthy".  Also checks that the master is actually
 a master and not a replica.
@@ -282,16 +340,17 @@ FAIL:
     current master is not responding to polling, or the handyrep or polling
     method configuration is wrong
 
-poll_server
------------
+poll
+----
 
-Uses the configured polling method to check the designated server for availability.  Updates the status dictionary in the process.  Can only determine up/down,
+Uses the configured polling method to check the designated server for availability.
+Updates the status dictionary in the process.  Can only determine up/down,
 and cannot determine if the master has issues; as a result, will not
 change "warning" to "healthy".
 
 ::
 
-    poll_server
+    poll
         servername
 
 Returns RD
@@ -334,6 +393,8 @@ Checks the master server to make sure it's fully operating, including checking
 that we can connect, we can write data, and that ssh and control commands
 are available.  Updates the status dictionary.
 
+Not available in web API, see "verify_server" below.
+
 ::
 
     verify_master
@@ -363,6 +424,8 @@ Checks that the replica is running and is in replication.  Also checks
 that we can connect to the database and that we have a working
 control connection for the server.  Uses the replication_status plugin.  Updates the status dictionary.
 
+Not available in web API, see "verify_server" below.
+
 ::
 
     verify_replica
@@ -392,6 +455,8 @@ verify_server
 Shell function for verify_replica and verify_master, which checks the role
 of the server and then runs the appropriate check.
 
+This function is available through the Web API as the way to access verify_master or verify_replica.
+
 verify_all
 ----------
 
@@ -669,22 +734,58 @@ information about servers once servers.save has been created.
 
     alter_server_def
         servername ServerName
-        kwargs
+        serverprops
 
 servername
     The existing server whose details are to be changed.
 
-kwargs
+serverprops
     a set of key-value pairs for settings to change.  Settings
     may be "changed" to the existing value, so it is permissible
     to pass in an entire dictionary of the server config with
     one changed setting.
 
-Returns RD with extra fields
+Note that serverprops may need special handling in the Web API, since URLs only
+accept strings.  See the Web Service Daemon documentation.
+
+Returns RD with extra fields:
 
 definition
     the resulting new definition for the server
 
+add_server
+----------
+
+Change details of a server after initialization.  Required
+because the .conf file is not considered the canonical
+information about servers once servers.save has been created.
+
+::
+
+    add_server
+        servername string
+        serverprops
+
+servername
+    The existing server whose details are to be changed.
+
+serverprops
+    a set of key-value pairs for server settings.  Note that the
+    hostname setting is required, and the new server will be
+    rejected without it.
+
+Note that serverprops may need special handling in the Web API, since URLs only
+accept strings.  See the Web Service Daemon documentation.
+
+New servers are always added as "disabled", and need to be enabled as a
+separate action.  Further, "role" defaults to "replica" if not supplied.
+
+Returns RD with extra fields:
+
+definition
+    the resulting new definition for the server
+
+
 clean_archive
 -------------
 
@@ -714,7 +815,7 @@ connection_proxy_init
 ---------------------
 
 Set up the connection proxy configuration according to the configured
-connection failover plugin.  Not all plugins support initialization.
+connection failover plugin.  Not all connection proxy plugins support initialization.
 
 ::
 
@@ -731,10 +832,6 @@ FAIL
     initialization
 
 
-
-
-
-
 
 
 

docs/setup.rst

@@ -10,9 +10,12 @@ HandyRep is a Python application which has the following module dependencies:
 * fabric 1.8.0+
 * paramiko
 * jinja2
-* uWSGI
 * psycopg2 2.5+
 
+Plus, if you are using the Daemon:
+
+* flask 0.8+
+
 Plus these general packages:
 
 * postgresql client software
@@ -122,27 +125,12 @@ handyrep_table
 handyrep_user
     User handyrep uses when updating its own status data.
 
-handyrep_db_pass
-    Password for that user, if required.
-
 postgres_superuser
     Name of the PostgreSQL superuser.  Usually "postgres".
-
-superuser_pass
-    Password for same.
 
 replication_user
     Name of the user used for streaming replication connections.  Often the same as the superuser.
 
-replication_pass
-    Password for same.
-
-admin_password
-    API password for HandyRep administration rights in the REST interface.
-
-read_password
-    API password for HandyRep read-only rights.
-
 templates_dir
     Directory where the templates are stored.
 
@@ -152,6 +140,25 @@ test_ssh_command
 push_alert_method
     If we are pushing alerts to the monitoring system, the name of the plugin used to do that.  If left blank, HandyRep will not attempt to push alerts.
 
+Section passwords
+-----------------
+
+handyrep_db_pass
+    Password for the handyrep database user, if required.
+
+superuser_pass
+    Password for "postgres".
+
+replication_pass
+    Password for replication user
+
+admin_password
+    API password for HandyRep administration rights in the REST interface.
+
+read_password
+    API password for HandyRep read-only rights.
+
+
 Section failover
 ----------------
 
@@ -164,8 +171,8 @@ poll_method
 poll_interval
     Number of seconds between polling all of the servers.
 
-verify_interval
-    Number of sections between running a full verify on all servers.
+verify_frequency
+    Do a full verify after this number of polling cycles.
 
 fail_retries
     If polling or other connections to a server fails, how many times should HandyRep keep trying to connect before declaring failure?
@@ -295,13 +302,14 @@ hostname
     The DNS name or IP address of the server.
 
 role
-    The role of the server in replication.  Options include master, replica, archive, and proxy, but any label can be used.  HandyRep cares only about "master" and "replica"; other labels are there for administrator information only.
+    The role of the server in replication.  Options include master, replica, archive, and proxy, but any label can be used.  HandyRep cares only about "master" and "replica"; other labels are there for administrator information only,
+    or to support certain plugins (such as "multi_pgbouncer"), or for archiving.
 
 failover_priority
     The priority of this server to be the new master in a failover event, if using the select_by_priority method, or if breaking ties with other methods.  Lower numbers are chosen first.
 
 enabled
-    Is this server enabled for replication?  Note that non-database servers will be marked as "False" even though they may be used for some other purpose (i.e. archive storage).
+    Is this server enabled for replication?  Note that non-database servers may be marked as "False" even though they may be used for some other purpose (i.e. archive storage).
 
 In addition to the above, each server definition may change any of the various settings in server_defaults.
 

docs/usage.rst

@@ -6,9 +6,10 @@ HandyRep is a RESTful network service or library which manages status monitoring
 Daemon Usage
 ------------
 
-HandyRep ships with the HandyRep WSGI Daemon, which is the standard way to use HandyRep.  This Daemon runs constantly on the HandyRep server, performing scheduled checks and tasks, and responding to GET and POST requests for status information and to perform manual actions (such as cloning a new replica).  In this mode, HandyRep supplies a service-oriented interface for managing your database cluster; you run the daemon and whenever you need to make changes to the cluster, you send it commands.
+HandyRep ships with the HandyRep Flask Daemon, which is the standard way to use HandyRep.  This Daemon runs constantly on the HandyRep server, performing scheduled checks and tasks, and responding to GET and POST requests for status information and to perform manual actions (such as cloning a new replica).  In this mode, HandyRep supplies a service-oriented interface for managing your database cluster; you run the daemon and whenever you need to make changes to the cluster, you send it commands.
 
-Note that the Daemon is single-process; HandyRep does not currently do any multiprocess activity.
+Note that the Daemon is single-process; HandyRep does not currently do any multiprocess activity.  As such, the web server
+you are using as a container for the Daemon needs to support single-process configuration.
 
 Library Usage
 -------------
@@ -32,8 +33,8 @@ HandyRep is designed around the following assumptions about your PostgreSQL clus
 
 If you are doing log archiving, HandyRep also makes the following assumptions:
 
-* Only one (network or mounted) location for archiving
-* Assumes that all replicas can access the same archive
+* Only one (network or relative mounted) location for archiving
+* Assumes that all replicas can access the same (relative) archive
 * Assumes that all nodes can use the same archiving scripts, if pushing scripts is enabled
 * Assumes that the postgres user does all archive work
 
@@ -98,8 +99,4 @@ HandyRep maintains status information about the master, the replicas, and the ov
 Log
 ---
 
-HandyRep keeps a log of all of its actions, at a location configured in handyrep.conf.  The tail of this log is available via the API so that you can check recent actions including failover.
-
-
-
-
+HandyRep keeps a log of all of its actions, at a location configured in handyrep.conf.  The tail of this log is available via the API so that you can check recent actions including failover.