Review and tweaking of the entire deployment hierarchy

--HG--
branch : 2.1

docs/main/Deployment/Alternate.rst

@@ -3,80 +3,76 @@
 Alternate Deployment Patterns
 =============================
 
+This document attempts to discuss alternate patterns for deploying
+TurboGears |version|.  It is written with the assumption that you
+have at least read the :ref:`deploy_standard`, as most documents
+will simply discuss the differences from a standard deployment.
+
 .. note::
 
    New developers should likely use the :ref:`deploy_standard` (if possible).
    The choices involved in alternate installations can be daunting if you
    aren't yet familiar with the various components.
 
-Questions to answer for yourself:
-
-* :ref:`deploy_web_server` -- what application will actually accept
-  the http requests from the client?  This is normally a service that must
-  be installed as "root" in order to claim the standard ports (80 or 443)
-* :ref:`deploy_which_database` -- what server will you use to store your
-  model data?
-* :ref:`deploy_source` -- will you deploy with source-code-checkouts, built
-  eggs, whole-application binary checkouts, zc.buildout packages,
-  PIP packages or puppet scripts?
-
 .. _deploy_web_server:
 
 Web-Server Choice
 ------------------
 
-The most common way to install TurboGears |version| for production use
-is to use the `Apache`_ web-server running the mod_wsgi extension
-to host TurboGears in a reasonably performant easily deployed
-and flexible configuration.
-
-There are many other approaches to deploying a production environment,
-and choosing which approach is best for your needs is a non-trivial
-task.
-
-* Apache with mod_wsgi tends to be the "default" choice.  It is widely
-  available, well documented, stable and easily supported by Linux
-  sysadmins in production environments.
-* Apache with mod_proxy or mod_rewrite can be used to have Apache handle
-  "the rest" of your site, while passing only your TurboGears application's
-  requests through to a Paster server running on a non-privileged port.
-* Apache with FastCGI can be used if necessary, such as when you do not have
-  control of your Apache server or a policy requires suexec or the like for
-  all "user" scripts.
-* `Nginx`_ is often preferred by those who need speed above all else
-* The built-in Paster server will occasionablly be used by those
-  who are deploying small internal sites with no more than a handful
-  of users.
-* IIS users may want to experiment with the WSGI support from the
+The web-server, which actually receives and processes HTTP requests
+from clients and converts them to WSGI requests for your TurboGears
+project, can significantly impact the performance and scalability of
+your site.
+
+* :ref:`apache_mod_wsgi` -- the standard way to deploy on Apache
+* :ref:`apache_mod_proxy` -- runs Apache as a front-end with
+  a Paste web-server running on a local port.  Allows you to run
+  the Paste server as *any* user
+* :ref:`FastCGI` -- runs Apache as a front-end with a `FastCGI`
+  process using Mod-Rewrite to make the CGI appear at the correct
+  point in the server's URL-space.
+* `paster serve production.ini` -- while not recommended for large
+  or high-traffic sites, Paste's web-server can often serve for small
+  internal sites with few users.  See :ref:`deploy_daemon` for a
+  discussion of how to keep your server running.
+* :ref:`Nginx` -- an alternative asynchronous high-performance web-server
+  which can reverse-proxy TurboGears
+* :ref:`Light HTTPD <lighttpd_fcgi>` -- has built-in FastCGI support, so can
+  be used to reverse-proxy TurboGears
+* `Twisted Web2`_ -- likely only of interest if you are already using
+  Twisted for your application and simply want to host a TurboGears
+  application within it.  Twisted's WSGI implementation is *not*
+  heavily optimized, so should not be used for high-performance sites.
+* MS-IIS users may want to experiment with the WSGI support from the
   `ISAPI-WSGI`_ project.
 
 .. todo:: document use of `isapi-wsgi`_ with TurboGears
+.. todo:: Difficulty: Hard. Document use of IIS with TurboGears thru a proxy.
 
 .. _`Apache`: http://httpd.apache.org/
-.. _`Nginx`: http://nginx.org/
 .. _`ISAPI-WSGI`: http://code.google.com/p/isapi-wsgi/
 
 .. _deploy_which_database:
 
 Database Choice
 ----------------
 
-Normally users choose either MySQL or PostgreSQL as their production
-database back-end, but Oracle or MSSQL can also be used.  The built-in
-SQLite database should not be used for production sites as a general
-rule.  Obviously if you have used MongoDB/Ming you will need to deploy
-against a MongoDB database instead.
-
-TurboGears 2 provides a solid HTTP server built in, and for many
-internal corporate deployments or low traffic sites you can just fire
-up the TurboGears |version| app and point people at it.
-
-This can be as simple as running::
-
-  paster serve production.ini
-
-But it's also likely that you may want to automatically restart your
-TurboGears |version| app if the server reboots, or you may want to set
-it up as a windows service. Unfortunately these things can be very
-operating system specific, but fortunately they aren't
-TurboGears specific.
+If you are using SQLAlchemy (the default ORM for TurboGears |version|),
+then by-and-large your choice of database back-end is a matter of preference.
+
+* :ref:`deploy_postgresql` -- is a robust, mature, well documented
+  free database server which meets or exceeds most new user's needs.
+* MySQL -- allows you to trade robustness (ACID compliance, for instance)
+  for raw speed and some exotic features that are add-ons for PostgreSQL
+* Oracle -- if your site is an Oracle shop with specialized Oracle admins
+  it may be appropriate to use an Oracle DB for your TurboGears application
+* SQLite -- can be used for *extremely* small sites (e.g. a local web-server
+  intended solely to be used by a single user).  It is *extremely* easy to
+  set up and comes with later versions of Python.
+* MSSQL -- if you are already using MSSQL for your site, and have admins who
+  maintain the servers, it may be appropriate to use MSSQL for your TurboGears
+  application.
+
+.. _`Twisted Web2`: http://blog.vrplumber.com/index.php?/archives/2421-TurboGears-as-a-Twisted-WSGI-Application-in-125-seconds.html
+
+.. todo:: Add section on "repeatable deployment options"

docs/main/Deployment/Checkout.rst

@@ -19,12 +19,17 @@ it using the `develop` option to setup.py.
    $ sudo -u www-data svn checkout file:///var/svn/myapp/production myapp
    $ cd myapp
    $ sudo -u www-data bash
+   $ mkdir python-eggs
    $ source /usr/local/pythonenv/myapp/bin/activate
    $ python setup.py develop
    $ exit
 
 by default :ref:`modwsgi_deploy <deploy_modwsgi_deploy>` will have specified
-that `production.ini` is in the root directory of this checkout.  See
+that `production.ini` is in the root directory of this checkout. See
 :ref:`deploy_ini_scc` for details on why you might **not** want that file to be
 checked into your main repository.
 
+Similarly, you will need to make sure that your Beaker session and cache
+directories are not sub-directories of the source code checkout if you
+are planning on deleting and re-checking-out the source for each release.
+(See :ref:`deploy_ini_beaker` for details).

docs/main/Deployment/Code.rst

@@ -1,7 +1,7 @@
 .. _deploy_code:
 
-Deploying Your Project
-======================
+Deploying Your Project Code
+===========================
 
 There are a number of ways you can deploy your application's code in
 production.  While there is an "officially" standard way to deploy

docs/main/Deployment/DBServer.rst

@@ -1,7 +1,7 @@
 .. _deploy_db:
 
-Deploy a Production Database
-=============================
+Production Database
+===================
 
 Most production sites will use a dedicated database server rather than
 relying on the in-process SQLite engine.  Dedicated servers are generally
@@ -23,20 +23,29 @@ TurboGears.
    use strong passwords for all accounts, *even* if you only expose the
    DB on a "trusted" port.
 
+Either PostgreSQL or MySQL is a good default choice for a database server,
+using either one is considered part of a :ref:`deploy_standard` and should
+"just work".
+
 .. _deploy_postgresql:
 
 PostgreSQL
 -----------
 
-PostgreSQL is a mature, robust, efficient ACID database server.  It
+PostgreSQL is a mature, robust, efficient `ACID`_ database server.  It
 is available for all major platforms, and has GUI administrative tools
-(though almost all "serious" users use the robust command-line tools).
+(though almost all "serious" users use the command-line tools).
+
+.. _`ACID`: http://en.wikipedia.org/wiki/ACID
 
 PostgreSQL is very well packaged on most Linux distributions, generally
 the packages will automatically create a default `database cluster`
 so that all you need to do is to create a user and a database, then
 configure your application to use that database:
 
+Create (DB) User and Database
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 .. code-block:: bash
 
     $ sudo aptitude install postgresql
@@ -46,7 +55,12 @@ configure your application to use that database:
     $ sudo -u postgres createdb --owner=username databasename
 
 at this point you have a database server and a user account that can
-access (just) the one database you've created.  If you want, you can
+access (just) the one database you've created.
+
+Test Database Connection
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you want, you can
 test the database using the command-line psql client from PostgreSQL:
 
 .. code-block:: bash
@@ -74,9 +88,12 @@ them immediately against your database.
     command in psql!  This is a raw connection to the database and
     you are logged in as the owner of the database.
 
+Alter Production Config
+~~~~~~~~~~~~~~~~~~~~~~~
+
 Once you are satisfied that your database is defined and accessible,
-you alter your production.ini file.  The SQLAlchemy URL should point
-at the database you've created:
+you can alter your :ref:`deploy_ini` file to reference it.  The
+SQLAlchemy URL should point at the database you've created:
 
 .. code-block:: ini
 
@@ -90,22 +107,34 @@ at the database you've created:
    check the production.ini file into your development repository, instead
    check it into your configuration-management database (e.g. etckeeper),
    and restrict the file's read permissions as appropriate to allow only the
-   server process to read it.
+   server process (www-data) to read it.
+
+   See :ref:`deploy_ini_scc`
+
+Install Driver
+~~~~~~~~~~~~~~
 
 You need to add a PostgreSQL database driver to  your VirtualEnv to
-be able to access the server:
+be able to access the server.
 
 .. code-block:: bash
 
     (tg2env)$ easy_install psycopg2
 
-Now you can initialize your application's database:
+Initialize Database
+~~~~~~~~~~~~~~~~~~~
+
+Now you can initialize your application's database (see :ref:`deploy_ini` for
+how to create the `production.ini` file):
 
 .. code-block:: bash
 
     (tg2env)$ paster setup-app production.ini
     (tg2env)$ paster serve production.ini
 
+References
+~~~~~~~~~~
+
 Obviously this is only scratching the surface of PostgreSQL installation
 and maintenance.  For further information:
 
@@ -115,13 +144,20 @@ and maintenance.  For further information:
 
 .. _`The PostgreSQL Docs`: http://www.postgresql.org/docs/8.4/interactive/index.html
 
-.. todo:: Document setup of MySQL
-.. todo:: Document setup of MongoDB
-.. todo:: Document setup of Oracle (low priority)
-.. todo:: Document setup of MSSQL (low priority)
-
 What's Next?
 -------------
 
 * :ref:`deploy_standard` -- if you are deploying your application, you likely want
   to continue working through the standard deployment pattern
+* :ref:`deploy_which_database` -- discusses how to go about choosing an alternate
+  database engine.
+* :ref:`dbdriverinstall` -- discusses initial setup of database drivers
+
+
+.. todo:: Priority high: Document setup of MySQL
+.. todo:: Priority low: Document setup of Oracle
+.. todo:: Priority low: Document setup of MSSQL
+.. todo:: Priority low: Document deployment issues with SQLite
+
+.. todo:: Priority medium: Document setup of MongoDB/Ming (not here)
+.. todo:: Priority low: Document setup of CouchDB (not here)

docs/main/Deployment/Daemon.rst

@@ -16,6 +16,14 @@ There are a number of options to accomplish this:
 Which one you choose is likely up to your familiarity level with the
 particular tool.
 
+.. note::
+   For *extremely* small non-critical sites, it can sometimes be expedient
+   to use the `screen` tool to start a `paster serve production.ini`
+   process and then disconnect from the screen.  This isn't recommended,
+   as a power-cycle of the machine will require you to rush back from
+   your vacation to ssh in and re-start the server, but sometimes you
+   do this kind of thing just to get the job done *now*.
+
 .. todo:: Provide sample init script
 .. todo:: Provide sample upstart
 .. todo:: Provide sample supervisord config