Browse files

Add a Development page and a Testing Checklist to the documentation.

  • Loading branch information...
1 parent 6197353 commit 61d51bae23eb6ebe6d19619e766977bee4138fbc @jmafc jmafc committed Oct 23, 2012
Showing with 258 additions and 14 deletions.
  1. +109 −0 docs/devel.rst
  2. +1 −0 docs/index.rst
  3. +5 −1 docs/install.rst
  4. +2 −3 docs/overview.rst
  5. +141 −10 docs/testing.rst
@@ -0,0 +1,109 @@
+.. _development:
+The following details the tools needed to contribute to the
+development of Pyrseas. If you have any doubts or questions, please
+subscribe to the `Pyrseas-general mailing list
+<>`_ and post a
+message to the list. In addition, see *Version Control* below on how
+to set up a GitHub account to participate in development.
+- Git
+- Python
+- PostgreSQL
+- Psycopg2
+- Tox
+Version Control
+Pyrseas uses `Git <>`_ to control changes to its
+source code. As mentioned under :ref:`download`, the master Git
+`repository <>`_ is located at GitHub.
+To install Git, either `download and install
+<>`_ the latest stable release for your
+platform or follow the `Pro Git` `installation instructions
+<>`_. For
+most Linux users, ``apt-get`` or ``yum`` (depending on Linux flavor)
+will be the simplest means to install the ``git-core`` package. For
+Windows, downloading the installer and selecting ``Git Bash`` gives
+you not only Git but a Bash shell, which is handy if you're coming
+from a Linux/Unix background.
+Once Git is installed, change to a suitable directory and clone the
+master repository::
+ git clone git://
+ git clone
+To be able to create a fork on GitHub, open an issue or participate in
+Pyrseas development, you'll first have to `create a GitHub account
+Programming Language
+To contribute to Pyrseas, you need at least one version of `Python
+<>`_. You can develop using Python 3, but since
+we want to continue supporting Python 2, you'll want to install Python
+2.7 or 2.6 in addition to Python 3.3 or 3.2.
+If Python is not already available on your machine, either `download
+and install one or both <>`_ of the
+production releases for your platform, follow the applicable
+installation instructions given in `The Hitchhiker’s Guide to Python!
+<>`_ or install it from your
+platform's package management system.
+Database Installation
+To participate in Pyrseas development, you'll also need one or more
+installations of `PostgreSQL <>`_, versions
+9.2, 9.1, 9.0 or 8.4. If you only have limited space, it is
+preferable to install one of the latest two versions.
+The versions can be obtained as binary packages or installers from the
+` website <>`_. The
+site also includes instructions for installing from package management
+systems or building it from source.
+To access PostgreSQL from Python, you'll have to install the `Psycopg
+<>`_ adapter. You can either follow the
+instructions in `Psycopg's site <>`_,
+or install it from your package management system. Note that if you
+install both Python 2 and 3, you will have to install two packages,
+e.g., ``python-psycopg2`` and ``python3-psycopg2``.
+Other Libraries and Tools
+The ``dbtoyaml`` and ``yamltodb`` utilities use the `PyYAML
+<>`_ library. You can install it from
+the PyYAML site, or possibly from your package management system. For
+Windows 64-bit, please read the note under :ref:`installers`.
+If using the Pyrseas utilities with Python 2.6, you will need to
+install the ``argparse`` module from the `Python Package Index
+<>`_. For later Python versions,
+this is already included in the Python standard library.
+To easily run the Pyrseas tests against various Python/PostgreSQL
+version combinations, install `Tox
+<>`_. Please refer to
+:ref:`testing` for more information.
@@ -42,6 +42,7 @@ Contents
+ devel
.. toctree::
@@ -19,7 +19,7 @@ Requirements
Pyrseas provides tools for `PostgreSQL <>`_,
so obviously you need **PostgreSQL** to start with. Pyrseas has been
-tested with PG 8.4, 9.0 and 9.1, and we'll certainly keep up with
+tested with PG 8.4, 9.0, 9.1 and 9.2, and we'll certainly keep up with
future releases. Please refer to section III, `Server Administration
<>`_ of
the PostgreSQL documentation for details on installation, setup and
@@ -52,6 +52,8 @@ library. This may be available as a package for your operating system
or it can be downloaded from the `Python Package Index
+.. _download:
@@ -147,6 +149,8 @@ your Python installation. The source and bytecode files will go in
the ``site-packages`` folder, e.g.,
+.. _installers:
Python Installers
@@ -41,7 +41,7 @@ Version Control
The case for implementing a tool to facilitate version control over
-SQL databases has been made in a couple of blog posts: `Version
+SQL databases was made in a couple of blog posts: `Version
Control, Part 1: Pre-SQL
and `Version Control, Part 2: SQL Databases
@@ -91,8 +91,7 @@ Pyrseas like the hero.
.. [#] The common English name for Perseas is Perseus and the Ancient
Greek name is Perseos. However, in modern Greek Περσέας_ is the
- more common spelling for the mythical hero. The project would be
- Πυρσέας or ΠΥΡΣΕΑΣ in Greek.
+ more common spelling for the mythical hero.
.. _Περσέας:
@@ -1,7 +1,9 @@
+.. _testing:
-The majority of Pyrseas' capabilites are exercised and verified via
+The majority of Pyrseas' capabilities are exercised and verified via
unit tests written using Python's `unittest framework
<>`_. The tests can be
run from the command line by most users, e.g.,
@@ -13,22 +15,24 @@ run from the command line by most users, e.g.,
python ForeignKeyToMapTestCase
python TriggerToSqlTestCase.test_create_trigger
+ python -m unittest discover
The first ``python`` command above runs all tests related to tables,
mapping, creating, dropping, etc. The next command runs the subset of
tests related to mapping tables with foreign keys and the following
one executes a single test to generate SQL to create a trigger. The
-final command runs through all the tests suites in the ``dbobject``
+fourth command runs through all the tests suites in the ``dbobject``
+subdirectory. The final command does the same but using the test
+discovery feature available from Python 2.7.
Environment Variables
By default, the tests use a PostgreSQL database named
``pyrseas_testdb`` which is created if it doesn't already exist. The
-tests are run as the logged in user, using the "USER" Unix/Linux
-environment variable if defined. They access PostgreSQL on the local
-host using the default port number (5432).
+tests are run as the logged in user, using the ``USER`` Unix/Linux
+environment variable (or ``USERNAME`` under Windows). They access
+PostgreSQL on the local host using the default port number (5432).
The following four environment variables can be used to change the
defaults described above:
@@ -47,10 +51,137 @@ access to it, the user role will need CREATEDB privilege.
Most tests do not require special privileges. However, tests that
define dynamically loaded functions (e.g.,
:func:`test_create_c_lang_function` in :mod:``)
-require SUPERUSER privilege.
+require SUPERUSER privilege. Such tests will be skipped if the user
+lacks the privilege.
-Most tests do not require installation of supporting packages.
-However, tests that define dynamically loaded functions (see above)
-require that the `contrib/spi module
+Most tests do not require installation of supporting PostgreSQL
+packages. However, tests that define dynamically loaded functions
+(see above) require that the `contrib/spi module
<>`_ be
+On Windows, it is necessary to install Perl in order to run some of
+the tests. A suitable choice is Strawberry Perl which can be
+downloaded from However, the
+default installation is placed in ``C:\strawberry`` and can hold a
+single Perl version. Furthermore, PostgreSQL 8.4 and 9.0 are linked
+with Perl 5.10 whereas PostgreSQL 9.1 and 9.2 are linked with 5.14.
+It is recommended that Perl 5.10 be installed as this gives the fewest
+test failures. See `this blog post
+for more details.
+The COLLATION tests, run under PostgreSQL 9.1 and 9.2, require the
+``fr_FR.utf8`` locale (or ``French.France.1252`` language on Windows)
+to be installed.
+Testing Checklist
+The following is a summary list of steps needed to test Pyrseas on a
+new machine. Refer to :ref:`development` for details on how to
+accomplish a given installation task. "Package manager" refers to the
+platform's package management system utility such as ``apt-get`` or
+``yum``. Installation from PyPI can be done with either ``pip`` or
+``easy_install``. Some operations require administrative or superuser
+privileges, at either the operating system or PostgreSQL level.
+ - Install Git using package manager or from
+ (on Windows, prefer Git Bash)
+ - ``git clone git://``
+ - Install Python 2.7 and 3.2, using package manager or from
+ installers at
+ - Install PostgreSQL 9.2, 9.1, 9.0 and 8.4, using package manager or
+ binary installers at
+ .. note:: On Linux, make sure you install the contrib and plperl
+ packages, e.g., on Debian, postgresql-contrib-n.n and
+ postgresql-plperl-n.n (where `n.n` is the PostgreSQL
+ version number)
+ - Install Psycopg2, using package manager, or from PyPI
+ ( or
+ .. note:: On Windows, it is best to first install the 2008
+ Microsoft Visual Express Studio from `here`_. An
+ alternative that may work is to use `MinGW
+ <>`_. See `these blog`_ `posts`_ for
+ more details.
+ .. _here:
+ .. _these blog:
+ .. _posts:
+ - Install PyYAML, using package manager, or from PyPI
+ ( or
+ - Install Tox, from PyPI (
+ .. note:: Psycopg2, PyYAML and Tox all have to be installed twice,
+ i.e., once under Python 2.7 and another under 3.2.
+ - On Windows, install Perl (see discussion above under
+ "Restrictions"). On Linux, usually Perl is already available.
+ - As **postgres** user, using psql or pgAdmin, create a test user,
+ e.g., your name. The user running tests must have at a minimum
+ createdb privilege, in order to create the test database. To run
+ *all* the tests, the user also needs superuser privilege.
+ - Create a PostgreSQL password file, e.g., on Linux: ``~/.pgpass``, on
+ Windows: ``%APPDATA%\postgresql\pgpass.conf``.
+ - Using psql or pgAdmin, create roles **user1** and **user2**.
+ - Create directories to hold tablespaces, e.g., ``/extra/pg/9.1/ts1``
+ on Linux, ``C:\\extra\\pg\\9.1\\ts1`` on Windows. The directories
+ need to be owned by the **postgres** user. This may be tricky on
+ older Windows versions, but the command ``cacls <dir> /E /G
+ postgres:F`` should suffice. Using psql or pgAdmin, create
+ tablespaces **ts1** and **ts2**, e.g., ``CREATE TABLESPACE ts1
+ LOCATION '<directory>'`` (on Windows, you'll have to use, e.g.,
+ ``E'C:\\dir\\ts1'``, to specify the directory).
+ - On Windows, for PostgreSQL 9.2, the default installation is owned
+ by the Network Service account, so the ``cacls`` command should
+ be ``cacls <dir> /E /G networkservices:F``.
+ .. note:: The creation of users/roles and tablespaces has to be
+ repeated for each PostgreSQL version.
+ - Install the locale ``fr_FR.utf8`` on Linux/Unix or the language
+ ``French.France.1252`` on Windows.
+ - On Debian and derivatives, this can be done with the command::
+ sudo dpkg-reconfigure locales
+ - On Windows, open the Control Panel, select Date, Time, Language,
+ and Regional Options, then Regional and Language Options (or Add
+ other languages), click on the Advanced tab in the dialog and
+ then choose “French (France)” from the dropdown. Finally, click
+ OK and respond to any subsequent prompts to install the locale,
+ including rebooting the machine.
+ - Change to the Pyrseas source directory (created by the second step above).
+ - Define the ``PYTHONPATH`` environment variable to the Pyrseas source
+ directory, e.g., on Linux, ``export PYTHONPATH=$PWD``, on
+ Windows, ``set PYTHONPATH=%USERPROFILE%\somedir\Pyrseas``.
+ - Define the environment variables ``PG84_PORT``, ``PG90_PORT``,
+ ``PG91_PORT`` and ``PG92_PORT`` to point to the corresponding
+ PostgreSQL ports.
+ - Invoke ``tox``. This will create two virtualenvs in a ``.tox``
+ subdirectory--one for Python 2.7 and another for 3.2, install
+ Pyrseas and its prerequisites (Psycopg2 and PyYAML) into each
+ virtualenv and run the unit tests for each combination of
+ PostgreSQL and Python.

0 comments on commit 61d51ba

Please sign in to comment.