[release-v1.6][doc]: Update docs generally.

doc/common-error-messages.rst

@@ -132,4 +132,11 @@ This occurs when upgrading to CKAN 1.5.1 with a database with duplicate user nam
 ``ERROR:  must be member of role "okfn"`` & ``WARNING:  no privileges could be revoked for "public"``
 =====================================================================================================
 
-These are seen when loading a CKAN database from another machine. It is the result of the database tables being owned by a user that doesn't exist on the new machine. The owner of the table is not important, so this error is harmless and can be ignored.
+These are seen when loading a CKAN database from another machine. It is the result of the database tables being owned by a user that doesn't exist on the new machine. The owner of the table is not important, so this error is harmless and can be ignored.
+
+``IOError: [Errno 13] Permission denied: '/var/log/ckan/colorado/colorado.log'``
+================================================================================
+
+This is usually seen when you run the paster command with one user, and CKAN is deployed on Apache (for example) which runs as another user. The usual remedy is to run the paster command as user ``www-data``. i.e..::
+
+sudo -u www-data paster ...

doc/extensions.rst

@@ -6,10 +6,10 @@ This is where it gets interesting! The CKAN software can be customised with 'ext
 
 Extensions allow you to customise CKAN for your own requirements, without interfering with the basic CKAN system.
 
-.. warning:: This is an advanced topic. At the moment, you need to have prepared your system to work with extensions, as described in :doc:`prepare-extensions`. We are working to make the most popular extensions more easily available as Debian packages. 
+.. warning:: This is an advanced topic.
 
-Finding Extensions
-------------------
+Choosing Extensions
+-------------------
 
 All CKAN extensions are listed on the official `Extension listing on the CKAN
 wiki <http://wiki.ckan.net/List_of_Extensions>`_.
@@ -37,25 +37,23 @@ You can install an extension on a CKAN instance as follows.
 
 .. note::
 
-  Core extensions do not need to be installed -- just enabled (see below).
+  'Core' extensions do not need to be installed -- just enabled (see below).
 
-1. First, ensure you are working within your virtualenv (see :doc:`prepare-extensions` if you are not sure what this means)::
+#. Locate your CKAN virtual environment (pyenv) in your filesystem. It is usually in a directory similar to this: ``/var/lib/ckan/INSTANCE_NAME/pyenv``
 
-   . /home/ubuntu/pyenv/bin/activate
+If it is not here, to get the definitive answer, check your CKAN Apache configuration (``/etc/apache2/sites-enabled``) for your WSGIScriptAlias (e.g. ``/var/lib/ckan/colorado/wsgi.py``) which has an ``execfile`` instruction. The first parameter is the pyenv directory plus ``/bin/activate_this.py``. e.g. ``/var/lib/ckan/colorado/pyenv/bin/activate_this.py`` means the pyenv dir is: ``/var/lib/ckan/colorado/pyenv``.
 
-2. Install the extension package code using ``pip``.
+#. Install the extension package code into your pyenv using ``pip``.
 
  For example, to install the Disqus extension, which allows users to comment on datasets::
 
-       pip install -E ~/var/srvc/ckan.net/pyenv git+https://github.com/okfn/ckanext-disqus.git
-
- The ``-E`` parameter is for your CKAN Python environment (e.g. ``~/var/srvc/ckan.net/pyenv``). 
+       /var/lib/ckan/INSTANCE_NAME/pyenv/bin/pip install -E /var/lib/ckan/INSTANCE_NAME/pyenv -e git+https://github.com/okfn/ckanext-disqus.git#egg=ckanext-disqus
 
  Prefix the source URL with the repo type (``hg+`` for Mercurial, ``git+`` for Git).
 
  The dependency you've installed will appear in the ``src/`` directory under your Python environment. 
 
-Now the extension is installed you need to enable it.
+Now the extension is installed, so now you can enable it.
 
 
 Enabling an Extension

doc/index.rst

@@ -6,7 +6,7 @@ This Administration Guide covers how to set up and manage `CKAN <http://ckan.org
 
 * The first two sections cover your two options for installing CKAN: package or source install.
 * The rest of the first half of the Guide, up to :doc:`authorization`, cover setup and basic admin. 
-* The second half of the Guide, from :doc:`prepare-extensions` onwards, covers advanced tasks, including extensions and forms. 
+* The second half of the Guide, from :doc:`extensions` onwards, covers advanced tasks, including extensions and forms. 
 
 For high-level information on what CKAN is, see the `CKAN website <http://ckan.org>`_.
 
@@ -23,9 +23,8 @@ Contents:
    loading_data
    paster
    authorization
-   prepare-extensions
    extensions
-   plugins
+   writing-extensions
    forms
    form-integration
    database_dumps

doc/install-from-source.rst

@@ -150,72 +150,65 @@ locations:
 6. Setup a PostgreSQL database
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-  List existing databases:
+List existing databases:
 
   ::
 
-sudo -u postgres psql -l
+    sudo -u postgres psql -l
 
-  It is advisable to ensure that the encoding of databases is 'UTF8', or 
-  internationalisation may be a problem. Since changing the encoding of PostgreSQL
-  may mean deleting existing databases, it is suggested that this is fixed before
-  continuing with the CKAN install.
+It is advisable to ensure that the encoding of databases is 'UTF8', or 
+internationalisation may be a problem. Since changing the encoding of PostgreSQL
+may mean deleting existing databases, it is suggested that this is fixed before
+continuing with the CKAN install.
 
-  Next you'll need to create a database user if one doesn't already exist.
+Next you'll need to create a database user if one doesn't already exist.
 
   .. tip ::
 
-If you choose a database name, user or password which are different from the example values suggested below then you'll need to change the sqlalchemy.url value accordingly in the CKAN configuration file you'll create in the next step.
+    If you choose a database name, user or password which are different from the example values suggested below then you'll need to change the sqlalchemy.url value accordingly in the CKAN configuration file that you'll create in the next step.
 
-  Here we create a user called ``ckanuser`` and will enter ``pass`` for the password when prompted:
+Here we create a user called ``ckanuser`` and will enter ``pass`` for the password when prompted:
 
   ::
 
-sudo -u postgres createuser -S -D -R -P ckanuser
+    sudo -u postgres createuser -S -D -R -P ckanuser
 
-  Now create the database (owned by ``ckanuser``), which we'll call ``ckantest``:
+Now create the database (owned by ``ckanuser``), which we'll call ``ckantest``:
 
   ::
 
-sudo -u postgres createdb -O ckanuser ckantest
+    sudo -u postgres createdb -O ckanuser ckantest
 
 7. Create a CKAN config file
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-  Make sure you are in an activated environment (see step 3) so that Python
-  Paste and other modules are put on the python path (your command prompt will
-  start with ``(pyenv)`` if you have) then change into the ``ckan`` directory
-  which will have been created when you installed CKAN in step 4 and create the
-  CKAN config file using Paste. These instructions call it ``development.ini`` since that is the required name for running the CKAN tests. But for a server deployment then you might want to call it say after the server hostname e.g. ``test.ckan.net.ini``.
+Make sure you are in an activated environment (see step 3) so that Python
+Paste and other modules are put on the python path (your command prompt will
+start with ``(pyenv)`` if you have) then change into the ``ckan`` directory
+which will have been created when you installed CKAN in step 4 and create the
+CKAN config file using Paste. These instructions call it ``development.ini`` since that is the required name for running the CKAN tests. But for a server deployment then you might want to call it say after the server hostname e.g. ``test.ckan.net.ini``.
 
   ::
 
-cd pyenv/src/ckan
-paster make-config ckan development.ini
+    cd pyenv/src/ckan
+    paster make-config ckan development.ini
 
-  If you used a different database name or password when creating the database
-  in step 5 you'll need to now edit ``development.ini`` and change the
-  ``sqlalchemy.url`` line, filling in the database name, user and password you used.
+If you used a different database name or password when creating the database
+in step 5 you'll need to now edit ``development.ini`` and change the
+``sqlalchemy.url`` line, filling in the database name, user and password you used.
 
   ::
   
-sqlalchemy.url = postgresql://ckanuser:pass@localhost/ckantest
+    sqlalchemy.url = postgresql://ckanuser:pass@localhost/ckantest
 
-  If you're using a remote host with password authentication rather than SSL authentication, use::
+If you're using a remote host with password authentication rather than SSL authentication, use::
 
-sqlalchemy.url = postgresql://<user>:<password>@<remotehost>/ckan?sslmode=disable
+    sqlalchemy.url = postgresql://<user>:<password>@<remotehost>/ckan?sslmode=disable
 
-  .. caution ::
+.. caution ::
 
-  Advanced users: If you have installed CKAN using the Fabric file capability (deprecated),
-  your config file will be called something like ``pyenv/ckan.net.ini``. 
-  This is fine but CKAN probably won't be 
-  able to find your ``who.ini`` file. To fix this edit ``pyenv/ckan.net.ini``, 
-  search for the line ``who.config_file = %(here)s/who.ini`` and change it
-  to ``who.config_file = who.ini``.
+  Legacy installs of CKAN may have the config file in the pyenv directory, e.g. ``pyenv/ckan.net.ini``. This is fine but CKAN probably won't be able to find your ``who.ini`` file. To fix this edit ``pyenv/ckan.net.ini``, search for the line ``who.config_file = %(here)s/who.ini`` and change it to ``who.config_file = who.ini``.
 
-  We are moving to a new deployment system where this incompatibility 
-  will be fixed.
 
 8. Create database tables
 ~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -243,14 +236,14 @@ sqlalchemy.url = postgresql://<user>:<password>@<remotehost>/ckan?sslmode=disabl
 9. Create the cache directory
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-  You need to create the Pylon's cache directory specified by 'cache_dir' 
-  in the config file.
+You need to create the Pylon's cache directory specified by 'cache_dir' 
+in the config file.
 
-  (from the ``pyenv/src/ckan`` directory):
+(from the ``pyenv/src/ckan`` directory):
 
   ::
 
-mkdir data
+    mkdir data
 
 
 10. Setup Solr
@@ -262,8 +255,8 @@ Set appropriate values for the ``ckan.site_id`` and ``solr_url`` config variable
 
 ::
 
- ckan.site_id=my_ckan_instance
- solr_url=http://127.0.0.1:8983/solr
+    ckan.site_id=my_ckan_instance
+    solr_url=http://127.0.0.1:8983/solr
 
 
 
@@ -277,12 +270,12 @@ Set appropriate values for the ``ckan.site_id`` and ``solr_url`` config variable
 
   ::
 
-paster serve development.ini
+    paster serve development.ini
 
 12. Point your web browser at: http://127.0.0.1:5000/
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
- The CKAN homepage should load.
+The CKAN homepage should load.
 
 Finally, make sure that tests pass, as described in :ref:`basic-tests`.
 

doc/paster.rst

@@ -14,14 +14,61 @@ Paster is run on the command line on the server running CKAN. This section cover
 Understanding Paster
 ====================
 
-The basic paster format is:: 
+At its simplest, paster commands can be thought of like this::
+
+  paster <ckan commands>
+
+But there are various extra elements to the commandline that usually need adding. We shall build them up:
+
+#. Enabling CKAN commands
+=========================
+
+Paster is used for many things aside from CKAN. You usually need to tell paster that you want to enable the CKAN commands::
+
+  paster --plugin=ckan <ckan commands>
+
+You know you need to do this if you get the error ``Command 'user' not known`` for a valid CKAN command.
+
+(Alternatively, CKAN commands are enabled by default if your current directory is the CKAN source directory)
+
+#. Pointing to your CKAN config
+===============================
+
+Paster needs to know where your CKAN config file is (so it knows which database and search index to deal with etc.)::
 
   paster --plugin=ckan <ckan commands> --config=<config file>
 
+If you forget to specify ``--config`` then you will get error ``AssertionError: Config filename '/home/okfn/development.ini' does not exist.``
+
+(Paster defaults to looking for development.ini in the current directory.)
+
 For example, to initialise a database::
 
   paster --plugin=ckan db init --config=/etc/ckan/std/std.ini
 
+#. Virtual environments
+=======================
+
+You often need to run paster within your CKAN virtual environment (pyenv). If CKAN was installed as 'source' then you can activate it as usual before running the paster command::
+
+  . ~/pyenv/bin/activate
+  paster --plugin=ckan db init --config=/etc/ckan/std/std.ini
+
+The alternative, which also suits a CKAN 'package' install, is to simply give the full path to the paster in your pyenv::
+
+  /var/lib/ckan/std/pyenv/bin/paster --plugin=ckan db init --config=/etc/ckan/std/std.ini
+
+
+#. Running Paster on a deployment
+=================================
+
+If CKAN is deployed with Apache on this machine, then you should run paster as the same user, which is usually ``www-data``. This is because paster will write to the same CKAN logfile as the Apache process and file permissions need to match. 
+
+ For example::
+
+  sudo -u www-data /var/lib/ckan/std/pyenv/bin/paster --plugin=ckan db init --config=/etc/ckan/std/std.ini
+
+Otherwise you will get an error such as: ``IOError: [Errno 13] Permission denied: '/var/log/ckan/std/std.log'``.
 
 .. _paster-help: