Merge branch 'master' of github.com:okfn/ckan into 1038-organization-…

…hierarchy

Conflicts:
	ckan/config/deployment.ini_tmpl
	ckan/new_authz.py
	ckan/templates/organization/read.html
	ckan/tests/logic/test_auth.py
	doc/configuration.rst
	doc/solr-setup.rst

.travis.yml

@@ -5,7 +5,8 @@ python:
 env:
   - PGVERSION=9.1
   - PGVERSION=8.4
-script: ./bin/travis-build
+install: ./bin/travis-install-dependencies
+script: ./bin/travis-run-tests
 notifications:
   irc:
     channels:

CHANGELOG.rst

@@ -12,10 +12,30 @@ v2.2
 
 API changes and deprecations:
 
+* The Solr schema file is now always named ``schema.xml`` regardless of the
+  CKAN version. Old schema files have been kept for backwards compatibility
+  but users are encouraged to point to the new unified one.
+* The `ckan.api_url` has been completely removed and it can no longer be used
 * The edit() and after_update() methods of IPackageController plugins are now
   called when updating a resource using the web frontend or the
   resource_update API action [#1052]
 
+v2.1.1 2013-11-8
+================
+
+Bug fixes:
+ * Fix errors on preview on non-root locations (#960)
+ * Fix place-holder images on non-root locations (#1309)
+ * Don't accept invalid URLs in resource proxy (#1106)
+ * Make sure came_from url is local (#1039)
+ * Fix logout redirect in non-root locations (#1025)
+ * Wrong auth checks for sysadmins on package_create (#1184)
+ * Don't return private datasets on package_list (#1295)
+ * Stop tracking failing when no lang/encoding headers (#1192)
+ * Fix for paster db clean command getting frozen
+ * Fix organization not set when editing a dataset (#1199)
+ * Fix PDF previews (#1194)
+ * Fix preview failing on private datastore resources (#1221)
 
 v2.1 2013-08-13
 ===============
@@ -96,6 +116,18 @@ Known issues:
  * Under certain authorization setups the frontend for the groups functionality
    may not work as expected (See #1176 #1175).
 
+v2.0.3 2013-11-8
+================
+
+Bug fixes:
+ * Fix errors on preview on non-root locations (#960)
+ * Don't accept invalid URLs in resource proxy (#1106)
+ * Make sure came_from url is local (#1039)
+ * Fix logout redirect in non-root locations (#1025)
+ * Don't return private datasets on package_list (#1295)
+ * Stop tracking failing when no lang/encoding headers (#1192)
+ * Fix for paster db clean command getting frozen
+
 
 v2.0.2 2013-08-13
 =================

CONTRIBUTING.rst

@@ -23,7 +23,7 @@ contributions to CKAN.
 
 
 ----------------
-Reporting Issues
+Reporting issues
 ----------------
 
 If you've found a bug in CKAN, open a new issue on CKAN's `GitHub Issues`_ (try
@@ -47,7 +47,7 @@ For contributing translations to CKAN, see
 .. _coding standards:
 
 ----------------
-Coding Standards
+Coding standards
 ----------------
 
 When writing code for CKAN, try to respect our coding standards:
@@ -61,17 +61,19 @@ When writing code for CKAN, try to respect our coding standards:
    css-coding-standards
    javascript-coding-standards
    testing-coding-standards
+   upgrading-dependencies
 
 * `CKAN coding standards <http://docs.ckan.org/en/latest/ckan-coding-standards.html>`_
 * `Python coding standards <http://docs.ckan.org/en/latest/python-coding-standards.html>`_
 * `HTML coding standards <http://docs.ckan.org/en/latest/html-coding-standards.html>`_
 * `CSS coding standards <http://docs.ckan.org/en/latest/css-coding-standards.html>`_
 * `JavaScript coding standards <http://docs.ckan.org/en/latest/javascript-coding-standards.html>`_
 * `Testing coding standards <http://docs.ckan.org/en/latest/testing-coding-standards.html>`_
+* `Upgrading CKAN's dependencies <http://docs.ckan.org/en/latest/upgrading-dependencies.html>`_
 
 
 ---------------
-Commit Messages
+Commit messages
 ---------------
 
 Generally, follow the `commit guidelines from the Pro Git book`_:
@@ -104,7 +106,7 @@ Here's an example of a good CKAN commit message::
 
 
 -------------------------------
-Frontend Development Guidelines
+Frontend development guidelines
 -------------------------------
 
 .. toctree::
@@ -126,7 +128,7 @@ Frontend Development Guidelines
 
 
 ---------------------
-Writing Documentation
+Writing documentation
 ---------------------
 
 The quickest and easiest way to contribute documentation to CKAN is to sign up
@@ -149,7 +151,7 @@ the `documentation guidelines <http://docs.ckan.org/en/latest/documentation-guid
 .. _making a pull request:
 
 ---------------------
-Making a Pull Request
+Making a pull request
 ---------------------
 
 Once you've written some CKAN code or documentation, you can submit it for
@@ -230,7 +232,7 @@ This section will walk you through the steps for making a pull request.
 .. _create a pull request on GitHub: https://help.github.com/articles/creating-a-pull-request
 
 
-Merging a Pull Request
+Merging a pull request
 ======================
 
 If you're reviewing a pull request for CKAN, when merging a branch into master:

MANIFEST.in

@@ -5,9 +5,13 @@ recursive-include ckan/config *.xml
 recursive-include ckan/i18n *
 recursive-include ckan/templates *
 recursive-include ckan *.ini
-recursive-include ckanext/**/public *
-recursive-include ckanext/**/templates *
-recursive-include ckanext/**/i18n *
+
+recursive-include ckanext/*/i18n *
+recursive-include ckanext/*/public *
+recursive-include ckanext/*/templates *
+recursive-include ckanext/*/theme/public *
+recursive-include ckanext/*/theme/templates *
+
 prune .git
 include CHANGELOG.txt
 include ckan/migration/migrate.cfg

bin/travis-install-dependencies

@@ -0,0 +1,53 @@
+#!/bin/bash
+
+# Exit immediately if any command fails
+set -e
+
+# Drop Travis' postgres cluster if we're building using a different pg version
+TRAVIS_PGVERSION='9.1'
+if [ $PGVERSION != $TRAVIS_PGVERSION ]
+then
+  sudo -u postgres pg_dropcluster --stop $TRAVIS_PGVERSION main
+  # Make psql use $PGVERSION
+  export PGCLUSTER=$PGVERSION/main
+fi
+
+# Install postgres and solr
+sudo apt-get update -qq
+sudo apt-get install postgresql-$PGVERSION solr-jetty libcommons-fileupload-java:amd64=1.2.2-1
+
+if [ $PGVERSION == '8.4' ]
+then
+  # force postgres to use 5432 as it's port
+  sudo sed -i -e 's/port = 5433/port = 5432/g' /etc/postgresql/8.4/main/postgresql.conf
+fi
+
+sudo service postgresql restart
+
+# Setup postgres' users and databases
+sudo -u postgres psql -c "CREATE USER ckan_default WITH PASSWORD 'pass';"
+sudo -u postgres psql -c "CREATE USER datastore_default WITH PASSWORD 'pass';"
+sudo -u postgres psql -c 'CREATE DATABASE ckan_test WITH OWNER ckan_default;'
+sudo -u postgres psql -c 'CREATE DATABASE datastore_test WITH OWNER ckan_default;'
+
+export PIP_USE_MIRRORS=true
+pip install -r requirements.txt
+pip install -r dev-requirements.txt
+
+python setup.py develop
+
+# Install npm dpes for mocha
+npm install -g mocha-phantomjs phantomjs
+
+paster db init -c test-core.ini
+
+# If Postgres >= 9.0, we don't need to use datastore's legacy mode.
+if [ $PGVERSION != '8.4' ]
+then
+  sed -i -e 's/.*datastore.read_url.*/ckan.datastore.read_url = postgresql:\/\/datastore_default:pass@\/datastore_test/' test-core.ini
+  paster datastore set-permissions postgres -c test-core.ini
+else
+  sed -i -e 's/.*datastore.read_url.*//' test-core.ini
+fi
+
+cat test-core.ini

bin/travis-run-tests

@@ -0,0 +1,27 @@
+#!/bin/sh
+
+# Configure Solr
+echo "NO_START=0\nJETTY_HOST=127.0.0.1\nJETTY_PORT=8983\nJAVA_HOME=$JAVA_HOME" | sudo tee /etc/default/jetty
+sudo cp ckan/config/solr/schema.xml /etc/solr/conf/schema.xml
+sudo service jetty restart
+
+# Run mocha front-end tests
+# We need ckan to be running for some tests
+paster serve test-core.ini &
+sleep 5 # Make sure the server has fully started
+mocha-phantomjs http://localhost:5000/base/test/index.html
+# Did an error occur?
+MOCHA_ERROR=$?
+# We are done so kill ckan
+killall paster
+
+# And finally, run the nosetests
+nosetests --ckan --reset-db --with-pylons=test-core.ini --nologcapture ckan ckanext
+# Did an error occur?
+NOSE_ERROR=$?
+
+[ "0" -ne "$MOCHA_ERROR" ] && echo MOCKA tests have failed
+[ "0" -ne "$NOSE_ERROR" ] && echo NOSE tests have failed
+
+# If an error occurred in our tests make sure travis knows
+exit `expr $MOCHA_ERROR + $NOSE_ERROR`

ckan/ckan_nose_plugin.py

@@ -19,21 +19,26 @@ def startContext(self, ctx):
 
         if 'new_tests' in repr(ctx):
             # We don't want to do the stuff below for new-style tests.
+            if not CkanNose.settings.reset_database:
+                model.repo.tables_created_and_initialised = True
             return
 
         if isclass(ctx):
             if hasattr(ctx, "no_db") and ctx.no_db:
                 return
-            if self.is_first_test or CkanNose.settings.ckan_migration:
+            if (not CkanNose.settings.reset_database
+                    and not CkanNose.settings.ckan_migration):
+                model.Session.close_all()
+                model.repo.tables_created_and_initialised = True
+                model.repo.rebuild_db()
+                self.is_first_test = False
+            elif self.is_first_test or CkanNose.settings.ckan_migration:
                 model.Session.close_all()
                 model.repo.clean_db()
                 self.is_first_test = False
                 if CkanNose.settings.ckan_migration:
                     model.Session.close_all()
                     model.repo.upgrade_db()
-            # init_db is run at the start of every class because
-            # when you use an in-memory sqlite db, it appears that
-            # the db is destroyed after every test when you Session.Remove().
 
             ## This is to make sure the configuration is run again.
             ## Plugins use configure to make their own tables and they
@@ -43,6 +48,9 @@ def startContext(self, ctx):
             for plugin in PluginImplementations(IConfigurable):
                 plugin.configure(config)
 
+            # init_db is run at the start of every class because
+            # when you use an in-memory sqlite db, it appears that
+            # the db is destroyed after every test when you Session.Remove().
             model.repo.init_db()
 
     def options(self, parser, env):
@@ -66,6 +74,11 @@ def options(self, parser, env):
             dest='segments',
             help='A string containing a hex digits that represent which of'
                  'the 16 test segments to run. i.e 15af will run segments 1,5,a,f')
+        parser.add_option(
+            '--reset-db',
+            action='store_true',
+            dest='reset_database',
+            help='drop database and reinitialize before tests are run')
 
     def wantClass(self, cls):
         name = cls.__name__