[merge] from release-v1.7

ckan · May 9, 2012 · b9ee2fa · b9ee2fa
2 parents c33113b + aa3e060
commit b9ee2fa
Show file tree

Hide file tree

Showing 28 changed files with 216 additions and 42 deletions.
diff --git a/.gitattributes b/.gitattributes
@@ -1,3 +1,2 @@
 *.po -diff
 *.mo -diff
-*.pot -diff
diff --git a/.tx/config b/.tx/config
@@ -1,7 +1,7 @@
 [main]
 host = http://www.transifex.net
 
-[ckan.1-6]
+[ckan.1-7]
 file_filter = ckan/i18n/<lang>/LC_MESSAGES/ckan.po
 source_file = ckan/i18n/ckan.pot
 source_lang = en

diff --git a/CHANGELOG.txt b/CHANGELOG.txt
@@ -1,6 +1,45 @@
 CKAN CHANGELOG
 ++++++++++++++
 
+v1.7 2012-05-09
+===============
+
+Major:
+ * Updated SOLR schema (#2327). Note: This will require and update of the SOLR schema file and a reindex.
+ * Support for Organization based workflow, with membership determinig access permissions to datasets (#1669,#2255)
+ * Related items such as visualizations, applications or ideas can now be added to datasets (#2204)
+ * Restricted vocabularies for tags, allowing grouping related tags together (#1698)
+ * Internal analytics that track number of views and downloads for datasets and resources (#2251)
+ * Consolidated multilingual features in an included extension (#1821,#1820)
+ * Atom feeds for publishers, tags and search results (#1593,#2277)
+ * RDF dump paster command (#2303)
+ * Better integration with the DataStore, based on ElasticSearch, with nice helper docs (#1797)
+ * Updated the Recline data viewer with new features such as better graphs and a map view (#2236,#2283)
+ * Improved and redesigned documentation (#2226,#2245,#2248)
+
+Minor:
+ * Groups can have an image associated (#2275)
+ * Basic resource validation (#1711)
+ * Ability to search without accents for accented words (#906)
+ * Weight queries so that title is more important than rest of body (#1826)
+ * Enhancements in the dataset and resource forms (#1506)
+ * OpenID can now be disabled (#1830)
+ * API and forms use same validation (#1792)
+ * More robust bulk search indexing, with options to ignore exceptions and just refresh (#1616i,#2232)
+ * Modify where the language code is placed in URLs (#2261)
+ * Simplified licenses list (#1359)
+ * Add extension point for dataset view (#1741)
+
+Bug fixes:
+ * Catch exceptions on the QA archiver (#1809)
+ * Error when changing language when CKAN is mounted in URL (#1804)
+ * Naming of a new package/group can clash with a route (#1742)
+ * Can't delete all of a package's resources over REST API (#2266)
+ * Group edit form didn't allow adding multiple datasets at once (#2292)
+ * Fix layout bugs in IE 7 (#1788)
+ * Bug with Portugese translation and Javascript (#2318)
+ * Fix broken parse_rfc_2822 helper function (#2314)
+
 v1.6 2012-02-24
 ===============
 
@@ -158,7 +197,7 @@ Minor:
   * Speed up tag reading (98d72)
   * Cope with new WebOb version 1 (#1267)
   * Avoid exceptions caused by bots hitting error page directly (#1176)
-  * Too minor to mention: #1234, 
+  * Too minor to mention: #1234,
 
 Bug fixes:
   * Re-adding tags to a package failed (since 1.4.1 in Web UI, 1.4 in API) (#1239)
@@ -197,7 +236,7 @@ Major:
   * Refactor Web interface to use logic layer rather than model objects directly. Forms now defined in navl schema and designed in HTML template. Forms use of Formalchemy is deprecated. (#1078)
 
 Minor:
-  * Links in user-supplied text made less attractive to spammers (nofollow) #1181 
+  * Links in user-supplied text made less attractive to spammers (nofollow) #1181
   * Package change notifications - remove duplicates (#1149)
   * Metadata dump linked to (#1169)
   * Refactor authorization code to be common across Package, Group and Authorization Group (#1074)
@@ -223,7 +262,7 @@ Bug fixes:
   * Visitor can't create packages on new CKAN install - since v1.3.3 (#1090)
   * OpenID user pages couldn't be accessed - since v1.3.2 (#1056)
   * Default site_url configured to ckan.net, so pages obtains CSS from ckan.net- since v1.3 (#1085)
- 
+
 
 v1.3.3 2011-04-08
 =================
@@ -256,7 +295,7 @@ Major:
   * Resources can have extra fields (although not in web interface yet) (#826)
   * CSW Harvesting - numerous of fixes & improvements. Ready for deployment. (#738 etc)
   * Language switcher (82002)
- 
+
 Minor:
   * Wordpress integration refactored as a Middleware plugin (#1013)
   * Unauthorized actions lead to a flash message (#366)
@@ -308,7 +347,7 @@ Highlights of changes:
   * WordPress twentyten compatible theming (#797)
   * Caching support (ETag) (#693)
   * Harvesting GEMINI2 metadata records from OGC CSW servers (#566)
- 
+
 Minor:
   * New API key header (#466)
   * Group metadata now revisioned (#231)
@@ -326,14 +365,14 @@ Highlights of changes:
   * Resource search in API (#336)
   * Visual theming of CKAN now easy (#340, #320)
   * Greater integration with external Web UIs (#335, #347, #348)
-  * Plug-ins can be configured to handle web requests from specified URIs and 
+  * Plug-ins can be configured to handle web requests from specified URIs and
     insert HTML into pages.
 
 Minor:
   * Search engine optimisations e.g. alphabetical browsing (#350)
   * CSV and JSON dumps improved (#315)
 
-  
+
 v1.0.2 2010-08-27
 =================
 
@@ -384,7 +423,7 @@ Highlights of changes:
     (#247)
 
 Bugfixes:
- 
+
   * Resources were losing their history (#292)
   * Extra fields now work with spaces in the name (#278, #280) and
     international characters (#288)
@@ -426,7 +465,7 @@ For a full listing of tickets see: <http://ckan.org/milestone/v0.11>. Main highl
 
     * Change to UUIDs for revisions and all domain objects
     * Improved search performance and better pagination
-    * Significantly improved performance in API and WUI via judicious caching 
+    * Significantly improved performance in API and WUI via judicious caching
 
 
 v0.10 2009-09-30
@@ -446,7 +485,7 @@ v0.10 2009-09-30
     * Convert to use formalchemy for all forms (#76)
     * Use paginate in webhelpers (#118)
 
-  * Minor: 
+  * Minor:
 
     * Add author and maintainer attributes to package (#91)
     * Change package state in the WUI (delete and undelete) (#126)
@@ -491,7 +530,7 @@ v0.6 2008-07-08
   * Autocompletion (+ suggestion) of tags when adding tags to a package.
   * Paginated lists for packages, tags, and revisions.
   * RESTful machine API for package access, update, listing and creation.
-  * API Keys for users who wish to modify information via the REST API. 
+  * API Keys for users who wish to modify information via the REST API.
   * Update to vdm v0.2 (SQLObject) which fixes ordering of lists.
   * Better immunity to SQL injection attacks.
 

diff --git a/ckan/config/middleware.py b/ckan/config/middleware.py
@@ -134,7 +134,7 @@ def make_app(global_conf, full_stack=True, static_files=True, **app_conf):
     if asbool(config.get('ckan.page_cache_enabled')):
         app = PageCacheMiddleware(app, config)
 
-    # Tracking
+    # Tracking add config option
     if asbool(config.get('ckan.tracking_enabled', 'false')):
         app = TrackingMiddleware(app, config)
     return app

diff --git a/ckan/logic/action/get.py b/ckan/logic/action/get.py
@@ -484,6 +484,11 @@ def resource_show(context, data_dict):
 
 def resource_status_show(context, data_dict):
 
+    try:
+        import ckan.lib.celery_app as celery_app
+    except ImportError:
+        return {'message': 'queue is not installed on this instance'}
+
     model = context['model']
     id = get_or_bust(data_dict, 'id')
 

diff --git a/ckan/model/__init__.py b/ckan/model/__init__.py
@@ -83,14 +83,16 @@ def init_db(self):
                 self.upgrade_db()
                 ## make sure celery tables are made as celery only makes them after
                 ## adding a task
-                import ckan.lib.celery_app as celery_app
-                import celery.db.session as celery_session
-
-                ##This creates the database tables it is a slight hack to celery.
-                backend = celery_app.celery.backend
-                celery_result_session = backend.ResultSession()
-                engine = celery_result_session.bind
-                celery_session.ResultModelBase.metadata.create_all(engine)
+                try:
+                    import ckan.lib.celery_app as celery_app
+                    import celery.db.session as celery_session
+                    ##This creates the database tables it is a slight hack to celery.
+                    backend = celery_app.celery.backend
+                    celery_result_session = backend.ResultSession()
+                    engine = celery_result_session.bind
+                    celery_session.ResultModelBase.metadata.create_all(engine)
+                except ImportError:
+                    pass
 
                 self.init_configuration_data()
                 self.tables_created_and_initialised = True

diff --git a/ckan/templates/_util.html b/ckan/templates/_util.html
@@ -128,7 +128,7 @@
   <py:def function="related_summary(related)">
   <li id="related-item-${related.id}" class="span3">
       <div class="thumbnail">
-        <button py:if="c.user and (c.userobj.id == related.owner_id or h.check_access('package_update',{'id':c.pkg.id}))" class="close" data-action="delete" data-related-id="${related.id}">×</button>
+        <button py:if="c.user and (c.userobj.id == related.owner_id or h.check_access('package_update',{'id':c.pkg.id}))" class="close" onclick="related_delete('${related.id}');">×</button>
         <a href="${related.url}" class="image">
           <img src="${related.image_url}" width="210" py:if="related.image_url" />
           <img src="/images/photo-placeholder.png" width="210" py:if="not related.image_url" />

diff --git a/ckan/templates/authorization_group/authz.html b/ckan/templates/authorization_group/authz.html
@@ -6,6 +6,12 @@
   <py:def function="page_heading">Authorization: ${c.authorization_group_name}</py:def>
 
   <div py:match="content">
+
+    <div class="alert alert-error">
+        Warning: Authorization groups are deprecated and no longer supported. They will be removed
+        completely on the next CKAN release.
+    </div>
+
     <h3>Update Existing Roles</h3>
 
     <form id="theform" method="POST">

diff --git a/ckan/templates/authorization_group/edit.html b/ckan/templates/authorization_group/edit.html
@@ -6,6 +6,12 @@
   <py:def function="page_heading">Edit: ${c.authorization_group.name if c.authorization_group else ''}</py:def>
 
   <div py:match="content">
+
+    <div class="alert alert-error">
+        Warning: Authorization groups are deprecated and no longer supported. They will be removed
+        completely on the next CKAN release.
+    </div>
+
     ${Markup(c.form)}
   </div>
 

diff --git a/ckan/templates/authorization_group/index.html b/ckan/templates/authorization_group/index.html
@@ -7,6 +7,12 @@
   <py:def function="page_heading">Authorization Groups</py:def>
 
   <div py:match="content">
+
+    <div class="alert alert-error">
+        Warning: Authorization groups are deprecated and no longer supported. They will be removed
+        completely on the next CKAN release.
+    </div>
+
     <p i18n:msg="item_count">There are <strong>${c.page.item_count}</strong> authorization groups.</p>
 
     ${c.page.pager()}

diff --git a/ckan/templates/authorization_group/new.html b/ckan/templates/authorization_group/new.html
@@ -6,6 +6,12 @@
   <py:def function="page_heading">New Authorization Group</py:def>
 
   <div py:match="content">
+
+    <div class="alert alert-error">
+        Warning: Authorization groups are deprecated and no longer supported. They will be removed
+        completely on the next CKAN release.
+    </div>
+
     ${Markup(c.form)}
   </div>
 

diff --git a/ckan/templates/authorization_group/read.html b/ckan/templates/authorization_group/read.html
@@ -7,6 +7,12 @@
   <py:def function="page_heading">${c.authorization_group.name}</py:def>
 
   <div py:match="content">
+
+    <div class="alert alert-error">
+        Warning: Authorization groups are deprecated and no longer supported. They will be removed
+        completely on the next CKAN release.
+    </div>
+
     <h3>Members</h3>
     <p i18n:msg="item_count">There are ${c.page.item_count} users in this authorization group.</p>
     ${c.page.pager()}

diff --git a/ckan/templates/layout_base.html b/ckan/templates/layout_base.html
@@ -176,11 +176,6 @@ <h3 class="widget-title">Sections</h3>
                   Revisions
                 </a>
               </li>
-              <li>
-                <a href="${h.url(controller='authorization_group', action='index')}">
-                  Authorization Groups
-                </a>
-              </li>
               <li>
                 <a href="${h.url_for('ckanadmin_index')}">
                   Site Admin

diff --git a/ckan/templates/package/related_list.html b/ckan/templates/package/related_list.html
@@ -40,6 +40,25 @@ <h3>Related items <i class="related-help icon-info-sign" rel="popover" data-cont
   </div>
 
   <py:def function="optional_head">
+    <script type="text/javascript" py:if="c.user">
+        function related_delete(related_id) {
+          var data = { 'id' : related_id }
+          $.ajax({
+            type: "post",
+            url: CKAN.SITE_URL + '/api/3/action/related_delete',
+            data: JSON.stringify(data),
+            success: function (data) {
+              window.location.reload();
+            },
+            error: function(err, txt, w) {
+              // This needs to be far more informative.
+              var msg = '<strong>Error:</strong> Unable to delete related item';
+              $('<div class="alert alert-error" />').html(msg).hide().prependTo($('div#main')).fadeIn();
+            }
+          });
+
+        }
+    </script>
     <py:if test="config.get('rdf_packages')">
       <link rel="alternate" type="application/rdf+xml" title="RDF/XML" href="${config['rdf_packages'] + '/' + c.pkg.id + '.rdf' }" />
       <link rel="alternate" type="application/turtle" title="RDF/Turtle" href="${config['rdf_packages'] + '/' + c.pkg.id + '.ttl' }" />

diff --git a/ckan/tests/logic/test_action.py b/ckan/tests/logic/test_action.py
@@ -2,6 +2,7 @@
 import json
 from pprint import pprint
 from nose.tools import assert_equal, assert_raises
+from nose.plugins.skip import SkipTest
 from pylons import config
 
 import ckan
@@ -1014,7 +1015,11 @@ def test_38_user_role_bulk_update(self):
 
     def test_40_task_resource_status(self):
 
-        import ckan.lib.celery_app as celery_app
+        try:
+            import ckan.lib.celery_app as celery_app
+        except ImportError:
+            raise SkipTest('celery not installed')
+
         backend = celery_app.celery.backend
         ##This creates the database tables as a side effect, can not see another way
         ##to make tables unless you actually create a task.

diff --git a/ckan_deb/usr/bin/ckan-setup-solr b/ckan_deb/usr/bin/ckan-setup-solr
@@ -22,7 +22,7 @@ sed \
     -e "s,#JETTY_PORT=8080,JETTY_PORT=8983," \
     -i /etc/default/jetty
 mv /usr/share/solr/conf/schema.xml /usr/share/solr/conf/schema.xml.`date --utc "+%Y-%m-%d_%T"`.bak 
-ln -s /usr/lib/pymodules/python2.6/ckan/config/solr/schema-1.3.xml /usr/share/solr/conf/schema.xml
+ln -s /usr/lib/pymodules/python2.6/ckan/config/solr/schema-1.4.xml /usr/share/solr/conf/schema.xml
 service jetty stop
 service jetty start
 echo "done."
diff --git a/ckanext/multilingual/plugin.py b/ckanext/multilingual/plugin.py
@@ -28,7 +28,7 @@ def translate_data_dict(data_dict):
             continue
         elif isinstance(value, basestring):
             terms.add(value)
-        elif isinstance(value, int):
+        elif isinstance(value, (int, long)):
             continue
         else:
             for item in value:
@@ -75,7 +75,7 @@ def translate_data_dict(data_dict):
                 translated_flattened[key] = fallback_translations.get(
                         value, value)
 
-        elif isinstance(value, (int, dict)):
+        elif isinstance(value, (int, long, dict)):
             translated_flattened[key] = value
 
         else:

diff --git a/doc/api-v2.rst b/doc/api-v2.rst
@@ -6,7 +6,6 @@ The CKAN API version 1 & 2 is separated into three parts.
 
 * `Model API`_
 * `Search API`_
-* `Util API`_
 
 The resources, methods, and data formats of each are described below.
 

diff --git a/doc/background-tasks.rst b/doc/background-tasks.rst
@@ -23,11 +23,14 @@ Enabling Background Tasks
 
 To manage and run background tasks requires a job queue and CKAN uses celery_
 (plus the CKAN database) for this purpose. Thus, to use background tasks you
-need to install and run celery_.
+need to install and run celery_. As of CKAN 1.7, celery is a required library
+and will be already installed after a default CKAN install.
 
 Installation of celery_ will normally be taken care of by whichever component
 or extension utilizes it so we skip that here.
 
+.. _celery: http://celeryproject.org/
+
 To run the celery daemon you have two options:
 
 1. In development setup you can just use paster. This can be done as simply

diff --git a/doc/commenting.rst b/doc/commenting.rst
@@ -17,7 +17,7 @@ Once installed and enabled, the presence of comments on a given page is
 configured by your theme (see the documentation in the disqus extension for
 details).
 
-In the default CKAN theme (as of v1.6.1), comments will be shown only on dataset
+In the default CKAN theme (as of v1.7), comments will be shown only on dataset
 and resource pages (and recent comments only on the
 :doc:`administrative-dashboard`).