Merge branch 'master' of github.com:ckan/ckan into 1377-javascript-tu…

…torial

CHANGELOG.rst

@@ -7,22 +7,183 @@
 Changelog
 ---------
 
-v2.2
-====
+v2.2 2014-02-04
+===============
+
+Note: This version does not require a requirements upgrade on source installations
+
+Note: This version requires a database upgrade
+
+Note: This version requires a Solr schema upgrade (The Solr schema file has
+been renamed, the schema file from the previous release is compatible
+with this version, but users are encouraged to point to the new one,
+see "API changes and deprecations")
+
+
+Major:
+ * Brand new automatic importer of tabular data to the DataStore, the
+   DataPusher. This is much more robust and simple to deploy and maintain than
+   its predecesor (ckanext-datastorer). Whole new UI for re-importing data to
+   the DataStore and view the import logs (#932, #938, #940, #981, #1196, #1200
+   ...)
+ * Completely revamped file uploads that allow closer integration with resources
+   and the DataStore, as well as making easir to integrate file uploads in other
+   features. For example users can now upload images for organizations and
+   groups. See "API changes and deprecations" if you are using the current
+   FileStore. (#1273, #1173 ... )
+ * UI and API endpoints for resource reordering (#1277)
+ * Backend support for organization hierarchy, allowing parent and children
+   organizations. Frontend needs to be implemented in extensions (#1038)
+ * User invitations: it is now possible to create new users with just their
+   email address. An invite email is sent to them, allowing to change their user
+   name and password (#1178)
+ * Disable user registration with a configuration option (#1226)
+ * Great effort in improving documentation, specially for customizing CKAN, with
+   a complete tutorial for writing extensions and customizing the theme. User
+   and sysadmin guides have also been moved to the main documentation
+   (#943, #847, #1253)
+
+Minor:
+ * Homepage modules to allow predefined layouts (#1126)
+ * Ability to delete users (#1163)
+ * Dedicated dataset groups page for displaying and managing them (#1102)
+ * Implement organization_purge and group_purge action functions (#707)
+ * Improve package_show performance (#1078)
+ * Support internationalization of rendered dates and times (#1041)
+ * Improve plugin load handling (#549)
+ * Authorization function auditing for action functions (#1060)
+ * Improve datetime rendering (#518)
+ * New SQL indexes to improve performance (#1164)
+ * Changes in requirements management (#1149)
+ * Add offset/limit to package_list action (#1179)
+ * Document all available configuraton options (#848)
+ * Make CKAN sqlalchemy 0.8.4 compatible (#1427)
+ * UI labelling and cleanup (#1030)
+ * Better UX for empty groups/orgs (#1094)
+ * Improve performance of group_dictize when the group has a lot of packages
+   (#1208)
+ * Hide __extras from extras on package_show (#1218)
+ * "Clear all" link within each facet block is unnecessary  (#1263)
+ * Term translations of organizations (#1274)
+ * '--reset-db' option for when running tests (#1304)
+
+Bug fixes:
+ * Fix plugins load/unload issues (#547)
+ * Improve performance when new_activities not needed (#1013)
+ * Resource preview breaks when CSV headers include percent sign (#1067)
+ * Package index not rebuilt when resources deleted (#1081)
+ * Don't accept invalid URLs in resource proxy (#1106)
+ * UI language reset after account creation (#1429)
+ * Catch non-integer facet limits (#1118)
+ * Error when deleting custom tags (#1114)
+ * Organization images do not display on Organization user dashboard page
+   (#1127)
+ * Can not reactivate a deleted dataset from the UI (#607)
+ * Non-existent user profile should give error (#1068)
+ * Recaptcha not working in CKAN 2.0 (jinja templates) (#1070)
+ * Groups and organizations can be visited with interchangeable URLs (#1180)
+ * Dataset Source (url) and Version fields missing (#1187)
+ * Fix problems with private / public datasets and organizations (#1188)
+ * group_show should never return private data (#1191)
+ * When editing a dataset, the organization field is not set (#1199)
+ * Fix resource_delete action (#1216)
+ * Fix trash purge action redirect broken for CKAN instances not at / (#1217)
+ * Title edit for existing dataset changes the URL (#1232)
+ * 'facet.limit' in package_search wrongly handled (#1237)
+ * h.SI_number_span doesn't close <span /> correctly (#1238)
+ * CkanVersionException wrongly raised (#1241)
+ * (group|organization)_member_create only accepts username (and not id) (#1243)
+ * package_create uses the wrong parameter for organization (#1257)
+ * ValueError for non-int limit and offset query params (#1258)
+ * Visibility field value not kept if there are errors on the form (#1265)
+ * package_list should not return private datasets (#1295)
+ * Fix 404 on organization activity stream and about page (#1298)
+ * Fix placeholder images broken on non-root locations (#1309)
+ * "Add Dataset" button shown on org pages when not authorized (#1348)
+ * Fix exception when visiting organization history page (#1359)
+ * Fix search ordering on organization home page (#1368)
+ * datastore_search_sql failing for some anonymous users (#1373)
+ * related_list logic function throws a 503 without any parameters (#1384)
+ * Disabling activity_streams borks editing groups and user (#1421)
+ * Member Editing Fixes (#1454)
+ * Bulk editing broken in IE7 (#1455)
+ * Fix group deletion in IE7 (#1460)
+ * And many, many more!
 
 API changes and deprecations:
+ * The Solr schema file is now always named ``schema.xml`` regardless of the
+   CKAN version. Old schema files have been kept for backwards compatibility
+   but users are encouraged to point to the new unified one (#1314)
+ * The FileStore and file uploads have been completely refactored and simplified
+   to only support local storage backend. The links from previous versions of
+   the FileStore to hosted files will still work, but there is a command
+   available to migrate the files to new Filestore. See this page for more
+   details:
+   http://docs.ckan.org/en/latest/filestore.html#filestore-21-to-22-migration
+ * By default, the authorization for any action defined from an extension will
+   require a logged in user, otherwise a :py:class:`ckan.logic.NotAuthorized`
+   exception will be raised. If an action function allows anonymous access (eg
+   search, show status, etc) the ``auth_allow_anonymous_access`` decorator
+   (available on the plugins toolkit) must be used (#1210)
+ * ``package_search`` now returns results with custom schemas applied like
+   ``package_show``, a ``use_default_schema`` parameter was added to request the
+   old behaviour, this change may affect customized search result templates
+   (#1255)
+ * The ``ckan.api_url`` configuration option has been completely removed and it
+   can no longer be used (#960)
+ * The ``edit`` and ``after_update`` methods of IPackageController plugins are now
+   called when updating a resource using the web frontend or the
+   resource_update API action (#1052)
+ * Dataset moderation has been deprecated, and the code will probably be removed
+   in later CKAN versions (#1139)
+ * Some front end libraries have been updated, this may affect existing custom
+   themes: Bootstrap 2.0.3 > 2.3.2, Font Awesome 3.0.2 > 3.2.1,
+   jQuery 1.7.2 > 1.10.2 (#1082)
+ * SQLite is officially no longer supported as the tests backend
+
+Troubleshooting:
+ * Exception on startup after upgrading from a previous CKAN version::
+
+     AttributeError: 'instancemethod' object has no attribute 'auth_audit_exempt'
+
+   Make sure that you are not loading a 2.1-only plugin (eg ``datapusher-ext``)
+   and update all the plugin in your configuration file to the latest stable
+   version.
+
+ * Exception on startup after upgrading from a previous CKAN version::
+
+     File "/usr/lib/ckan/default/src/ckan/ckan/lib/dictization/model_dictize.py", line 330, in package_dictize
+         result_dict['metadata_modified'] = pkg.metadata_modified.isoformat()
+     AttributeError: 'NoneType' object has no attribute 'isoformat'
+
+   One of the database changes on this version is the addition of a
+   ``metadata_modified`` field in the package table, that was filled during the
+   DB migration process. If you have previously migrated the database and revert
+   to an older CKAN version the migration process may have failed at this step,
+   leaving the fields empty. Also make sure to restart running processes like
+   harvesters after the update to make sure they use the new code base.
+
+v2.1.2 2014-02-04
+=================
 
-* The Solr schema file is now always named ``schema.xml`` regardless of the
-  CKAN version. Old schema files have been kept for backwards compatibility
-  but users are encouraged to point to the new unified one.
-* The `ckan.api_url` has been completely removed and it can no longer be used
-* The edit() and after_update() methods of IPackageController plugins are now
-  called when updating a resource using the web frontend or the
-  resource_update API action [#1052]
-* package_search now returns results with custom schemas applied like
-  package_show, a use_default_schema parameter was added to request the
-  old behaviour, this change may affect customized search result templates
-  (#1255)
+Bug fixes:
+ * Fix context for group/about setup_template_variables (#1433)
+ * Call setup_template_variables in group/org read, about and bulk_process (#1281)
+ * Remove repeated sort code in package_search (#1461)
+ * Ensure that check_access is called on activity_create (#1421)
+ * Fix visibility validator (#1188)
+ * Remove p.toolkit.auth_allow_anonymous_access as it is not available on 2.1.x (#1373)
+ * Add organization_revision_list to avoid exception on org history page (#1359)
+ * Fix activity and about organization pages (#1298)
+ * Show 404 instead of login page on user not found (#1068)
+ * Don't show Add Dataset button on org pages unless authorized (#1348)
+ * Fix datastore_search_sql authorization function (#1373)
+ * Fix extras deletion (#1449)
+ * Better word breaking on long words (#1398)
+ * Fix activity and about organization pages (#1298)
+ * Remove limit of number of arguments passed to ``user add`` command.
+ * Fix related_list logic function (#1384)
+ * Avoid UnicodeEncodeError on feeds when params contains non ascii characters
 
 v2.1.1 2013-11-8
 ================
@@ -120,6 +281,17 @@ Known issues:
  * Under certain authorization setups the frontend for the groups functionality
    may not work as expected (See #1176 #1175).
 
+v2.0.4 2014-02-04
+=================
+
+Bug fixes:
+ * Fix extras deletion (#1449)
+ * Better word breaking on long words (#1398)
+ * Fix activity and about organization pages (#1298)
+ * Show 404 instead of login page on user not found (#1068)
+ * Remove limit of number of arguments passed to ``user add`` command.
+ * Fix related_list logic function (#1384)
+
 v2.0.3 2013-11-8
 ================
 
@@ -188,7 +360,7 @@ Note: This version requires a database upgrade
 
 Note: This version requires a Solr schema upgrade
 
-Organizations based authorization (see :doc:`authorization`):
+Organizations based authorization (see :doc:`/maintaining/authorization`):
  CKAN's new "organizations" feature replaces the old authorization system
  with a new one based on publisher organizations. It replaces the "Publisher
  Profile and Workflow" feature from CKAN 1.X, any instances relying on it will
@@ -200,7 +372,7 @@ Organizations based authorization (see :doc:`authorization`):
  * New authorization ini file options
 
 
-New frontend (see :doc:`theming/index`):
+New frontend (see :doc:`/theming/index`):
  CKAN's frontend has been completely redesigned, inside and out. There is
  a new default theme and the template engine has moved from Genshi to
  Jinja2. Any custom templates using Genshi will need to be updated, although

ckan/__init__.py

@@ -1,6 +1,6 @@
-__version__ = '2.2a'
+__version__ = '2.3a'
 
-__description__ = 'Comprehensive Knowledge Archive Network (CKAN) Software'
+__description__ = 'CKAN Software'
 __long_description__ = \
 '''CKAN software provides a hub for datasets. The flagship site running CKAN 
 is theDataHub.org but it is also used for dozens of other open data websites
@@ -11,7 +11,6 @@
 population density database, the voting records of MPs, or 30 years of 
 US patents.
 
-CKAN is an abbreviation for 'Comprehensive Knowledge Archive Network'. 
 Those familiar with Freshmeat or CPAN can think of CKAN as providing an
 analogous service for open data and knowledge. 
 '''

ckan/lib/navl/validators.py

@@ -65,13 +65,13 @@ def ignore(key, data, errors, context):
     value = data.pop(key, None)
     raise StopOnError
 
-def default(defalult_value):
+def default(default_value):
 
     def callable(key, data, errors, context):
 
         value = data.get(key)
         if not value or value is missing:
-            data[key] = defalult_value
+            data[key] = default_value
 
     return callable
 

ckan/logic/__init__.py

@@ -526,8 +526,8 @@ def side_effect_free(action):
     '''A decorator that marks the given action function as side-effect-free.
 
     Action functions decorated with this decorator can be called with an HTTP
-    GET request to the :doc:`Action API </api>`. Action functions that don't
-    have this decorator must be called with a POST request.
+    GET request to the :doc:`Action API </api/index>`. Action functions that
+    don't have this decorator must be called with a POST request.
 
     If your CKAN extension defines its own action functions using the
     :py:class:`~ckan.plugins.interfaces.IActions` plugin interface, you can use
@@ -559,9 +559,9 @@ def auth_sysadmins_check(action):
     to call an action function.
 
     Normally sysadmins are allowed to call any action function (for example
-    when they're using the :doc:`Action API </api>` or the web interface),
-    if the user is a sysadmin the action function's authorization function
-    will not even be called.
+    when they're using the :doc:`Action API </api/index>` or the web
+    interface), if the user is a sysadmin the action function's authorization
+    function will not even be called.
 
     If an action function is decorated with this decorator, then its
     authorization function will always be called, even if the user is a

ckan/logic/auth/__init__.py

@@ -15,6 +15,9 @@ def _get_object(context, data_dict, name, class_name):
         if not data_dict:
             data_dict = {}
         id = data_dict.get('id', None)
+        if not id:
+            raise logic.ValidationError('Missig id, can not get {0} object'
+                                        .format(class_name))
         obj = getattr(model, class_name).get(id)
         if not obj:
             raise logic.NotFound

ckan/logic/auth/delete.py

@@ -1,8 +1,9 @@
 import ckan.logic as logic
 import ckan.new_authz as new_authz
-from ckan.logic.auth import get_package_object, get_group_object, get_related_object
+from ckan.logic.auth import get_group_object, get_related_object
 from ckan.logic.auth import get_resource_object
 import ckan.logic.auth.create as _auth_create
+import ckan.logic.auth.update as _auth_update
 from ckan.lib.base import _
 
 
@@ -12,14 +13,9 @@ def user_delete(context, data_dict):
 
 
 def package_delete(context, data_dict):
-    user = context['user']
-    package = get_package_object(context, data_dict)
-
-    authorized = new_authz.has_user_permission_for_group_or_org(package.owner_org, user, 'delete_dataset')
-    if not authorized:
-        return {'success': False, 'msg': _('User %s not authorized to delete package %s') % (user, package.id)}
-    else:
-        return {'success': True}
+    # Defer auhtorization for package_delete to package_update, as deletions
+    # are essentially changing the state field
+    return _auth_update.package_update(context, data_dict)
 
 def resource_delete(context, data_dict):
     model = context['model']