Merge branch 'master' of github.com:ckan/ckan into joetsoi-more-packa…

…ge-controller-tests

.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,10 @@
+### CKAN Version if known (or site URL)
+
+
+### Please describe the expected behaviour
+
+
+### Please describe the actual behaviour
+
+
+### What steps can be taken to reproduce the issue? 

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,15 @@
+Fixes #
+
+### Proposed fixes:
+
+
+
+### Features:
+
+- [ ] includes tests covering changes
+- [ ] includes updated documentation
+- [ ] includes user-visible changes
+- [ ] includes API changes
+- [ ] includes bugfix for possible backport
+
+Please [X] all the boxes above that apply

.travis.yml

@@ -1,10 +1,20 @@
 language: python
+sudo: required
 python:
   - "2.6"
   - "2.7"
 env:
-  - PGVERSION=9.1
-  - PGVERSION=8.4
+  global:
+    - CKAN_POSTGRES_DB=ckan_test
+    - CKAN_POSTGRES_USER=ckan_default
+    - CKAN_POSTGRES_PWD=pass
+    - CKAN_DATASTORE_POSTGRES_DB=datastore_test
+    - CKAN_DATASTORE_POSTGRES_WRITE_USER=ckan_default
+    - CKAN_DATASTORE_POSTGRES_READ_USER=datastore_default
+    - CKAN_DATASTORE_POSTGRES_READ_PWD=pass
+  matrix:
+    - PGVERSION=9.1
+    - PGVERSION=8.4
 matrix:
   exclude:
     - python: "2.7"

.tx/config

@@ -1,7 +1,7 @@
 [main]
 host = https://www.transifex.com
 
-[ckan.2-2]
+[ckan.2-4]
 file_filter = ckan/i18n/<lang>/LC_MESSAGES/ckan.po
 source_file = ckan/i18n/ckan.pot
 source_lang = en

CHANGELOG.rst

@@ -7,12 +7,158 @@
 Changelog
 ---------
 
-v2.4
-====
+v2.5.1 2015-12-17
+=================
+
+Note: This version requires a requirements upgrade on source installations
+
+Note: This version requires a database upgrade
+
+Note: This version does not require a Solr schema upgrade
+
+Major:
+ * CKAN extension language translations integrated using ITranslations interface (#2461, #2643)
+ * Speed improvements for displaying a dataset (#2234), home page (#2554), searching (#2382, #2724) and API actions: package_show (#1078) and user_list (#2752).
+ * An interface to replace the file uploader, allowing integration with other cloud storage providers (IUploader interface) (#2510)
+
+Minor:
+ * package_purge API action added (#1572)
+ * revision_list API action now has paging (#1431)
+ * Official Ubuntu 14.04 LTS support (#1651)
+ * Require/validate current password before allowing a password change (#1940)
+ * recline_map_view now recognizes GeoJSON fileds (#2387)
+ * Timezone setting (#2494)
+ * Updating a resource via upload now saves the last_modified value in the resource (#2519)
+ * DataPusher can be customized using the new IDataPusher interface (#2571)
+ * Exporting and importing users, with their passwords (if sysadmin) (#2647)
+
+Bug fixes:
+ * Fix to allow uppercase letters in local part of email when sending user invitations (#2415)
+ * License pick-list changes would cause old values in datasets to be overwritten when edited (#2472)
+ * Schema was being passed to package_create_default_resource_views (#2484)
+ * Arabic translation format string issue (#2493)
+ * Error when deleting organizations (#2512)
+ * When DataPusher had an error storing a resource in Data Store, the resource data page gave an error (#2518)
+ * Data preview failed when it comes from a server that gives 403 error from a HEAD request (#2530)
+ * 'paster views create' failed for non-default dataset types (#2532)
+ * DataPusher didn't work for TSV files (#2553)
+ * DataPusher failed sometimes due to 'type mismatch' (#2581)
+ * IGroupForm wasn't allowing new groups (of type 'group') to use group_form (#2617, #2640)
+ * group_purge left behind a Member if it has a parent group/org (#2631)
+ * organization_purge left orphaned datasets still with owner_id (#2632)
+ * Fix Markdown rendering issue
+ * Return default error page on fanstatic errors
+ * Prevent authentication when using API callbacks
+
+Changes and deprecations
+------------------------
+
+* The old RDF templates to output a dataset in RDF/XML or N3 format have been
+  removed. These can be now enabled using the ``dcat`` plugin on *ckanext-dcat*:
+
+    https://github.com/ckan/ckanext-dcat#rdf-dcat-endpoints
+
+* The library used to render markdown has been changed to python-markdown. This
+  introduces both ``python-markdown`` and ``bleach`` as dependencies, as ``bleach``
+  is used to clean any HTML provided to the markdown processor.
+
+* This is the last version of CKAN to support Postgresql 8.x, 9.0 and 9.1. The
+  next minor version of CKAN will require Postgresql 9.2 or later.
+
+
+v2.5.0 2015-12-17
+=================
+
+Cancelled release
+
+
+v2.4.2 2015-12-17
+=================
+
+Note: This version requires a requirements upgrade on source installations
+
+Bug fixes:
+ * Fix Markdown rendering issue
+ * Return default error page on fanstatic errors
+ * Prevent authentication when using API callbacks
+
+
+v2.4.1 2015-09-02
+=================
+
+Note: #2554 fixes a regression where ``group_list`` and ``organization_list``
+      where returning extra additional fields by default, causing performance
+      issues. This is now fixed, so the output for these actions no longer returns
+      ``users``, ``extras``, etc.
+      Also, on the homepage template the ``c.groups`` and ``c.group_package_stuff``
+      context variables are no longer available.
+
+
+Bug fixes:
+
+* Fix dataset count in templates and show datasets on featured org/group (#2557)
+* Fix autodetect for TSV resources (#2553)
+* Improve character escaping in DataStore parameters
+* Fix "paster db init" when celery is configured with a non-database backend
+* Fix severe performance issues with groups and orgs listings (#2554)
+
+
+v2.4.0 2015-07-22
+=================
+
+Note: This version requires a database upgrade
+
+Note: This version requires a Solr schema upgrade
+
+Major:
+ * CKAN config can now be set from environment variables and via the API (#2429)
+
+Minor:
+ * API calls now faster: ``group_show``, ``organization_show``, ``user_show``,
+   ``package_show``, ``vocabulary_show`` & ``tag_show`` (#1886, #2206, #2207,
+   #2376)
+ * Require/validate current password before allowing a password change (#1940)
+ * Added ``organization_autocomplete`` action (#2125)
+ * Default authorization no longer allows anyone to create datasets etc (#2164)
+ * ``organization_list_for_user`` now returns organizations in hierarchy if they
+   exist for roles set in ``ckan.auth.roles_that_cascade_to_sub_groups`` (#2199)
+ * Improved accessibility (text based browsers) focused on the page header
+   (#2258)
+ * Improved IGroupForm for better customizing groups and organization behaviour
+   (#2354)
+ * Admin page can now be extended to have new tabs (#2351)
+
+
+Bug fixes:
+ * Command line ``paster user`` failed for non-ascii characters (#1244)
+ * Memory leak fixed in datastore API (#1847)
+ * Modifying resource didn't update it's last updated timestamp (#1874)
+ * Datastore didn't update if you uploaded a new file of the same name as the
+   existing file (#2147)
+ * Files with really long file were skipped by datapusher (#2057)
+ * Multi-lingual Solr schema is now updated so it works again (#2161)
+ * Resource views didn't display when embedded in another site (#2238)
+ * ``resource_update`` failed if you supplied a revision_id (#2340)
+ * Recline could not plot GeoJSON on a map (#2387)
+ * Dataset create form 404 error if you added a resource but left it blank (#2392)
+ * Editing a resource view for a file that was UTF-8 and had a BOM gave an
+   error (#2401)
+ * Email invites had the email address changed to lower-case (#2415)
+ * Default resource views not created when using a custom dataset schema (#2421,
+   #2482)
+ * If the licenses pick-list was customized to remove some, datasets with old
+   values had them overwritten when edited (#2472)
+ * Recline views failed on some non-ascii characters (#2490)
+ * Resource proxy failed if HEAD responds with 403 (#2530)
+ * Resource views for non-default dataset types couldn't be created (#2532)
 
 Changes and deprecations
 ------------------------
 
+* The default of allowing anyone to create datasets, groups and organizations
+  has been changed to False. It is advised to ensure you set all of the
+  :ref:`config-authorization` options explicitly in your CKAN config. (#2164)
+
 * The ``package_show`` API call does not return the ``tracking_summary``,
   keys in the dataset or resources by default any more.
 
@@ -23,27 +169,67 @@ Changes and deprecations
   `new_tests` directory has moved to `tests` and the `new_authz.py`
   module has been renamed `authz.py`. Code that imports names from the
   old locations will continue to work in this release but will issue
-  a deprecation warning.
+  a deprecation warning. (#1753)
 
-* Add text to account links in header, fixes text based browser support #2258
+* ``group_show`` and ``organization_show`` API calls no longer return the
+  datasets by default (#2206)
 
-* Add middleware that cleans up the response string after it has been
-  served, stabilizes memory usage for large requests #1847
+  Custom templates or users of this API call will need to pass
+  ``include_datasets=True`` to include datasets in the response.
 
 * The ``vocabulary_show`` and ``tag_show`` API calls no longer returns the 
   ``packages`` key - i.e. datasets that use the vocabulary or tag. 
   However ``tag_show`` now has an ``include_datasets`` option. (#1886)
 
-* `organization_list_for_user` now returns organizations in hierarchy if they
-  exist for roles set in `ckan.auth.roles_that_cascade_to_sub_groups`.
+* Config option ``site_url`` is now required - CKAN will not abort during
+  start-up if it is not set. (#1976)
 
-* Update license keys to match opendefinition.org #2110
+v2.3.3 2015-12-17
+=================
 
-* The ``group_show`` and ``organization_show`` API calls do not return
-  ``datasets`` by default any more.
+Bug fixes:
+ * Fix Markdown rendering issue
+ * Return default error page on fanstatic errors
+ * Prevent authentication when using API callbacks
+
+v2.3.3 2015-12-17
+=================
+
+Note: This version requires a requirements upgrade on source installations
+
+Bug fixes:
+ * Fix Markdown rendering issue
+ * Return default error page on fanstatic errors
+ * Prevent authentication when using API callbacks
+
+
+v2.3.2 2015-09-02
+=================
+
+Bug fixes:
+* Fix autodetect for TSV resources (#2553)
+* Improve character escaping in DataStore parameters
+* Fix "paster db init" when celery is configured with a non-database backend
+
+
+v2.3.1 2015-07-22
+=================
+
+Bug fixes:
+ * Resource views won't display when embedded in another site (#2238)
+ * ``resource_update`` failed if you supplied a revision_id (#2340)
+ * Recline could not plot GeoJSON on a map (#2387)
+ * Dataset create form 404 error if you added a resource but left it blank (#2392)
+ * Editing a resource view for a file that was UTF-8 and had a BOM gave an
+   error (#2401)
+ * Email invites had the email address changed to lower-case (#2415)
+ * Default resource views not created when using a custom dataset schema (#2421,
+   #2482)
+ * If the licenses pick-list was customized to remove some, datasets with old
+   values had them overwritten when edited (#2472)
+ * Recline views failed on some non-ascii characters (#2490)
+ * Resource views for non-default dataset types couldn't be created (#2532)
 
-  Custom templates or users of this API call will need to pass
-  ``include_datasets=True`` to include datasets in the response.
 
 v2.3 2015-03-04
 ===============
@@ -200,7 +386,6 @@ Bug fixes:
  * Make resource_create auth work against package_update (#2037)
  * Fix DataStore permissions check on startup (#1374)
  * Fix datastore docs link (#2044)
- * Fix resource extras getting lost on resource update (#2158)
  * Clean up field names before rendering the Recline table (#2319)
  * Don't "normalize" resource URL in recline view (#2324)
  * Don't assume resource format is there on text preview (#2320)
@@ -341,6 +526,27 @@ Troubleshooting:
 
   Also see the previous point for other ``who.ini`` changes.
 
+v2.2.4 2015-12-17
+=================
+
+Note: This version requires a requirements upgrade on source installations
+
+Bug fixes:
+ * Fix Markdown rendering issue
+ * Return default error page on fanstatic errors
+ * Prevent authentication when using API callbacks
+
+v2.2.3 2015-07-22
+=================
+
+Bug fixes:
+ * Allow uppercase emails on user invites (#2415)
+ * Fix broken boolean validator (#2443)
+ * Fix auth check in resources_list.html (#2037)
+ * Key error on resource proxy (#2425)
+ * Ignore revision_id passed to resources (#2340)
+ * Add reset for reset_key on successful password change (#2379)
+
 v2.2.2 2015-03-04
 =================
 
@@ -562,6 +768,25 @@ Troubleshooting:
    leaving the fields empty. Also make sure to restart running processes like
    harvesters after the update to make sure they use the new code base.
 
+v2.1.6 2015-12-17
+=================
+
+Note: This version requires a requirements upgrade on source installations
+
+Bug fixes:
+ * Fix Markdown rendering issue
+ * Return default error page on fanstatic errors
+ * Prevent authentication when using API callbacks
+
+v2.1.5 2015-07-22
+=================
+
+Bug fixes:
+ * Fix broken boolean validator (#2443)
+ * Key error on resource proxy (#2425)
+ * Ignore revision_id passed to resources (#2340)
+ * Add reset for reset_key on successful password change (#2379)
+
 v2.1.4 2015-03-04
 =================
 
@@ -720,6 +945,25 @@ Known issues:
  * Under certain authorization setups the frontend for the groups functionality
    may not work as expected (See #1176 #1175).
 
+v2.0.8 2015-12-17
+=================
+
+Note: This version requires a requirements upgrade on source installations
+
+Bug fixes:
+ * Fix Markdown rendering issue
+ * Return default error page on fanstatic errors
+ * Prevent authentication when using API callbacks
+
+v2.0.7 2015-07-22
+=================
+
+Bug fixes:
+ * Fix broken boolean validator (#2443)
+ * Key error on resource proxy (#2425)
+ * Ignore revision_id passed to resources (#2340)
+ * Add reset for reset_key on successful password change (#2379)
+
 v2.0.6 2015-03-04
 =================