Skip to content

Commit

Permalink
Merge branch 'master' into remove-vdm
Browse files Browse the repository at this point in the history
  • Loading branch information
amercader committed Sep 20, 2019
2 parents 3bffbd6 + f2e5bf9 commit 9b8b2a3
Show file tree
Hide file tree
Showing 230 changed files with 24,442 additions and 10,242 deletions.
4 changes: 2 additions & 2 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ version: 2
jobs:
test:
docker:
- image: python:2
- image: python:2-stretch
environment:
CKAN_DATASTORE_POSTGRES_DB: datastore_test
CKAN_DATASTORE_POSTGRES_READ_USER: datastore_read
Expand All @@ -29,7 +29,7 @@ jobs:
- checkout

- run: |
# SO Dependencies
# OS Dependencies
apt update
case $CIRCLE_NODE_INDEX in
$NODE_TESTS_CONTAINER)
Expand Down
94 changes: 93 additions & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,24 @@ Changelog
v.2.9.0 TBA
==================

* This version requires changes to the ``who.ini`` configuration file. If your
setup doesn't use the one bundled with this repo, you will have to manually
change the following lines::

use = ckan.lib.auth_tkt:make_plugin

to::

use = ckan.lib.repoze_plugins.auth_tkt:make_plugin

And also::

use = repoze.who.plugins.friendlyform:FriendlyFormPlugin

to::

use = ckan.lib.repoze_plugins.friendly_form:FriendlyFormPlugin

* When upgrading from previous CKAN versions, the Activity Stream needs a
migrate_package_activity.py running for displaying the history of dataset
changes. This can be performed while CKAN is running or stopped (whereas the
Expand Down Expand Up @@ -79,6 +97,39 @@ Removals and deprecations:
dataset blueprint in CKAN>=2.9, and, in the same time, it's still working for
Pylons package controller in CKAN<2.9

v.2.8.3 2019-07-03
==================

General notes:
* Note: This version does not requires a requirements upgrade on source installations
* Note: This version does not requires a database upgrade
* Note: This version does not require a Solr schema upgrade

Fixes:

* Fix `include_total` in `datastore_search` (`#4446 <https://github.com/ckan/ckan/issues/4446>`_)
* Fix problem with reindex-fast (`#4352 <https://github.com/ckan/ckan/issues/4352>`_)
* Fix `ValueError` in `url_validator` (`#4629 <https://github.com/ckan/ckan/issues/4629>`_)
* Strip local path when uploading file in IE (`#4608 <https://github.com/ckan/ckan/issues/4608>`_)
* Increase size of h1 headings to 1.8em (`#4665 <https://github.com/ckan/ckan/issues/4665>`_)
* Fix broken div nesting in the `user/read_base.html` (`#4672 <https://github.com/ckan/ckan/issues/4672>`_)
* `package_search` parameter `fl` accepts list-like values (`#4464 <https://github.com/ckan/ckan/issues/4464>`_)
* Use `chained_auth_function` with core auth functions (`#4491 <https://github.com/ckan/ckan/issues/4491>`_)
* Allow translation of custom licenses (`#4594 <https://github.com/ckan/ckan/issues/4594>`_)
* Fix delete button links (`#4598 <https://github.com/ckan/ckan/issues/4598>`_)
* Fix hardcoded root paths (`#4662 <https://github.com/ckan/ckan/issues/4662>`_)
* Fix reCaptcha (`#4732 <https://github.com/ckan/ckan/issues/4732>`_)
* Fix incremented follower-counter (`#4767 <https://github.com/ckan/ckan/issues/4767>`_)
* Fix breadcrumb on /datasets (`#4405 <https://github.com/ckan/ckan/issues/4405>`_)
* Fix `root_path` when using mod_wsgi (`#4452 <https://github.com/ckan/ckan/issues/4452>`_)
* Correctly insert root_path for urls generated with _external flag (`#4722 <https://github.com/ckan/ckan/issues/4722>`_)
* Make reorder resources button translatable (`#4838 <https://github.com/ckan/ckan/issues/4838>`_)
* Fix `feeds` urls generation (`#4854 <https://github.com/ckan/ckan/pull/4854>`_)
* More robust auth functions for `resource_view_show` (`#4827 <https://github.com/ckan/ckan/issues/4827>`_)
* Allow to customize the DataProxy URL (`#4874 <https://github.com/ckan/ckan/issues/4874>`_)
* Allow custom CKAN callback URL for the DataPusher (`#4878 <https://github.com/ckan/ckan/issues/4878>`_)
* Add `psycopg>=2.8` support (`#4841 <https://github.com/ckan/ckan/pull/4841>`_)

v.2.8.2 2018-12-12
==================

Expand Down Expand Up @@ -252,6 +303,28 @@ Changes and deprecations:
to specify this argument explicitly, as in future it'll become required.
* The ``ckan.recaptcha.version`` config option is now removed, since v2 is the only valid version now (#4061)

v.2.7.6 2019-07-03
==================

General notes:
* Note: This version does not requires a requirements upgrade on source installations
* Note: This version does not requires a database upgrade
* Note: This version does not require a Solr schema upgrade

Fixes:

* Fix problem with reindex-fast (`#4352 <https://github.com/ckan/ckan/issues/4352>`_)
* Fix `include_total` in `datastore_search` (`#4446 <https://github.com/ckan/ckan/issues/4446>`_)
* Fix `ValueError` in `url_validator` (`#4629 <https://github.com/ckan/ckan/issues/4629>`_)
* Strip local path when uploading file in IE (`#4608 <https://github.com/ckan/ckan/issues/4608>`_)
* Increase size of h1 headings to 1.8em (`#4665 <https://github.com/ckan/ckan/issues/4665>`_)
* Fix broken div nesting in the `user/read_base.html` (`#4672 <https://github.com/ckan/ckan/issues/4672>`_)
* Use `get_action` to call activity actions (`#4684 <https://github.com/ckan/ckan/issues/4684>`_)
* Make reorder resources button translatable (`#4838 <https://github.com/ckan/ckan/issues/4838>`_)
* More robust auth functions for `resource_view_show` (`#4827 <https://github.com/ckan/ckan/issues/4827>`_)
* Allow to customize the DataProxy URL (`#4874 <https://github.com/ckan/ckan/issues/4874>`_)
* Allow custom CKAN callback URL for the DataPusher (`#4878 <https://github.com/ckan/ckan/issues/4878>`_)

v2.7.5 2018-12-12
=================

Expand Down Expand Up @@ -286,7 +359,7 @@ General notes:
This is due to a bug in the psycopg2 version pinned to the release. To solve
it, upgrade psycopg2 with the following command::

pip install --upgrade psycopg2==2.7.3.2
pip install --upgrade psycopg2==2.8.2

* This release does not require a Solr schema upgrade, but if you are having the
issues described in #3863 (datasets wrongly indexed in multilingual setups),
Expand Down Expand Up @@ -376,6 +449,7 @@ Major changes:
* Common requests code for Flask and Pylons (#3212)
* Generate complete datastore dump files (#3344)
* A new system for asynchronous background jobs (#3165)
* Chaining of action functions (#3494)

Minor changes:
* Renamed example theme plugin (#3576)
Expand Down Expand Up @@ -450,6 +524,24 @@ Deprecations:
jobs (http://docs.ckan.org/en/latest/maintaining/background-tasks.html). Extensions can still
of course use Celery but they will need to handle the management themselves.

v.2.6.8 2019-07-03
==================

General notes:
* Note: This version does not requires a requirements upgrade on source installations
* Note: This version does not requires a database upgrade
* Note: This version does not require a Solr schema upgrade

Fixes:

* Fix broken div nesting in the `user/read_base.html` (`#4672 <https://github.com/ckan/ckan/issues/4672>`_)
* Strip local path when uploading file in IE (`#4608 <https://github.com/ckan/ckan/issues/4608>`_)
* Increase size of h1 headings to 1.8em (`#4665 <https://github.com/ckan/ckan/issues/4665>`_)
* Fix `ValueError` in `url_validator` (`#4629 <https://github.com/ckan/ckan/issues/4629>`_)
* More robust auth functions for `resource_view_show` (`#4827 <https://github.com/ckan/ckan/issues/4827>`_)
* Allow to customize the DataProxy URL (`#4874 <https://github.com/ckan/ckan/issues/4874>`_)
* Allow custom CKAN callback URL for the DataPusher (`#4878 <https://github.com/ckan/ckan/issues/4878>`_)

v2.6.7 2018-12-12
=================

Expand Down
2 changes: 1 addition & 1 deletion ckan/authz.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from logging import getLogger

from ckan.common import config
from paste.deploy.converters import asbool
from ckan.common import asbool

import ckan.plugins as p
import ckan.model as model
Expand Down
10 changes: 10 additions & 0 deletions ckan/cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@
datastore,
translation,
dataset,
plugin_info,
notify,
tracking,
minify,
less
)

from ckan.config.middleware import make_app
Expand Down Expand Up @@ -44,3 +49,8 @@ def ckan(ctx, config, *args, **kwargs):
ckan.add_command(datastore.datastore)
ckan.add_command(translation.translation)
ckan.add_command(dataset.dataset)
ckan.add_command(plugin_info.plugin_info)
ckan.add_command(notify.notify)
ckan.add_command(tracking.tracking)
ckan.add_command(minify.minify)
ckan.add_command(less.less)
2 changes: 1 addition & 1 deletion ckan/cli/datastore.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

import ckanext.datastore as datastore_module
from ckanext.datastore.backend.postgres import identifier
from ckanext.datastore.controller import DUMP_FORMATS, dump_to
from ckanext.datastore.view import DUMP_FORMATS, dump_to

log = logging.getLogger(__name__)

Expand Down
95 changes: 95 additions & 0 deletions ckan/cli/less.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# encoding: utf-8

import click
import subprocess
import os

from ckan.common import config
from ckan.cli import error_shout


_custom_css = {
u'fuchsia': u'''
@layoutLinkColor: #E73892;
@footerTextColor: mix(#FFF, @layoutLinkColor, 60%);
@footerLinkColor: @footerTextColor;
@mastheadBackgroundColor: @layoutLinkColor;
@btnPrimaryBackground: lighten(@layoutLinkColor, 10%);
@btnPrimaryBackgroundHighlight: @layoutLinkColor;
''',

u'green': u'''
@layoutLinkColor: #2F9B45;
@footerTextColor: mix(#FFF, @layoutLinkColor, 60%);
@footerLinkColor: @footerTextColor;
@mastheadBackgroundColor: @layoutLinkColor;
@btnPrimaryBackground: lighten(@layoutLinkColor, 10%);
@btnPrimaryBackgroundHighlight: @layoutLinkColor;
''',

u'red': u'''
@layoutLinkColor: #C14531;
@footerTextColor: mix(#FFF, @layoutLinkColor, 60%);
@footerLinkColor: @footerTextColor;
@mastheadBackgroundColor: @layoutLinkColor;
@btnPrimaryBackground: lighten(@layoutLinkColor, 10%);
@btnPrimaryBackgroundHighlight: @layoutLinkColor;
''',

u'maroon': u'''
@layoutLinkColor: #810606;
@footerTextColor: mix(#FFF, @layoutLinkColor, 60%);
@footerLinkColor: @footerTextColor;
@mastheadBackgroundColor: @layoutLinkColor;
@btnPrimaryBackground: lighten(@layoutLinkColor, 10%);
@btnPrimaryBackgroundHighlight: @layoutLinkColor;
''',
}


@click.command(
name=u'less',
short_help=u'Compile all root less documents into their CSS counterparts')
def less():
command = (u'npm', u'bin')
process = subprocess.Popen(
command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
universal_newlines=True)
output = process.communicate()
directory = output[0].strip()
if not directory:
error_shout(u'Command "{}" returned nothing. Check that npm is '
u'installed.'.format(u' '.join(command)))
less_bin = os.path.join(directory, u'lessc')

public = config.get(u'ckan.base_public_folder')

root = os.path.join(os.path.dirname(__file__), u'..', public, u'base')
root = os.path.abspath(root)
custom_less = os.path.join(root, u'less', u'custom.less')
for color in _custom_css:
f = open(custom_less, u'w')
f.write(_custom_css[color])
f.close()
_compile_less(root, less_bin, color)
f = open(custom_less, u'w')
f.write(u'// This file is needed in order for ./bin/less to '
u'compile in less 1.3.1+\n')
f.close()
_compile_less(root, less_bin, u'main')


def _compile_less(root, less_bin, color):
click.echo(u'compile {}.css'.format(color))
main_less = os.path.join(root, u'less', u'main.less')
main_css = os.path.join(root, u'css', u'{}.css'.format(color))
command = (less_bin, main_less, main_css)
process = subprocess.Popen(
command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
universal_newlines=True)
output = process.communicate()
click.echo(output)
80 changes: 80 additions & 0 deletions ckan/cli/minify.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# encoding: utf-8

import click
import os
import ckan.include.rjsmin as rjsmin
import ckan.include.rcssmin as rcssmin

_exclude_dirs = [u'vendor']


@click.command(name=u'minify')
@click.option(
u'--clean', is_flag=True, help=u'remove any minified files in the path.')
@click.argument(u'path', nargs=-1, type=click.Path())
def minify(clean, path):
u'''Create minified versions of the given Javascript and CSS files.'''
for base_path in path:
if os.path.isfile(base_path):
if clean:
_clear_minifyed(base_path)
else:
_minify_file(base_path)
elif os.path.isdir(base_path):
for root, dirs, files in os.walk(base_path):
dirs[:] = [d for d in dirs if d not in _exclude_dirs]
for filename in files:
path = os.path.join(root, filename)
if clean:
_clear_minifyed(path)
else:
_minify_file(path)
else:
# Path is neither a file or a dir?
continue


def _clear_minifyed(path):
u'''Remove the minified version of the file'''
path_only, extension = os.path.splitext(path)

if extension not in (u'.css', u'.js'):
# This is not a js or css file.
return

if path_only.endswith(u'.min'):
click.echo(u'removing {}'.format(path))
os.remove(path)


def _minify_file(path):
u'''Create the minified version of the given file.
If the file is not a .js or .css file (e.g. it's a .min.js or .min.css
file, or it's some other type of file entirely) it will not be
minifed.
:param path: The path to the .js or .css file to minify
'''
import ckan.lib.fanstatic_resources as fanstatic_resources
path_only, extension = os.path.splitext(path)

if path_only.endswith(u'.min'):
# This is already a minified file.
return

if extension not in (u'.css', u'.js'):
# This is not a js or css file.
return

path_min = fanstatic_resources.min_path(path)

source = open(path, u'r').read()
f = open(path_min, u'w')
if path.endswith(u'.css'):
f.write(rcssmin.cssmin(source))
elif path.endswith(u'.js'):
f.write(rjsmin.jsmin(source))
f.close()
click.echo(u"Minified file '{}'".format(path))
23 changes: 23 additions & 0 deletions ckan/cli/notify.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# encoding: utf-8

import click
from ckan.model import Session, Package, DomainObjectOperation
from ckan.model.modification import DomainObjectModificationExtension


@click.group(
name=u'notify',
short_help=u'Send out modification notifications.'
)
def notify():
pass


@notify.command(
name=u'replay',
short_help=u'Send out modification signals.'
)
def replay():
dome = DomainObjectModificationExtension()
for package in Session.query(Package):
dome.notify(package, DomainObjectOperation.changed)

0 comments on commit 9b8b2a3

Please sign in to comment.