Government of Canada CKAN Extension - Extension à CKAN du Gouvernement du Canada
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bin ab harvest deletions Sep 19, 2018
ckanext
conf/solr atipd: wildcard for indexing doubles Jun 13, 2018
links more requirements install fixes Jul 10, 2013
samples obd sample data: use statcan + tags May 30, 2017
.gitignore Merge branch 'master' into wet4-scheming Dec 22, 2015
.travis.yml travis: use old environment Jan 24, 2018
COPYING crown copyright Oct 3, 2018
COPYING.fr crown copyright Oct 3, 2018
MANIFEST.in metadata_schema ckanext internal access module Dec 19, 2012
README.rst readme: update ckan branch name Sep 19, 2018
requirements.txt
setup.py first cut of obd schema May 11, 2017
test-core.ini use chained_action Mar 28, 2017
transitional_orgs.jsonl New orgs and field for IPPD ATI Jul 16, 2018

README.rst

ckanext-canada

Government of Canada CKAN Extension - Extension à CKAN du Gouvernement du Canada

Features:

  • Forms and Validation for GoC Metadata Schema

Installation:

From a clean database you must run:

ckanapi load organizations -I transitional_orgs.jsonl

Once to create the organizations this extension requires before loading any data.

Plugins in this extension

canada_forms
dataset forms for Open Canada metadata schema
canada_public
base and public facing Open Canada templates (requires canada_forms and wet_theme from ckanext-wet-boew )
canada_internal
templates for internal site and registration (requires canada_forms and canada_public)
canada_package
package processing between CKAN and Solr
canada_obd
Open By Default site plugin

Requirements

Related projects, repositories, branches and CKAN plugins
Project Github group/repo Branch Plugins
CKAN open-data/ckan canada-v2.6 N/A
data.gc.ca extension open-data/ckanext-canada master
  • canada_forms
  • canada_internal
  • canada_public
  • canada_package
WET-BOEW theme open-data/ckanext-wet-boew master
  • wet_theme
Scheming extension open-data/ckanext-scheming master scheming_datasets
Fluent extension open-data/ckanext-fluent master N/A
ckanapi ckan/ckanapi master N/A
ckanext-googleanalytics ofkn/ckanext-googleanalytics master googleanalytics
Recombinant tables extension open-data/ckanext-recombinant master
  • recombinant

OD Configuration: development.ini or production.ini

The CKAN ini file needs the following settings for the registry server:

ckan.plugins = dcat dcat_json_interface googleanalytics canada_forms canada_internal
     canada_public canada_package canada_activity wet_boew_theme_gc_intranet datastore recombinant
     scheming_datasets fluent extendedactivity

For the public server use only:

ckan.plugins = dcat dcat_json_interface googleanalytics canada_forms
     canada_public canada_package wet_boew_gcweb
     scheming_datasets fluent

canada.portal_url = http://myserver.com

Both servers need:

licenses_group_url = file://<path to this extension>/ckanext/canada/public/static/licenses.json

ckan.i18n_directory = <path to this extension>/build

ckan.auth.create_dataset_if_not_in_organization = false

ckan.activity_streams_email_notifications = false

ckan.datasets_per_page = 10

googleanalytics.id = UA-1010101-1 (your analytics account id)
googleanalytics.account = Account name (i.e. data.gov.uk, see top level item at https://www.google.com/analytics)
loop11.key = bca42f834d21755c050b437dcc881bcde963b08e

OBD Configuration

We use a different list of plugins for Open By Default:

ckan.plugins = dcat dcat_json_interface googleanalytics canada_forms
     canada_obd canada_package wet_boew_gcweb scheming_datasets
     fluent cloudstorage

ckan.extra_resource_fields = language
Update OBD documents (example)::
touch /tmp/marker import_xml2obd.py pull ./production.ini ./obd-repo > /tmp/pull.log find ./obd-repo -type f -newer /tmp/marker > ./new.txt import_xml2obd.py ./obd-repo http://obd-dev.canadacentral.cloudapp.azure.com/ckan ./new.txt > ./data/obd-20170704.jsonl import_xml2obd.py upload http://obd-dev.canadacentral.cloudapp.azure.com/ckan <site API key> ./data/obd-20170704.jsonl ./obd-repo
Delete OBD documents (only change the dataset state):
import_xml2obd.py delete ./to_delete.csv ./obd-repo http://obd-dev.canadacentral.cloudapp.azure.com/ckan <site API key>
Verify OBD documents:

# check resource exists import_xml2obd.py <site_url> azure_user azure_key azure_container

# check duplicates import_xml2obd.py de-dup <site_url>

Configuration: Solr

This extension uses a custom Solr schema based on the ckan 2.6 schema. You can find the schema in the root directory of the project. Overwrite the default CKAN Solr schema with this one in order to enable search faceting over custom metadata fields.

You will need to rebuild your search index using:

paster --plugin ckan search-index rebuild

Compiling the updated French localization strings

Each time you install or update this extension you need to install the updated translations by running:

bin/build-combined-ckan-mo.sh

This script overwrites the ckan French translations by combining it with ours.