The purpose of this repo is to install a version of CKAN for development, an assumption is made that you are working on the catalogue but with some tweaks it could work for any version of ckan. (Once terraform 12 comes out the repo could(and should) be changed to a list / for loop)
This procedure does not make any assumptions about your OS, and therefore you must have installed
- pip
- python
- openssl (Mac - Note: You may also have to copy files over from Cellar [brew install] to /usr/local/lib)
- terraform (https://www.terraform.io/downloads.html) (v 0.11.7)
- wget
- git cli is logged in with automatic access to new repos on github/gogs
- libmagic c library is installed (
brew install libmagicon Mac)
- Copy terraform.tfvars.example to terraform.tfvars.
- (OPTIONAL) add a db dump file to ./db called ckan.dump and it will be imported automatically
- Modify the contents to match your needs
- Run
terraform initto initialize all the modules from remote - (OPTIONAL) Run
terraform planto see what the command intends to do - Run
terraform applyand you'll see the same output as step 4 above respond yes and let terraform do it's job
- Install path is relative to the root of this directory (the directory of this README file)
Some files are intentionally omitted from this install process as they are confidential. So follow the steps below
- source ${path.root}/${local.installPath}/venv/bin/activate;
- pip install ckanext-pdfview==0.0.5
- pip install sqlalchemy==0.9
- pip install genshi==0.7.1
- pip install flask_debugtoolbar==0.10.1
- Provide the bcdc_licenses.json file in {installPath}/ckanext-bcgov/ckanext/bcgov/scripts/data/bcdc_licenses.json
- Then run solr indexing, (See #Solr)
- If you do not want to be on the master branches for a particular Repo you must check it on and re-run the pip/python install.
- Activate the venv
source {installPath}/venv/bin/activate(Note you should now be able to run paster commands - cd into
{installPath}/ckanext-bcgov(or whatever extension you're working on) - Run with
paster serve --reload --reload-interval=2 ../conf/ckan.ini
There is a problem where if you run terraform apply as normal the venv will not be set up perfectly. This is due to it installing things into that environment in parallel. There is at present 3 work arounds. Listed in order of there effectiveness (higher numbers are better)
- Install the plugins manually (python setup.py develop) for every repo that it fetches
- Run
terraform apply -parallelism=1this limits parallelism and solves the problem but takes longer to install the non repos - Run
terraform applyafter it finishes nuke the .*.tf files in {installPath} and then runterraform apply -parallelism=1
I haven't found a good solution to this via code, that doesn't mean that one doesn't exist
To index the solr database run paster --plugin=ckan search-index rebuild -c {installPath}/conf/ckan.ini
To create the datastore you need to run an additional command.
- Activate the venv
source {installPath}/venv/bin/activate paster --plugin=ckan datastore set-permissions -c conf/ckan.ini | docker exec -i ckan_postgres-.{installPathWithDots} psql -U ckan
To run the datapusher
- Activate the venv
source {installPath}/venv/bin/activate - cd to {installPath}/datapusher
- python datapusher/main.py deployment/datapusher_settings.py
- Activate the venv
source {installPath}/venv/bin/activate(Note you should now be able to run paster commands paster --plugin=ckan user add <username> email=<address> -c {installPath}/conf/ckan.ini- (Optional if you want admin privileges)
paster --plugin=ckan sysadmin add <username> email=<address> -c {installPath}/conf/ckan.ini
Sometimes the terraform execution will fail for one of the repos. When it does this you will get an error along the lines of
1 error(s) occurred:
* module.dev.module.ckanextGaReportRepo.local_file.repo: Error running command...
When this happens delete the file in the install path that corresponding file.
In the case above .ckanext-ga-report.tf and retry the terraform apply (note it will only do work thats required)
If the postgres container starts and shutdown immediately the permissions on your db folder are probably incorrect. Make sure that they are global read/writable
Copyright 2019, Province of British Columbia.