Permalink
Switch branches/tags
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
381 lines (249 sloc) 12.1 KB

Installing CKAN from source

This section describes how to install CKAN from source. Although :doc:`install-from-package` is simpler, it requires Ubuntu 18.04 64-bit, Ubuntu 16.04 64-bit, or Ubuntu 14.04 64-bit. Installing CKAN from source works with other versions of Ubuntu and with other operating systems (e.g. RedHat, Fedora, CentOS, OS X). If you install CKAN from source on your own operating system, please share your experiences on our How to Install CKAN wiki page.

From source is also the right installation method for developers who want to work on CKAN.

1. Install the required packages

If you're using a Debian-based operating system (such as Ubuntu) install the required packages with this command for Ubuntu 18.04 and Ubuntu 16.04:

sudo apt-get install python-dev postgresql libpq-dev python-pip python-virtualenv git-core solr-jetty openjdk-8-jdk redis-server

or for Ubuntu 14.04:

sudo apt-get install python-dev postgresql libpq-dev python-pip python-virtualenv git-core solr-jetty openjdk-6-jdk redis-server

If you're not using a Debian-based operating system, find the best way to install the following packages on your operating system (see our How to Install CKAN wiki page for help):

Package Description
Python The Python programming language, v2.7
|postgres| The PostgreSQL database system, v9.3 or newer
libpq The C programmer's interface to PostgreSQL
pip A tool for installing and managing Python packages
virtualenv The virtual Python environment builder
Git A distributed version control system
Apache Solr A search platform
Jetty An HTTP server (used for Solr).
OpenJDK JDK The Java Development Kit (used by Jetty)
Redis An in-memory data structure store

2. Install CKAN into a Python virtual environment

Tip

If you're installing CKAN for development and want it to be installed in your home directory, you can symlink the directories used in this documentation to your home directory. This way, you can copy-paste the example commands from this documentation without having to modify them, and still have CKAN installed in your home directory:

mkdir -p ~/ckan/lib
sudo ln -s ~/ckan/lib |virtualenv_parent_dir|
mkdir -p ~/ckan/etc
sudo ln -s ~/ckan/etc |config_parent_dir|
  1. Create a Python virtual environment (virtualenv) to install CKAN into, and activate it:

    sudo mkdir -p |virtualenv|
    sudo chown `whoami` |virtualenv|
    virtualenv --no-site-packages |virtualenv|
    |activate|
    

Note

If your system uses Python3 by default (e.g. Ubuntu 18.04) make sure to create the virtualenv using the Python2.7 executable with the --python option:

sudo mkdir -p |virtualenv|
sudo chown `whoami` |virtualenv|
virtualenv --python=/usr/bin/python2.7 --no-site-packages |virtualenv|
|activate|

Important

The final command above activates your virtualenv. The virtualenv has to remain active for the rest of the installation and deployment process, or commands will fail. You can tell when the virtualenv is active because its name appears in front of your shell prompt, something like this:

(default) $ _

For example, if you logout and login again, or if you close your terminal window and open it again, your virtualenv will no longer be activated. You can always reactivate the virtualenv with this command:

|activate|
  1. Install the recommended setuptools version:

    pip install setuptools==|min_setuptools_version|
    
  2. Install the CKAN source code into your virtualenv.

    To install the latest stable release of CKAN (CKAN |latest_release_version|), run:

    pip install -e 'git+|git_url|@|latest_release_tag|#egg=ckan'
    

    If you're installing CKAN for development, you may want to install the latest development version (the most recent commit on the master branch of the CKAN git repository). In that case, run this command instead:

    pip install -e 'git+|git_url|#egg=ckan'
    

    Warning

    The development version may contain bugs and should not be used for production websites! Only install this version if you're doing CKAN development.

  3. Install the Python modules that CKAN requires into your virtualenv:

    pip install -r |virtualenv|/src/ckan/requirements.txt
    
  4. Deactivate and reactivate your virtualenv, to make sure you're using the virtualenv's copies of commands like paster rather than any system-wide installed copies:

    deactivate
    |activate|
    

3. Setup a PostgreSQL database

4. Create a CKAN config file

Create a directory to contain the site's config files:

sudo mkdir -p |config_dir|
sudo chown -R `whoami` |config_parent_dir|/
sudo chown -R `whoami` ~/ckan/etc

Create the CKAN config file:

paster make-config ckan |development.ini|

Edit the development.ini file in a text editor, changing the following options:

sqlalchemy.url

This should refer to the database we created in 3. Setup a PostgreSQL database above:

sqlalchemy.url = postgresql://|database_user|:pass@localhost/|database|

Replace pass with the password that you created in 3. Setup a PostgreSQL database above.

Tip

If you're using a remote host with password authentication rather than SSL authentication, use:

sqlalchemy.url = postgresql://|database_user|:pass@<remotehost>/|database|?sslmode=disable
site_id

Each CKAN site should have a unique site_id, for example:

ckan.site_id = default
site_url

Provide the site's URL (used when putting links to the site into the FileStore, notification emails etc). For example:

ckan.site_url = http://demo.ckan.org

Do not add a trailing slash to the URL.

5. Setup Solr

6. Link to who.ini

who.ini (the Repoze.who configuration file) needs to be accessible in the same directory as your CKAN config file, so create a symlink to it:

ln -s |virtualenv|/src/ckan/who.ini |config_dir|/who.ini

7. Create database tables

Now that you have a configuration file that has the correct settings for your database, you can :ref:`create the database tables <db init>`:

cd |virtualenv|/src/ckan
paster db init -c |development.ini|

You should see Initialising DB: SUCCESS.

Tip

If the command prompts for a password it is likely you haven't set up the sqlalchemy.url option in your CKAN configuration file properly. See 4. Create a CKAN config file.

8. Set up the DataStore

Note

Setting up the DataStore is optional. However, if you do skip this step, the :doc:`DataStore features </maintaining/datastore>` will not be available and the DataStore tests will fail.

Follow the instructions in :doc:`/maintaining/datastore` to create the required databases and users, set the right permissions and set the appropriate values in your CKAN config file.

9. You're done!

You can now use the Paste development server to serve CKAN from the command-line. This is a simple and lightweight way to serve CKAN that is useful for development and testing:

cd |virtualenv|/src/ckan
paster serve |development.ini|

Open http://127.0.0.1:5000/ in a web browser, and you should see the CKAN front page.

Now that you've installed CKAN, you should:

Note

The default authorization settings on a new install are deliberately restrictive. Regular users won't be able to create datasets or organizations. You should check the :doc:`/maintaining/authorization` documentation, configure CKAN accordingly and grant other users the relevant permissions using the :ref:`sysadmin account <create-admin-user>`.

Source install troubleshooting

Solr setup troubleshooting

Solr requests and errors are logged in the web server log files.

  • For Jetty servers, the log files are:

    /var/log/jetty/<date>.stderrout.log
    
  • For Tomcat servers, they're:

    /var/log/tomcat6/catalina.<date>.log
    

Unable to find a javac compiler

If when running Solr it says:

Unable to find a javac compiler; com.sun.tools.javac.Main is not on the classpath. Perhaps JAVA_HOME does not point to the JDK.

See the note in :ref:`setting up solr` about JAVA_HOME. Alternatively you may not have installed the JDK. Check by seeing if javac is installed:

which javac

If javac isn't installed, do:

sudo apt-get install openjdk-6-jdk

and then restart Solr:

For Ubuntu 16.04:

sudo service jetty8 restart

or for Ubuntu 14.04:

sudo service jetty restart

AttributeError: 'module' object has no attribute 'css/main.debug.css'

This error is likely to show up when debug is set to True. To fix this error, install frontend dependencies. See :doc:`/contributing/frontend/index`.

After installing the dependencies, run bin/less and then start paster server again.

If you do not want to compile CSS, you can also copy the main.css to main.debug.css to get CKAN running:

cp /usr/lib/ckan/default/src/ckan/ckan/public/base/css/main.css \
/usr/lib/ckan/default/src/ckan/ckan/public/base/css/main.debug.css

JSP support not configured

This is seen occasionally with Jetty and Ubuntu 14.04. It requires a solr-jetty fix:

cd /tmp
wget https://launchpad.net/~vshn/+archive/ubuntu/solr/+files/solr-jetty-jsp-fix_1.0.2_all.deb
sudo dpkg -i solr-jetty-jsp-fix_1.0.2_all.deb
sudo service jetty restart

ImportError: No module named 'flask_debugtoolbar'

This may show up if you are creating the database tables and you have enabled debug mode in the config file. Simply install the development requirements:

pip install -r /usr/lib/ckan/default/src/ckan/dev-requirements.txt