Skip to content

Install and use Solr 6.5 with CKAN

Bojan Zivkovic edited this page Feb 11, 2020 · 4 revisions

Overview

In this tutorial we are going to install Solr 6.5 on our server (no compilation required!) and set up a core for CKAN.

Solr 6.5 comes with a nice script for install the package in the system, it detect the OS and eventually creates the solr user for the daemon.

Java version

Check you have Java 8 (also known as Java 1.8). Earlier versions are incompatible with Solr 6. Java 9 and later are not recommended by SOLR and cause errors.

java -version

If you are on Ubuntu 18.04 it comes with Java 8 and Java 11, defaulting to 11. So get rid of 11:

sudo apt-get remove openjdk-11-jre-headless
java -version
    openjdk version "1.8.0_242"
    OpenJDK Runtime Environment (build 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08)
    OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)

Download and Install Solr

Go to the Apache Solr website and download Solr, in this tutorial we are going to use the .tgz format, but it works with .zip as well.

Download the package from the web and put it somewhere in your file-system, we are going to use the installer soon

wget https://archive.apache.org/dist/lucene/solr/6.5.1/solr-6.5.1.tgz

when the download is finished, unzip the package and go to the bin directory

unzip solr-6.5.1.tgz
cd solr-6.5.1/bin

as root, we need to create the install directory and execute the file install_solr_service.sh. This script accepts arguments if we need to install the daemon to listen ad different port or with another user.

Usage: install_solr_service.sh path_to_solr_distribution_archive OPTIONS

  The first argument to the script must be a path to a Solr distribution archive, such as solr-6.5.0.tgz
    (only .tgz or .zip are supported formats for the archive)

  Supported OPTIONS include:

    -d     Directory for live / writable Solr files, such as logs, pid files, and index data; defaults to /var/solr

    -i     Directory to extract the Solr installation archive; defaults to /opt/
             The specified path must exist prior to using this script.

    -p     Port Solr should bind to; default is 8983

    -s     Service name; defaults to solr

    -u     User to own the Solr files and run the Solr process as; defaults to solr
             This script will create the specified user account if it does not exist.

 NOTE: Must be run as the root user

If we want to install everything to /opt/solr

sudo install_solr_service.sh ../../solr-6.5.1.tgz

note that we use the .tgz file as argument to the script, change the path if you need.

Now the script installs the binaries under /opt/solr, it puts the init script under /etc/init.d/solr and eventually create the solr user.

Check it started ok:

sudo service solr status
    o solr.service - LSB: Controls Apache Solr as a Service
       Loaded: loaded (/etc/init.d/solr; generated)
       Active: active (exited) since Fri 2020-02-07 11:36:20 UTC; 20s ago
         Docs: man:systemd-sysv-generator(8)
      Process: 8964 ExecStop=/etc/init.d/solr stop (code=exited, status=1/FAILURE)
      Process: 9035 ExecStart=/etc/init.d/solr start (code=exited, status=0/SUCCESS)

Create and configure the core

Switch to the solr user and go to the bin directory

sudo su solr
cd /opt/solr/bin

now create the ckan core

./solr create -c ckan

If this command fails because "Failed to determine the port of a local Solr instance, cannot create ckan!" then it's because the main 'solr' service has errors. Check logs for errors: /var/solr/logs/solr.log and /var/solr/logs/solr-8983-console.log

When successful it will have created all the configuration files and directories. At this point, we can see the core listed in our solr admin http://localhost:8983/solr and we can proceed to edit the configuration files

cd /var/solr/data/ckan/conf

open solrconfig.xml and insert following line into the root <config> element:

<schemaFactory class="ClassicIndexSchemaFactory"/>

Delete this element:

<initParams path="/update/**">
  <lst name="defaults">
    <str name="update.chain">add-unknown-fields-to-the-schema</str>
  </lst>
</initParams>

and also delete this element:

<processor class="solr.AddSchemaFieldsUpdateProcessorFactory">
  <str name="defaultFieldType">strings</str>
  <lst name="typeMapping">
    <str name="valueClass">java.lang.Boolean</str>
    <str name="fieldType">booleans</str>
  </lst>
  <lst name="typeMapping">
    <str name="valueClass">java.util.Date</str>
    <str name="fieldType">tdates</str>
  </lst>
  <lst name="typeMapping">
    <str name="valueClass">java.lang.Long</str>
    <str name="valueClass">java.lang.Integer</str>
    <str name="fieldType">tlongs</str>
  </lst>
  <lst name="typeMapping">
    <str name="valueClass">java.lang.Number</str>
    <str name="fieldType">tdoubles</str>
  </lst>
</processor>

Next, remove the managed-schema file

rm managed-schema

and copy or symlink the schema.xml from CKAN:

cp /somewhere/over/the/rainbow/ckan/conf/solr/schema.xml .

or

ln -s /usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml schema.xml

Finally, restart solr

/etc/init.d/solr restart

or

sudo service solr restart

Check there are no errors when you browse: http://localhost:8983/solr/#/ckan

Config ini file

Make sure your solr_url in development.ini or production.ini is is pointing to:

solr_url = http://127.0.0.1:8983/solr/ckan
Clone this wiki locally
You can’t perform that action at this time.