From 9674f862ae6147d3778a05bc8d8168954e0e0e60 Mon Sep 17 00:00:00 2001 From: Sean Hammond Date: Thu, 21 Nov 2013 16:43:59 +0100 Subject: [PATCH] Reorganize Solr setup docs Sphinx doesn't support "inner pages" like the Solr setup page was trying to be, it breaks Sphinx's navigation. Move the single-core Solr setup docs into the source install docs, inline. And an appendices chapter to the docs, and move the multi-core Solr docs into it. Also add new docs for setting up a second Solr core when using multi-core. Also tidy up some Sphinx heading styles. --- doc/appendices/index.rst | 7 + doc/appendices/solr-multicore.rst | 218 ++++++++++++++++++++++++ doc/conf.py | 1 + doc/index.rst | 1 + doc/install-from-package.rst | 11 +- doc/install-from-source.rst | 157 +++++++++++++++--- doc/solr-setup.rst | 266 ------------------------------ 7 files changed, 371 insertions(+), 290 deletions(-) create mode 100644 doc/appendices/index.rst create mode 100644 doc/appendices/solr-multicore.rst delete mode 100644 doc/solr-setup.rst diff --git a/doc/appendices/index.rst b/doc/appendices/index.rst new file mode 100644 index 00000000000..371b7eb61a9 --- /dev/null +++ b/doc/appendices/index.rst @@ -0,0 +1,7 @@ +========== +Appendices +========== + +.. toctree:: + + solr-multicore diff --git a/doc/appendices/solr-multicore.rst b/doc/appendices/solr-multicore.rst new file mode 100644 index 00000000000..92ebcb6fb28 --- /dev/null +++ b/doc/appendices/solr-multicore.rst @@ -0,0 +1,218 @@ +.. _multicore solr setup: + +==================== +Multicore Solr setup +==================== + +Solr can be set up to have multiple configurations and search indexes on the +same machine. Each configuration is called a Solr *core*. Having multiple cores +is useful when you want different applications or different versions of CKAN to +share the same Solr instance, each application can have its own Solr core so +each can use a different ``schema.xml`` file. This is necessary, for example, +if you want to CKAN instances to share the same |solr| server and those two +instances are running different versions of CKAN that require differemt +``schema.xml`` files, or if the two instances have different |solr| schema +customizations. + +Each |solr| core in a multicore setup will have a different URL, for example:: + + http://localhost:8983/solr/ckan_default + http://localhost:8983/solr/some_other_site + +This section will show you how to create a multicore |solr| setup and create +your first core. If you already have a multicore setup and now you've setup a +second CKAN instance on the same machine and want to create a second |solr| +core for it, see :ref:`creating another solr core`. + +#. Create the file ``/usr/share/solr/solr.xml``, with the following contents:: + + + + + + + + + + This file lists the different Solr cores, in this example we have just a + single core called ``ckan_default``. + +#. Create the data directory for your Solr core, run this command in a + terminal:: + + sudo -u jetty mkdir /var/lib/solr/data/ckan_default + + This is the directory where Solr will store the search index files for + our core. + +#. Create the directory ``/etc/solr/ckan_default``, and move the + ``/etc/solr/conf`` directory into it:: + + sudo mkdir /etc/solr/ckan_default + sudo mv /etc/solr/conf /etc/solr/ckan_default/ + + This directory holds the configuration files for your Solr core. + +#. Replace the ``/etc/solr/ckan_default/schema.xml`` file with a symlink to + CKAN's ``schema.xml`` file:: + + sudo mv /etc/solr/ckan_default/conf/schema.xml /etc/solr/ckan_default/conf/schema.xml.bak + sudo ln -s /usr/lib/ckan/default/src/ckan/ckan/config/solr/schema-2.0.xml /etc/solr/ckan_default/conf/schema.xml + +#. Edit ``/etc/solr/ckan_default/conf/solrconfig.xml`` and change the + ```` tag to this:: + + ${dataDir} + + This configures our ``ckan_default`` core to use the data directory you + specified for it in ``solr.xml``. + +#. Create the directory ``/usr/share/solr/ckan_default`` and put a symlink + to the ``conf`` directory in it:: + + sudo mkdir /usr/share/solr/ckan_default + sudo ln -s /etc/solr/ckan_default/conf /usr/share/solr/ckan_default/conf + +#. Restart Solr: + + .. parsed-literal:: + + |restart_solr| + + You should now see your newly created ``ckan_default`` core if you open + http://localhost:8983/solr/ckan_default/admin/ in your web browser. + You can click on the *schema* link on this page to check that the core is + using the right schema (you should see ```` + near the top of the ``schema.xml`` file). The http://localhost:8983/solr/ + page will list all of your configured Solr cores. + +#. Finally, change the ``solr_url`` setting in your |development.ini| or + |production.ini| file to point to your new Solr core, for example:: + + solr_url = http://127.0.0.1:8983/solr/ckan_default + +If you have trouble when setting up Solr, see +:ref:`multicore solr troubleshooting` below. + + +.. _creating another solr core: + +-------------------------- +Creating another Solr core +-------------------------- + +.. The name of the second CKAN instance that we're going to setup a second core + for: +.. |ckan| replace:: my-second-ckan-instance + +.. The name of the second CKAN core we're going to set up: +.. |core| replace:: my-second-solr-core + +In this example we'll assume that: + +#. You've followed the instructions in :ref:`multicore solr setup` to create + a multicore setup and create your first core for your first CKAN instance. + +#. You've installed a second instance of CKAN in a second virtual environment + at /usr/lib/ckan/|ckan|, and now want to setup a second Solr core for it. + +You can ofcourse follow these instructions again to setup further Solr cores. + +#. Add the core to ``/usr/share/solr/solr.xml``. This file should now list + two cores. For example: + + .. parsed-literal:: + + + + + + + + + + + + +#. Create the data directory for your new core: + + .. parsed-literal:: + + sudo -u jetty mkdir /var/lib/solr/data/|core| + +#. Create the configuration directory for your new core, and copy the config + from your first core into it: + + .. parsed-literal:: + + sudo mkdir /etc/solr/|core| + sudo cp -R /etc/solr/ckan_default/conf /etc/solr/|core|/ + +#. Replace the /etc/solr/|core|/schema.xml file with a symlink to the + ``schema.xml`` file from your second CKAN instance: + + .. parsed-literal:: + + sudo rm /etc/solr/|core|/conf/schema.xml + sudo ln -s /usr/lib/ckan/|ckan|/src/ckan/ckan/config/solr/schema-2.0.xml /etc/solr/|core|/conf/schema.xml + +#. Create the /usr/share/solr/|core| directory and put a symlink to the + ``conf`` directory in it: + + .. parsed-literal:: + + sudo mkdir /usr/share/solr/|core| + sudo ln -s /etc/solr/|core|/conf /usr/share/solr/|core|/conf + +#. Restart |solr|: + + .. parsed-literal:: + + |restart_solr| + + You should now see both your Solr cores when you open + http://localhost:8983/solr/ in your web browser. + +#. Finally, change the :ref:`solr_url` setting in your + /etc/ckan/|ckan|/development.ini or /etc/ckan/|ckan|/production.ini file to + point to your new Solr core: + + .. parsed-literal:: + + solr_url = http://127.0.0.1:8983/solr/|core| + +If you have trouble when setting up Solr, see +:ref:`multicore solr troubleshooting`. + + +.. _multicore solr troubleshooting: + +-------------------------------------- +Multicore |solr| setup troubleshooting +-------------------------------------- + +.. seealso:: + + :ref:`Troubleshooting for single-core Solr setups ` + Most of these tips also apply to multi-core setups. + +No cores shown on |solr| index page +=================================== + +If no cores are shown when you visit the |solr| index page, and the admin +interface returns a 404 error, check the web server error log +(``/var/log/jetty/.stderrout.log`` if you're using Jetty, or +``/var/log/tomcat6/catalina..log`` for Tomcat). If you can find an error +similar to this one:: + + WARNING: [iatiregistry.org] Solr index directory '/usr/share/solr/iatiregistry.org/data/index' doesn't exist. Creating new index... + 07-Dec-2011 18:06:33 org.apache.solr.common.SolrException log + SEVERE: java.lang.RuntimeException: Cannot create directory: /usr/share/solr/iatiregistry.org/data/index + [...] + +Then ``dataDir`` is not properly configured. With our setup the data directory +should be under ``/var/lib/solr/data``. Make sure that you defined the correct +``dataDir`` in the ``solr.xml`` file and that in the ``solrconfig.xml`` file +you have the following configuration option:: + + ${dataDir} diff --git a/doc/conf.py b/doc/conf.py index 699b2c1718e..b439de0c7e7 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -49,6 +49,7 @@ .. |storage_dir| replace:: |storage_parent_dir|/default .. |reload_apache| replace:: sudo service apache2 reload .. |restart_apache| replace:: sudo service apache2 restart +.. |restart_solr| replace:: sudo service jetty restart .. |solr| replace:: Solr .. |restructuredtext| replace:: reStructuredText .. |nginx| replace:: Nginx diff --git a/doc/index.rst b/doc/index.rst index 42f345b1e27..1695d407182 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -51,3 +51,4 @@ advanced documentation last: test configuration changelog + appendices/index diff --git a/doc/install-from-package.rst b/doc/install-from-package.rst index 0c1a3a27eb5..8447b3bf1af 100644 --- a/doc/install-from-package.rst +++ b/doc/install-from-package.rst @@ -1,4 +1,5 @@ -Installing CKAN from Package +============================ +Installing CKAN from package ============================ This section describes how to install CKAN from package. This is the quickest @@ -8,7 +9,8 @@ development, you should follow :doc:`install-from-source` instead. .. _run-package-installer: -1. Install the CKAN Package +--------------------------- +1. Install the CKAN package --------------------------- On your Ubuntu 12.04 system, open a terminal and run these commands to install @@ -50,6 +52,7 @@ CKAN: sudo service apache2 restart +------------------------------ 2. Install PostgreSQL and Solr ------------------------------ @@ -67,8 +70,7 @@ CKAN: * Not starting jetty - edit /etc/default/jetty and change NO_START to be 0 (or comment it out). -#. Follow the instructions in :ref:`solr-single` or :ref:`solr-multi-core` to - setup |solr|. +#. Follow the instructions in :ref:`setting up solr` to setup |solr|. #. Follow the instructions in :ref:`postgres-setup` to setup |postgres|, then edit the :ref:`sqlalchemy.url` option in your |production.ini| file and @@ -84,6 +86,7 @@ CKAN: #. Also optionally, you can enable file uploads by following the instructions in :doc:`filestore`. +--------------- 3. You're done! --------------- diff --git a/doc/install-from-source.rst b/doc/install-from-source.rst index e94a818721a..e66d49515a3 100644 --- a/doc/install-from-source.rst +++ b/doc/install-from-source.rst @@ -1,4 +1,5 @@ -Installing CKAN from Source +=========================== +Installing CKAN from source =========================== This section describes how to install CKAN from source. Although @@ -12,8 +13,9 @@ wiki page. From source is also the right installation method for developers who want to work on CKAN. +-------------------------------- 1. Install the required packages -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +-------------------------------- If you're using a Debian-based operating system (such as Ubuntu) install the required packages with this command:: @@ -34,16 +36,17 @@ libpq `The C programmer's interface to PostgreSQL `_ virtualenv `The virtual Python environment builder `_ Git `A distributed version control system `_ -Apache Solr `A search platform `_ -Jetty `An HTTP server `_ (used for Solr) +Apache Solr `A search platform `_ +Jetty `An HTTP server `_ (used for Solr). OpenJDK 6 JDK `The Java Development Kit `_ ===================== =============================================== .. _install-ckan-in-virtualenv: +------------------------------------------------- 2. Install CKAN into a Python virtual environment -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------------------------------- .. tip:: @@ -122,8 +125,9 @@ d. Deactivate and reactivate your virtualenv, to make sure you're using the .. _postgres-setup: +------------------------------ 3. Setup a PostgreSQL database -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------------ List existing databases:: @@ -149,9 +153,9 @@ database user you just created: sudo -u postgres createdb -O |database_user| |database| -E utf-8 - +---------------------------- 4. Create a CKAN config file -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +---------------------------- Create a directory to contain the site's config files: @@ -196,24 +200,91 @@ site_id ckan.site_id = default +.. _setting up solr: + +------------- 5. Setup Solr -~~~~~~~~~~~~~ +------------- -Follow the instructions in :ref:`solr-single` or :ref:`solr-multi-core` to -setup Solr, then change the ``solr_url`` option in your CKAN config file to -point to your Solr server, for example:: +CKAN uses Solr_ as its search platform, and uses a customized Solr schema file +that takes into account CKAN's specific search needs. Now that we have CKAN +installed, we need to install and configure Solr. - solr_url=http://127.0.0.1:8983/solr +.. _Solr: http://lucene.apache.org/solr/ + +.. note:: + + These instructions explain how to setup |solr| with a single core. + If you want multiple applications, or multiple instances of CKAN, to share + the same |solr| server then you probably want a multi-core |solr| setup + instead. See :doc:`/appendices/solr-multicore`. + +.. note:: + + These instructions explain how to deploy Solr using the Jetty web + server, but CKAN doesn't require Jetty - you can deploy Solr to another web + server, such as Tomcat, if that's convenient on your operating system. + +#. Edit the Jetty configuration file (``/etc/default/jetty``) and change the + following variables:: + + NO_START=0 # (line 4) + JETTY_HOST=127.0.0.1 # (line 15) + JETTY_PORT=8983 # (line 18) + + Start the Jetty server:: + + sudo service jetty start + + You should now see a welcome page from Solr if you open + http://localhost:8983/solr/ in your web browser (replace localhost with + your server address if needed). + + .. note:: + + If you get the message ``Could not start Jetty servlet engine because no + Java Development Kit (JDK) was found.`` then you will have to edit the + ``JAVA_HOME`` setting in ``/etc/default/jetty`` to point to your machine's + JDK install location. For example:: + + JAVA_HOME=/usr/lib/jvm/java-6-openjdk-amd64/ + + or:: + + JAVA_HOME=/usr/lib/jvm/java-6-openjdk-i386/ + +#. Replace the default ``schema.xml`` file with a symlink to your CKAN schema + file. + There are multiple versions of CKAN's schema file in + |virtualenv|/src/ckan/ckan/config/solr/. Use the newest schema file whose version + number is the same as or older than the version of CKAN you're using. For + example, if you're using CKAN 2.1.1 then you would use the ``schema-2.0.xml`` + file (because there is no ``schema-2.1.xml`` file): + + .. parsed-literal:: + + sudo mv /etc/solr/conf/schema.xml /etc/solr/conf/schema.xml.bak + sudo ln -s |virtualenv|/src/ckan/ckan/config/solr/schema-2.0.xml /etc/solr/conf/schema.xml + + Now restart Solr: + + .. parsed-literal:: -.. toctree:: - :hidden: + |restart_solr| - solr-setup + and check that Solr is running by opening http://localhost:8983/solr/. + + +#. Finally, change the :ref:`solr_url` setting in your CKAN config file to + point to your Solr server, for example:: + + solr_url=http://127.0.0.1:8983/solr .. _postgres-init: +------------------------- 6. Create database tables -~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------- Now that you have a configuration file that has the correct settings for your database, you can create the database tables: @@ -231,8 +302,9 @@ You should see ``Initialising DB: SUCCESS``. ``sqlalchemy.url`` option in your CKAN configuration file properly. See `4. Create a CKAN config file`_. +----------------------- 7. Set up the DataStore -~~~~~~~~~~~~~~~~~~~~~~~ +----------------------- .. note :: Setting up the DataStore is optional. However, if you do skip this step, @@ -243,8 +315,9 @@ Follow the instructions in :doc:`datastore` to create the required databases and users, set the right permissions and set the appropriate values in your CKAN config file. +---------------------- 8. Link to ``who.ini`` -~~~~~~~~~~~~~~~~~~~~~~ +---------------------- ``who.ini`` (the Repoze.who configuration file) needs to be accessible in the same directory as your CKAN config file, so create a symlink to it: @@ -253,8 +326,9 @@ same directory as your CKAN config file, so create a symlink to it: ln -s |virtualenv|/src/ckan/who.ini |config_dir|/who.ini +--------------- 9. You're done! -~~~~~~~~~~~~~~~ +--------------- You can now use the Paste development server to serve CKAN from the command-line. This is a simple and lightweight way to serve CKAN that is @@ -277,3 +351,46 @@ Now that you've installed CKAN, you should: as Apache or Nginx. See :doc:`deployment`. * Begin using and customizing your site, see :doc:`/getting-started`. + + +------------------------------ +Source install troubleshooting +------------------------------ + +.. _solr troubleshooting: + +Solr setup troubleshooting +========================== + +Solr requests and errors are logged in the web server log files. + +* For Jetty servers, the log files are:: + + /var/log/jetty/.stderrout.log + +* For Tomcat servers, they're:: + + /var/log/tomcat6/catalina..log + +Unable to find a javac compiler +------------------------------- + +If when running Solr it says: + + Unable to find a javac compiler; com.sun.tools.javac.Main is not on the classpath. Perhaps JAVA_HOME does not point to the JDK. + +See the note in :ref:`setting up solr` about ``JAVA_HOME``. +Alternatively you may not have installed the JDK. +Check by seeing if ``javac`` is installed:: + + which javac + +If ``javac`` isn't installed, do:: + + sudo apt-get install openjdk-6-jdk + +and then restart Solr: + +.. parsed-literal:: + + |restart_solr| diff --git a/doc/solr-setup.rst b/doc/solr-setup.rst deleted file mode 100644 index 2b1279e9390..00000000000 --- a/doc/solr-setup.rst +++ /dev/null @@ -1,266 +0,0 @@ -.. _setting up solr: - -=============== -Setting up Solr -=============== - -CKAN uses Solr_ as search platform. This document describes different -topics related with the deployment and management of Solr from a CKAN -point of view. - -.. _Solr: http://lucene.apache.org/solr/ - -CKAN uses customized schema files that take into account its specific -search needs. Different versions of the schema file are found in -``ckan/ckan/config/solr`` - -The following instructions apply to Ubuntu 12.04 (Precise), the recommended -platform by the CKAN team. Other versions or distributions may need -slightly different instructions. - -.. note:: - - The following instructions deploy Solr on the Jetty server, but CKAN does - not require it, you can use Tomcat if that is more convenient on your - distribution. - - -.. _solr-single: - -Single Solr instance --------------------- - -In this case, there will be only one Solr endpoint that uses a single schema file. -This can be useful for a Solr server used by only a single CKAN instance, or -different instances that share the same schema version. - -To install Solr (if you are following the :doc:`install-from-source` or -:doc:`install-from-package` instructions, you already did this):: - - sudo apt-get install solr-jetty openjdk-6-jdk - -You'll need to edit the Jetty configuration file (``/etc/default/jetty``) with -the suitable values:: - - NO_START=0 # (line 4) - JETTY_HOST=127.0.0.1 # (line 15) - JETTY_PORT=8983 # (line 18) - -Start the Jetty server:: - - sudo service jetty start - -You should see welcome page from Solr when visiting (replace localhost with your -server address if needed):: - - http://localhost:8983/solr/ - -and the admin site:: - - http://localhost:8983/solr/admin - -.. note:: - - If you get the message ``Could not start Jetty servlet engine because no - Java Development Kit (JDK) was found.`` then you will have to edit the - ``JAVA_HOME`` setting in ``/etc/default/jetty`` to point to your machine's - JDK install location. For example:: - - JAVA_HOME=/usr/lib/jvm/java-6-openjdk-amd64/ - - or:: - - JAVA_HOME=/usr/lib/jvm/java-6-openjdk-i386/ - -This default setup will use the following locations in your file system: - -``/usr/share/solr`` - Solr home, with a symlink pointing to the configuration dir in ``/etc``. -``/etc/solr/conf`` - Solr configuration files. The more important ones are ``schema.xml`` and - ``solrconfig.xml``. -``/var/lib/solr/data/`` - This is where the index files are physically stored. - -You will obviously need to replace the default ``schema.xml`` file with the -CKAN one. To do so, create a symbolic link to the schema file in the config -folder. Use the latest schema version supported by the CKAN version you are -installing (it will generally be the highest one): - -.. parsed-literal:: - - sudo mv /etc/solr/conf/schema.xml /etc/solr/conf/schema.xml.bak - sudo ln -s |virtualenv|/src/ckan/ckan/config/solr/schema-2.0.xml /etc/solr/conf/schema.xml - -Now restart jetty:: - - sudo service jetty restart - -And check that Solr is running by browsing http://localhost:8983/solr/ which should offer the Administration link. - - -.. _solr-multi-core: - -Multiple Solr cores -------------------- - -Solr can also be set up to have multiple configurations and indexes on the -same instance. This is specially useful when you want other applications than CKAN -or different CKAN versions to use the same Solr instance. The different cores -will have different paths in the Solr server URL:: - - http://localhost:8983/solr/ckan-schema-1.2 # Used by CKAN up to 1.5 - http://localhost:8983/solr/ckan-schema-1.3 # Used by CKAN 1.5.1 - http://localhost:8983/solr/ckan-schema-1.4 # Used by CKAN 1.7 - http://localhost:8983/solr/ckan-schema-2.0 # Used by CKAN 2.0 - http://localhost:8983/solr/some-other-site # Used by another site - -To set up a multicore Solr instance, repeat the steps on the previous section -to configure a single Solr instance. - -Create a ``solr.xml`` file in ``/usr/share/solr``. This file will list the -different cores, and allows also to define some configuration options. -This is how cores are defined:: - - - - - - - - - - - - -Adjust the names to match the CKAN schema versions you want to run. - -Note that each core is configured with its own data directory. This is really important to prevent conflicts between cores. Now create them like this:: - - sudo -u jetty mkdir /var/lib/solr/data/ckan-schema-1.4 - sudo -u jetty mkdir /var/lib/solr/data/ckan-schema-2.0 - -For each core, we will create a folder in ``/usr/share/solr``, -with a symbolic link to a specific configuration folder in ``/etc/solr/``. -Copy the existing conf directory to the core directory and link it from -the home dir like this:: - - sudo mkdir /etc/solr/ckan-schema-1.4 - sudo mv /etc/solr/conf /etc/solr/ckan-schema-1.4/ - - sudo mkdir /usr/share/solr/ckan-schema-1.4 - sudo ln -s /etc/solr/ckan-schema-1.4/conf /usr/share/solr/ckan-schema-1.4/conf - -Now configure the core to use the data directory you have created. Edit ``/etc/solr/ckan-schema-1.4/conf/solrconfig.xml`` and change the ```` to this variable:: - - ${dataDir} - -This will ensure the core uses the data directory specified earlier in ``solr.xml``. - -Once you have your first core configured, to create new ones, you just need to -add them to the ``solr.xml`` file and copy the existing configuration dir:: - - sudo mkdir /etc/solr/ckan-schema-2.0 - sudo cp -R /etc/solr/ckan-schema-1.4/conf /etc/solr/ckan-schema-2.0 - - sudo mkdir /usr/share/solr/ckan-schema-2.0 - sudo ln -s /etc/solr/ckan-schema-2.0/conf /usr/share/solr/ckan-schema-2.0/conf - -Remember to ensure that each core points to the correct CKAN schema. To link -each schema to the relevant file on the CKAN source use the following: - -.. parsed-literal:: - - sudo rm /etc/solr/ckan-schema-2.0/conf/schema.xml - sudo ln -s |virtualenv|/src/ckan/ckan/config/solr/schema-2.0.xml /etc/solr/ckan-schema-2.0/conf/schema.xml - -Now restart jetty:: - - sudo service jetty restart - -And check that Solr is listing all the cores when browsing http://localhost:8983/solr/ - -Troubleshooting ---------------- - -Solr requests and errors are logged in the web server log. - -* For jetty servers, they are located in:: - - /var/log/jetty/.stderrout.log - -* For Tomcat servers, they are located in:: - - /var/log/tomcat6/catalina..log - -Some problems that can be found during the install: - -* When setting up a multi-core Solr instance, no cores are shown when visiting the - Solr index page, and the admin interface returns a 404 error. - - Check the web server error log if you can find an error similar to this one:: - - WARNING: [iatiregistry.org] Solr index directory '/usr/share/solr/iatiregistry.org/data/index' doesn't exist. Creating new index... - 07-Dec-2011 18:06:33 org.apache.solr.common.SolrException log - SEVERE: java.lang.RuntimeException: Cannot create directory: /usr/share/solr/iatiregistry.org/data/index - [...] - - The ``dataDir`` is not properly configured. With our setup the data directory should - be under ``/var/lib/solr/data``. Make sure that you defined the correct ``dataDir`` - in the ``solr.xml`` file and that in the ``solrconfig.xml`` file you have the - following configuration option:: - - ${dataDir} - -* When running Solr it says ``Unable to find a javac compiler; com.sun.tools.javac.Main is not on the classpath. Perhaps JAVA_HOME does not point to the JDK.`` - - See the note above about ``JAVA_HOME``. Alternatively you may not have installed the JDK. Check by seeing if javac is installed:: - - which javac - - If it isn't do:: - - sudo apt-get install openjdk-6-jdk - - and restart Solr. - -Handling changes in the CKAN schema ------------------------------------ - -At some point, changes in new CKAN versions will mean modifications in the schema -to support new features or fix defects. These changes won't be always backwards -compatible, so some changes in the Solr servers will need to be performed. - -If a CKAN instance is using a Solr server for itself, the schema can just be updated -on the Solr server and the index rebuilt. But if a Solr server is shared between -different CKAN instances, there may be conflicts if the schema is updated. - -CKAN uses the following conventions for supporting different schemas: - -* If needed, create a new schema file when releasing a new version of CKAN (i.e if there - are two or more different modifications in the schema file between CKAN releases, - only one new schema file is created). - -* Keep different versions of the Solr schema in the CKAN source, with a naming convention, - `schema-.xml`:: - - ckan/config/solr/schema-1.2.xml - ckan/config/solr/schema-1.3.xml - ckan/config/solr/schema-2.0.xml - -* Each new version of the schema file must include its version in the main `` tag:: - - - -* Solr servers used by more than one CKAN instance should be configured as multiple cores, - and provide a core for each schema version needed. The cores should be named following the - convention `schema-`, e.g.:: - - http:///solr/ckan-schema-1.4/ - http:///solr/ckan-schema-2.0/ - -When a new version of the schema becomes available, a new core is created, with a link to the -latest schema.xml file in the CKAN source. That way, CKAN instances that use an older version -of the schema can still point to the core that uses it, while more recent versions can point -to the latest one. When old versions of CKAN are updated, they only need to change their -:ref:`solr_url` setting to point to the suitable Solr core.