This project’s goal is to be able to run Apache Solr in an OSGi environment and bring the OSGi dynamic lifecycle to the Solr world.
|
Note
|
Not all of the Solr features are supported yet, clustering and sharding haven’t been tested. In theory every feature can be implemented in the future. Cellar based SolCloud implementation is planned. Currently only the configSet based cores can be used. |
The solr-osgi package bundled as offline karaf package. It can be installed and executed.
Compile project
./mvnw clean install
To run on MacOS / Linux, use
./run-karaf
after compilation.
Two examples are embedded, which can be installed.
For Northwind example - which is the SQL DataImport example - northwind example has to be started before feature is started.
docker run --name northwind -itd -e MYSQL_ROOT_PASSWORD=northwind -p 3306:3306 bylek/northwind-mysql
When karaf is running, open the browser:
Use karaf as username and password.
After install exampleCollection and exampleNorthwind feature, point the browser to http://localhost:8181/solr
Select northwind1 core.
Select Dataimport menu
Check Auto-Refresh Status and push Execute
After a couple of seconds, the following page shows that the data has been indexed.
Let’s start to search / browse. Select the Query function.
And execute one
And check the result.
Happy searching!
Any OSGi 6.0 container can be used for deployment. It is tested with Apache Karaf 4.3.6.
To install in karaf console:
feature:repo-add cxf feature:repo-add mvn:hu.blackbelt/solr-osgi-feature/<version>/xml/features feature:install -v solr-http
To be able to use the SolrCoreContainerManager’s functionality you have to create a configuration with pid solr.corecontainer
Create a file solr.corecontainer.cfg in karaf/deploy with the following content:
solrHome=/tmp/solr
karaf@root()> log:tail 2018-08-22 00:33:01,207 | INFO | tures-1-thread-1 | HttpServiceFactoryImpl | 181 - org.ops4j.pax.web.pax-web-runtime - 6.0.6 | Binding bundle: [org.restlet.ext.servlet [185]] to http service 2018-08-22 00:33:01,208 | DEBUG | tures-1-thread-1 | HttpServiceStarted | 181 - org.ops4j.pax.web.pax-web-runtime - 6.0.6 | Creating http service for: org.restlet.ext.servlet [185] 2018-08-22 00:33:01,210 | DEBUG | tures-1-thread-1 | HttpServiceProxy | 181 - org.ops4j.pax.web.pax-web-runtime - 6.0.6 | HttpServiceProxy created for HttpService org.ops4j.pax.web.service.internal.HttpServiceStarted@6de4a397 for bundle org.restlet.ext.servlet [185] 2018-08-22 00:33:01,214 | INFO | tures-1-thread-1 | FeaturesServiceImpl | 9 - org.apache.karaf.features.core - 4.1.2 | hu.blackbelt.solr-osgi-services/1.0.0.SNAPSHOT 2018-08-22 00:33:01,342 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | ___ _ Welcome to Apache Solr? version 7.0.1 2018-08-22 00:33:01,344 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | / __| ___| |_ _ Starting in standalone mode 2018-08-22 00:33:01,344 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | \__ \/ _ \ | '_| 2018-08-22 00:33:01,358 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | |___/\___/_|_| Start time: 2018-08-21T22:33:01.345Z 2018-08-22 00:33:01,459 | INFO | tures-1-thread-1 | SolrResourceLoader | 147 - org.apache.solr - 7.0.1.1 | solr home defaulted to 'solr/' (could not find system property or JNDI) 2018-08-22 00:33:01,867 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: / 2018-08-22 00:33:01,869 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: / 2018-08-22 00:33:01,882 | INFO | tures-1-thread-1 | SolrXmlConfig | 147 - org.apache.solr - 7.0.1.1 | MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@67117f44, but no JMX reporters were configured - adding default JMX reporter. 2018-08-22 00:33:02,120 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: / 2018-08-22 00:33:02,120 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: / 2018-08-22 00:33:02,121 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: / 2018-08-22 00:33:02,382 | WARN | tures-1-thread-1 | LogWatcher | 147 - org.apache.solr - 7.0.1.1 | Unable to read SLF4J version. LogWatcher will be disabled: java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder 2018-08-22 00:33:02,390 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.ZookeeperInfoHandler Expected tyoe: org.apache.solr.handler.admin.ZookeeperInfoHandler 2018-08-22 00:33:02,402 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.CollectionsHandler Expected tyoe: org.apache.solr.handler.admin.CollectionsHandler 2018-08-22 00:33:04,662 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.InfoHandler Expected tyoe: org.apache.solr.handler.admin.InfoHandler 2018-08-22 00:33:04,672 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.CoreAdminHandler Expected tyoe: org.apache.solr.handler.admin.CoreAdminHandler 2018-08-22 00:33:04,690 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.ConfigSetsHandler Expected tyoe: org.apache.solr.handler.admin.ConfigSetsHandler 2018-08-22 00:33:04,696 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.MetricsHandler Expected tyoe: org.apache.solr.handler.admin.MetricsHandler 2018-08-22 00:33:04,697 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.MetricsCollectorHandler Expected tyoe: org.apache.solr.handler.admin.MetricsCollectorHandler 2018-08-22 00:33:04,706 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.cloud.autoscaling.AutoScalingHandler Expected tyoe: org.apache.solr.cloud.autoscaling.AutoScalingHandler 2018-08-22 00:33:04,718 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.metrics.reporters.SolrJmxReporter Expected tyoe: org.apache.solr.metrics.SolrMetricReporter 2018-08-22 00:33:04,751 | INFO | tures-1-thread-1 | SolrJmxReporter | 147 - org.apache.solr - 7.0.1.1 | JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67117f44 2018-08-22 00:33:04,752 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.metrics.reporters.SolrJmxReporter Expected tyoe: org.apache.solr.metrics.SolrMetricReporter 2018-08-22 00:33:04,753 | INFO | tures-1-thread-1 | SolrJmxReporter | 147 - org.apache.solr - 7.0.1.1 | JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67117f44 2018-08-22 00:33:04,753 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.metrics.reporters.SolrJmxReporter Expected tyoe: org.apache.solr.metrics.SolrMetricReporter 2018-08-22 00:33:04,754 | INFO | tures-1-thread-1 | SolrJmxReporter | 147 - org.apache.solr - 7.0.1.1 | JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67117f44 2018-08-22 00:33:04,793 | INFO | tures-1-thread-1 | CorePropertiesLocator | 147 - org.apache.solr - 7.0.1.1 | Found 0 core definitions underneath /
To add a configSet template, copy the solr-osgi-example-configsets-exampleCollection projects jar to karaf/deploy
2018-08-22 00:33:04,850 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Solr content found in bundle hu.blackbelt.solr-osgi-example-configsets-exampleCollection Registering 2018-08-22 00:33:04,852 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/lang/stopwords_en.txt to configsets/exampleCollection/conf/lang/stopwords_en.txt 2018-08-22 00:33:04,858 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/params.json to configsets/exampleCollection/conf/params.json 2018-08-22 00:33:04,859 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/protwords.txt to configsets/exampleCollection/conf/protwords.txt 2018-08-22 00:33:04,859 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/schema.xml to configsets/exampleCollection/conf/schema.xml 2018-08-22 00:33:04,860 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/solrconfig.xml to configsets/exampleCollection/conf/solrconfig.xml 2018-08-22 00:33:04,861 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/stopwords.txt to configsets/exampleCollection/conf/stopwords.txt 2018-08-22 00:33:04,861 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/synonyms.txt to configsets/exampleCollection/conf/synonyms.txt 2018-08-22 00:33:04,869 | INFO | tures-1-thread-1 | SolrCoreContainerMan
The configSet files are processed and copied to the in-memory store.
To create a core based on a configSet, create a configuration with PID solr.core-<pid>
In karaf create a file solr.core-test1.cfg in karaf/deploy with the following content:
name=test1 configSet=exampleCollection
To install the HTTP service, create a configuration with PID solr.http
In karaf create a file solr.http.cfg in karaf/deploy with the following content:
contextRoot=/solr
After the steps above, Solr Admin should be accessible at http://localhost:8181/solr/
Core containers have very same parameters as the original ones, but not all of them can be used now.
There is a declarative component called SolrCoreContainerManager which monitors the following items:
-
solr.core or solr.core-<pid> configuration PID-s in ConfigurationAdmin service. This configuration defines core configurations. The properties are very similar to the ones used in the CoreAdmin API. Custom properties can be defined in admin-requests, 'property' prefix can be used on custom properties. Two mandatory properties are:
nameandconfigSet. -
All bundles which have the Solr-Configset tag in META-INF/MANIFEST.MF. It loads configSets to the CoreContainer. The cores can use those configSets as a template. The configSets are stored to an in-Memory filesystem which is used to load files by solr.
The manager creates SolrClient instances when the configSet is loaded which are referenced by solr.core configurations. In the future the core definitions which do not have configSets can be created as normal cores, but for that some file based resource management implementation is required.
When a core config or configSet bundle is removed, the related solrClients will be stopped. The data directory is not removed, so it can be reused later.