Indexing annotations in Solr

The following instructions show you how to index annotations in Solr. It is meant as a guide. The example paths are based on those found in the development VM created by islandora_vagrant. Paths on your system may vary depending on your Islandora environment. As Islandora setups vary, please seek the assistance of your local system administrator and Solr expert on how to best implement indexing of annotations in your local Islandora instance.

1. Copy XSLT file annotation_solr.xslt provided with Islandora Web Annotation into the proper location:

 > cd /var/lib/tomcat7/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal/index/FgsIndex/islandora_transforms/
 > cp /var/www/drupal/sites/all/modules/islandora_web_annotations/xsl/annotation_solr.xslt ./

2. Edit foxmlToSolr.xslt, located in /var/lib/tomcat7/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal/index/FgsIndex/. Add the following line at the end of the other <xsl:include/> elements:

<!-- begin: islandora_web_annotations setup -->
<xsl:include href="/var/lib/tomcat7/webapps/fedoragsearch/WEB-INF/classes/fgsconfigFinal/index/FgsIndex/islandora_transforms/annotation_solr.xslt"/>
<!-- end: islandora_web_annotations setup -->

3. Edit /usr/local/solr/collection1/conf/schema.xml and add the following dynamicField at the end of the other dynamicField elements:

Add this to end of your field elements (just before </fields> ):

<!-- begin: islandora_web_annotations setup -->
<dynamicField name="annotation_*" type="text" indexed="true" stored="true" multiValued="true"/>
<field name="annotation_rangeTimeStart" type="text" indexed="true" stored="true" />
<field name="annotation_rangeTimeStart_sort" type="double" indexed="true" stored="true" multiValued="false"/>
<field name="annotation_value" type="text" indexed="true" stored="true" multiValued="true"/>
<field name="annotation_value_render" type="text" indexed="true" stored="true" multiValued="false"/>
<!-- end: islandora_web_annotations setup -->

Add this to copyField elements above <copyField source="*" dest="catch_all_fields_mt"/>:

<!-- begin: islandora_web_annotations setup -->
<copyField source="annotation_rangeTimeStart" dest="annotation_rangeTimeStart_sort"/>
<copyField source="annotation_value" dest="annotation_value_render"/>
<!-- end: islandora_web_annotations setup -->

In case text fieldType is not available in your setup. Please add that text fieldType to types.

    <fieldType name="text" class="solr.TextField" positionIncrementGap="100">
      <analyzer type="index">
        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
        <!-- in this example, we will only use synonyms at query time
        <filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
        -->
        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>
        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.SnowballPorterFilterFactory" language="English" protected="protwords.txt"/>
        <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>
        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.SnowballPorterFilterFactory" language="English" protected="protwords.txt"/>
        <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
      </analyzer>
    </fieldType>

In solrconfig.xml you should see a tag <str name="qf"> under <requestHandler name=“/select” class=“solr.SearchHandler”>. Add:

annotation_rangeTimeStart^1 annotation_rangeTimeStart_sort^1 annotation_value^1 annotation_value_render^1

so that it might look like this:

<str name="qf">dc.title^5 dc.subject^3 dc.description^3 dc.creator^3 dc.contributor^3 dc.type^1 dc.relation^1 dc.publisher^1 mods_identifier_local_ms^3 ds.WARC_FILTER^1 text_nodes_HOCR_hlt^1 mods_subject_hierarchicalGeographic_region_ms^3 mods_identifier_hdl_mt^3 dc.identifier^3 PID^0.5 catch_all_fields_mt^0.1 annotation_rangeTimeStart^1 annotation_rangeTimeStart_sort^1 annotation_value^1 annotation_value_render^1</str>

4. Restart tomcat

> sudo service tomcat7 stop
> sudo service tomcat7 start

5. Verify the changes. Create a new annotation. Copy the PID in the pop-up notification. Then search for that annotation in Solr using the PID. Ensure that the annotation related fields are added, for example:

<arr name="annotation_title">
   <str>Annotation for islandora%3A218</str>
</arr>
<arr name="annotation_value">
   <str>test annotation value</str>
</arr>

6. Alternatively, you can add the annotation related fields to Advanced Search and search via the Drupal UI.