Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Update to latest apachesolr.

  • Loading branch information...
commit ded7dbdc07b04ef4c6dc9f490a68537c446afa2d 1 parent 69786e1
@sdboyer sdboyer authored
Showing with 2,620 additions and 3,410 deletions.
  1. +131 −3 sites/all/modules/contrib/apachesolr/CHANGELOG.txt
  2. +27 −15 sites/all/modules/contrib/apachesolr/Drupal_Apache_Solr_Service.php
  3. +53 −19 sites/all/modules/contrib/apachesolr/README.txt
  4. +31 −8 sites/all/modules/contrib/apachesolr/Solr_Base_Query.php
  5. +154 −41 sites/all/modules/contrib/apachesolr/apachesolr.admin.inc
  6. +77 −12 sites/all/modules/contrib/apachesolr/apachesolr.api.php
  7. +317 −160 sites/all/modules/contrib/apachesolr/apachesolr.index.inc
  8. +4 −3 sites/all/modules/contrib/apachesolr/apachesolr.info
  9. +26 −4 sites/all/modules/contrib/apachesolr/apachesolr.install
  10. +200 −0 sites/all/modules/contrib/apachesolr/apachesolr.interface.inc
  11. +386 −173 sites/all/modules/contrib/apachesolr/apachesolr.module
  12. +3 −3 sites/all/modules/contrib/apachesolr/apachesolr_access/apachesolr_access.info
  13. +4 −8 sites/all/modules/contrib/apachesolr/apachesolr_access/tests/apachesolr_access.test
  14. +61 −40 sites/all/modules/contrib/apachesolr/apachesolr_search.admin.inc
  15. +3 −3 sites/all/modules/contrib/apachesolr/apachesolr_search.info
  16. +24 −0 sites/all/modules/contrib/apachesolr/apachesolr_search.install
  17. +203 −169 sites/all/modules/contrib/apachesolr/apachesolr_search.module
  18. +41 −28 sites/all/modules/contrib/apachesolr/apachesolr_search.pages.inc
  19. +391 −68 sites/all/modules/contrib/apachesolr/drush/apachesolr.drush.inc
  20. +6 −0 sites/all/modules/contrib/apachesolr/plugins/facetapi/query_type_term.inc
  21. +0 −8 sites/all/modules/contrib/apachesolr/solr-conf/protwords.txt
  22. +0 −513 sites/all/modules/contrib/apachesolr/solr-conf/schema-solr3x.xml
  23. +0 −497 sites/all/modules/contrib/apachesolr/solr-conf/schema.xml
  24. +0 −743 sites/all/modules/contrib/apachesolr/solr-conf/solrconfig-solr3x.xml
  25. +0 −736 sites/all/modules/contrib/apachesolr/solr-conf/solrconfig.xml
  26. +202 −5 sites/all/modules/contrib/apachesolr/tests/Dummy_Solr.php
  27. +143 −55 sites/all/modules/contrib/apachesolr/tests/apachesolr_base.test
  28. +4 −3 sites/all/modules/contrib/apachesolr/tests/apachesolr_test/apachesolr_test.info
  29. +7 −1 sites/all/modules/contrib/apachesolr/tests/solr_base_query.test
  30. +121 −92 sites/all/modules/contrib/apachesolr/tests/solr_index_and_search.test
  31. +1 −0  sites/all/modules/contrib/features/includes/features.user.inc
View
134 sites/all/modules/contrib/apachesolr/CHANGELOG.txt
@@ -1,6 +1,134 @@
-Apache Solr integration 7.x-1.x-xxxx, xxxx-xx-xx
-------------------------------
-Issue #1515580 by Nick_vh | yannou: Fixed Deleting node is impossible, stdClass problem.
+Apache Solr integration 7.x-1.x, xxxx-xx-xx
+-----------------------------
+
+Apache Solr integration 7.x-1.1, 2012-10-15
+-----------------------------
+#1780200 by pwolanin: document basic auth in README.
+#1226274 by pwolanin: Fixes for schema.xml version numbering and organization.
+
+Apache Solr integration 7.x-1.0, 2012-10-13
+-----------------------------
+#761990 by pwolanin, jhedstrom, Nick_vh, jpmckinney | morningtime: Fixed 400 Bad Status if URL length limit exceeded.
+#1811364 by Nick_vh: Fixed Add newly created content type to the indexed bundles for all environments.
+#1811456 by rupl: Fixed Improve documentation of addFilter in apachesolr.api.php.
+#1773506 by drzraf: Fixed drush solr-search notice when file entity are part of the result.
+#1688150 by HalfChem, cam8001, jhedstrom, Nick_vh: Fixed Search query string gets double encoded when core Search block form is submitted.
+#1794602 by pwolanin, Nick_vh | ppmr: Fixed Undefined index: path in ApacheSolrDocument->__get() (line 304 of ...\apachesolr\Apache_Solr_Document.
+#1773506 by drzraf: Fixed drush solr-search notice when file entity are part of the result.
+#1764334 by cpliakas: Fixed Negative percentage of documents sent to the server for indexing.
+#1806300 by Nick_vh: Separate out the creation of the document from the index process.
+#1807552 by Nick_vh, wimvds: Fixed Site and url wrong when indexing multilingual content (using i18n module).
+#1759190 by mkalkbrenner: Fixed obsolete form code?.
+#1802288 by Nick_vh: Fixed Improve testing of node deletion.
+
+Apache Solr integration 7.x-1.0-rc5, 2012-10-01
+-----------------------------
+#1698050 by duellj: fix for apachesolr_clean_text() should strip extra spaces.
+#1764450 by cpliakas: Improve the docblocks in apachesolr.index.inc.
+#1800324 by pwolanin: Clean up and align node access tests.
+#1799330 by pwolanin: fix for Stats table missing t() calls.
+#1787370 by pwolanin: Don't complain about schema version if it doesn't match expected naming pattern.
+#1790590 by pwolanin: Fix DrupalSolrOnlineWebTestCase so it works with any multicore setup.
+#1799032 by pwolanin: Remove uneeded object reference in indexing function.
+#1793610 by zengenuity: Fix for regession - spelling suggestions missing.
+
+Apache Solr integration 7.x-1.0-rc4, 2012-09-21
+-----------------------------
+#1790894 by pwolanin: fix for Cloning an environment doesn't clone the bundles to be indexed.
+#1778050 by Nick_vh, pwolanin: fix for stale cache when CTools is already enabled.
+#1778266 by Nick_vh, mkalkbrenner: Refactoring of DrupalSolrOnlineWebTestCase to be used by Others.
+#1778266 by mkalkbrenner: Refactoring of DrupalSolrOnlineWebTestCase to be used by Others.
+#1789526 by pwolanin: Clarify lack of upgrade path from 6.x-1.x.
+#1765938 by cpliakas: Added Move the variable_get() for 'apachesolr_environments()" after the cache_set() so that URLs can be modified dynamically.
+#1782436 by cspitzlay: Fixed Error and obsolete hint in code comment.
+#1732900 by cpliakas: Added Change the wording of the generic 'Apache Solr server unavailable" error message.
+#1781040 by mkalkbrenner: Fixed Switch 'Enable spell check" does not work - spell check is allways on.
+#1786450 by Nick_vh: Fixed apachesolr.interface.inc is not always loaded.
+#1760592 by cpliakas, pwolanin: fix for core_search page does not use the current default search environment.
+#1759004 by mkalkbrenner, pwolanin: fix for apachesolr_search_custom_page_search_form_submit() kills all $_GET parameters.
+#1778150 by cpliakas, pwolanin: fix for SQL error in apachesolr_environment_load_subrecords().
+#1743138 by mundanity: Fixed Invalid argument when running Drush.
+#1751640 by cpliakas: Added drush commands to see the ID of the last item indexes as well as the next item queued for indexing.
+#1708150 by ianmthomasuk, Nick_vh, pwolanin: Added additional typing to function definitions.
+#1759110 by Nick_vh: Fixed process response fails if variable was set but never removed.
+#1741066 by Nick_vh: Added geo query type to make contrib's modules life easier.
+#1730840 by pwolanin: Normalize boolean and other values for parameters like hl.
+#1729836 by pwolanin: fix for "Results per page" option for a search page is broken.
+#1719302 by Nick_vh, msti: Fixed Running an empty facet query so the facet blocks can be displayed.
+#1722966 by greggles: Fixed make apachesolr_index_batch_index_finished() message more translateable.
+#1717628 by Nick_vh, pwolanin: Added 'Save and Edit" functionality for configuration pages - Follow ups.
+
+Apache Solr integration 7.x-1.0-rc3, 2012-08-10
+------------------------------
+#1682004 by Nick_vh: Add template hints for search results based on search page.
+#1722012 by mkalkbrenner, Nick_vh: fix for Missing argument 4 for apachesolr_multilingual_apachesolr_index_documents_alter().
+#1717490 by pwolanin: fix for Search result template suggestions relies on non-required fields.
+#1708212 by ianmthomasuk, Nick_vh: Fixed Duplicate field error when indexing single value fields.
+#1702788 by mkalkbrenner: Fixed content and teaser are empty if node->language is not default.
+#1708166 by ianmthomasuk: Fixed Thow exception if asked for invalid solr environment.
+#1717628 by Nick_vh | cpliakas: Added 'Save and Edit" functionality for configuration pages.
+#1718172 by Nick_vh, pwolanin: Fixed Move search result alter hook earlier, and make sure needed data is set.
+#1702526 by Nick_vh: Fixed apachesolr_search_page_load() is quirky.
+#1717958 by pwolanin: Fix for text_und fields should be exposed for searching in the admin section.
+#1708538 by Nick_vh: Fixed Move the start param above the query alter.
+#1700472 by pwolanin: Make 'apachesolr_search_process_response_callback' a search environment variable.
+#1439564 by recidive, neochief, Nick_vh, dawehner, jhedstrom: Added Index bundles Export/Import.
+#1652746 by martins.bertins | chrisssi: Fixed Notice: Undefined offset: 3 in _menu_translate().
+#1670198 by alanmackenzie | ruedu: Fixed Unable to export/import settings using features module.
+#1681946 by Josh Waihi: Fixed apachesolr_index_nodeapi_mass_delete() doesn't work.
+#1650096 by Nick_vh | cappadona: Fixed addParam() creates duplicate filters when dealing with fq.
+#1677050 by cpliakas: Fix for Cancel link when creating a new environment.
+#1673086 by pwolanin: Add drush commands to get, set, del environment variables.
+#1682004 by cpliakas: Added template hints for search results based on search page.
+#1672738 by Nick_vh: Added Allow a dynamic apachesolr_process_results() function.
+#1473722 by levialliance: Added Bundle specific overrides.
+#1408844 by Nick_vh, jrbeeman: Added As a site builder I want to override the environment settings in settings.php.
+#1652746 by chrisssi: Fixed Notice: Undefined offset: 3 in _menu_translate() (Zeile 775 von /var/www/includes/menu.inc).
+#1402748 by Nick_vh, killua99: Fixed Check in apachesolr_do_query() if the static of the query with that name already exists, and if so return it.
+
+Apache Solr integration 7.x-1.0-rc2, 2012-06-21
+------------------------------
+Renamed schema.xml and solrconfig.xml to rc2
+
+Apache Solr integration 7.x-1.0-rc1, 2012-06-21
+------------------------------
+#1642140 by Nick_vh: Fixed Make the search query also escape the slash and move away from menu_tail() to just 1 argument.
+#1518108 by grendzy: Added Devel interface for inspecting solr documents.
+#1402748 by Nick_vh: Fixed Check in apachesolr_do_query() if the static of the query with that name already exists, and if so return it.
+#1631702 by Nick_vh | khaled.zaidan: Fixed Report at admin/reports/apachesolr doesn't always display data from the current default solr server.
+#1647668 by paulmckibben: Fixed Error messages when indexing - Notice: Undefined property: stdClass::$nid in DatabaseStatementBase->fetchAllAssoc().
+#1475014 by ceng, Nick_vh: Fixed Drush command solr-delete-index ignores type arguments.
+#1588256 by InternetDevels.Com, Nick_vh | RaulMuroc: Fixed Warning: Invalid argument supplied for foreach() in apachesolr_entity_update() (line 1766 of /apachesolr/apachesolr.module).
+#1612530 by Nick_vh, willmoy: Added Results say '0 comments' when comment.module is disabled.
+#1522174 by pwolanin: Fixed hook_node_type_delete() is wrong.
+#1576710 by skwashd: Added Indexing with drush cron generates a lot of invalid cron notifications.
+#1543754 by iamEAP: Fixed Schema inconsistency introduced by apachesolr_update_7013.
+#1481564 by milesw: Added Allow other modules to react when there are no results.
+#1470794 by pwolanin: Added Show size of index on disk in index report.
+#1586320 by cpliakas, pwolanin: Added support for the ExternalFileField field type.
+#1613328 by cpliakas: Fixed The service_class() is not passed to apachesolr_server_status() when checking each server's status on the settings page.
+#1621142 by pwolanin: Fixed Broken logic in indexing of extra data.
+#1627484 by gnucifer: Fixed 'allowed operator" not transfered from field definitions.
+#1627604 by Nick_vh: Fixed Add query type for taxonomy_term_reference().
+#1609184 by pwolanin: Minor code cleanup in MLT response handling .
+#1611338 by remimikalsen: Added Allowing several bundles to share the same field name within a given entity type after field_mapping_alter().
+#1568126 by fearlsgroove, pwolanin: Fixed Apachesolr causes all simpletests to fail.
+#1609030 by pwolanin: Fix for Missing search page still executes search.
+#1572722 by eporama, cpliakas, wonder95: Added integration with the Entity References module.
+#1567614 by pwolanin: fix for Uninstall is not complete.
+#1559880 by pwolanin: fix for 'retain filters' checkbox does not work.
+#1558626 by pwolanin: fix for broken OR facet when value has spaces (committed in 16ef187).
+#1538244 by by julianmancera: Fix for Solr base query adds an extra slash in getPath() when there is no 'q' param.
+#1532214 by setvik, pwolanin: Fox for mergePolicy syntax has changed and throws error in Apache Solr 3.6, update config versions.
+#1536936 by pwolanin: Fix for Use json.nl=map for Luke and other json data.
+#1536628 by Nick_vh: Added exclude hook that allows to skip content without removing it from the table.
+#1536600 by Nick_vh: Fixed Performance optimizations when doing hundreds of deletes.
+#1515822 by pwolanin, Nick_vh: Added Support notion of parent entity for derived documents.
+#1519900 by pwolanin, Nick_vh, Georgique: Fixed Error in apachesolr_index_get_entities_to_index() function.
+
+Apache Solr integration 7.x-1.0-beta19, 2012-04-05
+------------------------------
+#1515580 by Nick_vh | yannou: Fixed Deleting node is impossible, stdClass problem.
Apache Solr integration 7.x-1.0-beta18, 2012-04-04
------------------------------
View
42 sites/all/modules/contrib/apachesolr/Drupal_Apache_Solr_Service.php
@@ -62,7 +62,7 @@
* methods for pinging, adding, deleting, committing, optimizing and searching.
*/
-class DrupalApacheSolrService {
+class DrupalApacheSolrService implements DrupalApacheSolrServiceInterface {
/**
* How NamedLists should be formatted in the output. This specifically effects facet counts. Valid values
* are 'map' (default) or 'flat'.
@@ -143,7 +143,7 @@ public function ping($timeout = 2) {
* @return
* (array) With all the system info
*/
- public function setSystemInfo() {
+ protected function setSystemInfo() {
$url = $this->_constructUrl(self::SYSTEM_SERVLET, array('wt' => 'json'));
if ($this->env_id) {
$this->system_info_cid = $this->env_id . ":system:" . drupal_hash_base64($url);
@@ -180,7 +180,12 @@ public function getSystemInfo() {
*/
protected function setLuke($num_terms = 0) {
if (empty($this->luke[$num_terms])) {
- $url = $this->_constructUrl(self::LUKE_SERVLET, array('numTerms' => "$num_terms", 'wt' => 'json'));
+ $params = array(
+ 'numTerms' => "$num_terms",
+ 'wt' => 'json',
+ 'json.nl' => self::NAMED_LIST_FORMAT,
+ );
+ $url = $this->_constructUrl(self::LUKE_SERVLET, $params);
if ($this->env_id) {
$cid = $this->env_id . ":luke:" . drupal_hash_base64($url);
$cache = cache_get($cid, 'cache_apachesolr');
@@ -267,25 +272,28 @@ public function getStatsSummary() {
'@deletes_total' => '',
'@schema_version' => '',
'@core_name' => '',
+ '@index_size' => '',
);
if (!empty($stats)) {
$docs_pending_xpath = $stats->xpath('//stat[@name="docsPending"]');
- $summary['@pending_docs'] = (int) trim($docs_pending_xpath[0]);
+ $summary['@pending_docs'] = (int) trim(current($docs_pending_xpath));
$max_time_xpath = $stats->xpath('//stat[@name="autocommit maxTime"]');
$max_time = (int) trim(current($max_time_xpath));
// Convert to seconds.
$summary['@autocommit_time_seconds'] = $max_time / 1000;
$summary['@autocommit_time'] = format_interval($max_time / 1000);
$deletes_id_xpath = $stats->xpath('//stat[@name="deletesById"]');
- $summary['@deletes_by_id'] = (int) trim($deletes_id_xpath[0]);
+ $summary['@deletes_by_id'] = (int) trim(current($deletes_id_xpath));
$deletes_query_xpath = $stats->xpath('//stat[@name="deletesByQuery"]');
- $summary['@deletes_by_query'] = (int) trim($deletes_query_xpath[0]);
+ $summary['@deletes_by_query'] = (int) trim(current($deletes_query_xpath));
$summary['@deletes_total'] = $summary['@deletes_by_id'] + $summary['@deletes_by_query'];
$schema = $stats->xpath('/solr/schema[1]');
$summary['@schema_version'] = trim($schema[0]);;
$core = $stats->xpath('/solr/core[1]');
$summary['@core_name'] = trim($core[0]);
+ $size_xpath = $stats->xpath('//stat[@name="indexSize"]');
+ $summary['@index_size'] = trim(current($size_xpath));
}
return $summary;
@@ -383,6 +391,7 @@ public function makeServletRequest($servlet, $params = array(), $options = array
// Add default params.
$params += array(
'wt' => 'json',
+ 'json.nl' => self::NAMED_LIST_FORMAT,
);
$url = $this->_constructUrl($servlet, $params);
@@ -416,7 +425,7 @@ protected function _sendRawPost($url, $options = array()) {
*
* This is just a wrapper around drupal_http_request().
*/
- protected function _makeHttpRequest($url, $options = array()) {
+ protected function _makeHttpRequest($url, array $options = array()) {
if (!isset($options['method']) || $options['method'] == 'GET' || $options['method'] == 'HEAD') {
// Make sure we are not sending a request body.
$options['data'] = NULL;
@@ -725,7 +734,7 @@ public function optimize($waitFlush = true, $waitSearcher = true, $timeout = 360
/**
* Like PHP's built in http_build_query(), but uses rawurlencode() and no [] for repeated params.
*/
- public function httpBuildQuery(array $query, $parent = '') {
+ protected function httpBuildQuery(array $query, $parent = '') {
$params = array();
foreach ($query as $key => $value) {
@@ -757,10 +766,7 @@ public function httpBuildQuery(array $query, $parent = '') {
*
* @throws Exception If an error occurs during the service call
*/
- public function search($query = '', $params = array(), $method = 'GET') {
- if (!is_array($params)) {
- $params = array();
- }
+ public function search($query = '', array $params = array(), $method = 'GET') {
// Always use JSON. See http://code.google.com/p/solr-php-client/issues/detail?id=6#c1 for reasoning
$params['wt'] = 'json';
// Additional default params.
@@ -773,9 +779,15 @@ public function search($query = '', $params = array(), $method = 'GET') {
// PHP's built in http_build_query() doesn't give us the format Solr wants.
$queryString = $this->httpBuildQuery($params);
// Check string length of the query string, change method to POST
- // if longer than 4000 characters (typical server handles 4096 max).
- // @todo - make this a per-server setting.
- if (strlen($queryString) > variable_get('apachesolr_search_post_threshold', 4000)) {
+ $len = strlen($queryString);
+ // Fetch our threshold to find out when to flip to POST
+ $max_len = apachesolr_environment_variable_get($this->env_id, 'apachesolr_search_post_threshold', 3600);
+
+ // if longer than $max_len (default 3600) characters
+ // we should switch to POST (a typical server handles 4096 max).
+ // If this class is used independently (without environments), we switch automatically to POST at an
+ // limit of 1800 chars.
+ if (($len > 1800) && (empty($this->env_id) || ($len > $max_len))) {
$method = 'POST';
}
View
72 sites/all/modules/contrib/apachesolr/README.txt
@@ -10,15 +10,18 @@ must be used in your Solr installation.
This module depends on the search framework in core. When used in combination
with core search module, Apache Solr is not the default search. Access it via a
-new tab on the default search page, called "Search". You may configure it
+new tab on the default search page, called "Site". You may configure it
to be default at ?q=admin/config/search/settings
Updating from 6.x
-----------------
-Make sure that you have first updated to the latest 6.x version on the relevant
-branch and that you have run all schema updates. You will have to install the
-new schema.xml and solrconfig.xml files, and restart the Solr server (or core)
-and delete your index and reindex all content.
+
+IMPORTANT: there is no upgrade path from 6.x-1.x or 6.x-2.x. If you previously
+installed those modules you must disable and uninstall them prior to
+installing 7.x-1.x.
+
+You will have to install the new schema.xml and solrconfig.xml files, and restart
+the Solr server (or core) and delete your index and reindex all content.
Installation
------------
@@ -35,31 +38,45 @@ subscribed to a service like Acquia Search.
The Debian/Ubuntu packages for Solr should NOT be used to install Solr.
For example, do NOT install the solr or solr-jetty packages.
-Download the latest Solr 1.4.x or 3.x release (e.g. 1.4.1 or 3.5.0) from:
+Download the latest Solr 1.4.x or 3.x release (e.g. 1.4.1 or 3.6.1) from:
http://www.apache.org/dyn/closer.cgi/lucene/solr/
Apache Lucene 3.1, 3.2 or 3.3, have a possible index corruption bug on
-server crash or power loss (LUCENE-3418). Solr 3.4 has a problem
-with SortMissingLast so Solr 3.5.0 is preferred.
+server crash or power loss (LUCENE-3418) and have bugs that interfere
+with the Drupal admin reports. Solr 3.4 has a problem with
+SortMissingLast so Solr 3.5.0 or later is strongly preferred.
-Unpack the tarball somewhere not visible to the web (not in your webserver
-docroot and not inside of your Drupal directory).
+Unpack the Solr tarball somewhere not visible to the web (not in your
+webserver docroot and not inside of your Drupal directory).
The Solr download comes with an example application that you can use for
testing, development, and even for smaller production sites. This
application is found at apache-solr-1.4.1/example.
+You must use 3 Solr configuration files that come with the Drupal
+module or the integration will not work correctly.
+
+For Solr 1.4 use the ones found in:
+solr-conf/solr-1.4/
+
+for Solr 3.5.0 or 3.6.1 use:
+solr-conf/solr-3.x/
+
+While the Solr 1.4 files will work for Solr 3.5+, they are not optimal
+and you will be missing important new features.
+
+For example, when deploying solr 1.4:
+
Move apache-solr-1.4.1/example/solr/conf/schema.xml and rename it to
-something like schema.bak. Then move the solr-conf/schema.xml that
-comes with this Drupal module to take its place. If you are using
-Solr 3.5 or later, you can use solr-conf/schema-solr3x.xml instead.
+something like schema.bak. Then move the solr-conf/solr-1.4/schema.xml
+that comes with this Drupal module to take its place.
Similarly, move apache-solr-1.4.1/example/solr/conf/solrconfig.xml and rename
-it like solrconfig.bak. Then move the solr-conf/solrconfig.xml that comes
-with this module to take its place.
+it like solrconfig.bak. Then move the solr-conf/solr-1.4/solrconfig.xml
+that comes with this module to take its place.
-Finally, move apache-solr-1.4.1/example/solr/conf/protwords.txt and rename
-it like protwords.bak. Then move the solr-conf/protwords.txt that comes
+Finally, move apache-solr-1.4.1/example/solr/conf/protwords.txt and rename it
+protwords.bak. Then move the solr-conf/solr-1.4/protwords.txt that comes
with this module to take its place.
Make sure that the conf directory includes the following files - the Solr core
@@ -93,8 +110,15 @@ Once this module is enabled, enable blocks for facets first at
Administer > Site configuration > Apache Solr > Enabled filters
then position them as you like at Administer > Site building > Blocks.
+Settings.php
+------------
+You can override environment settings using the following syntax in your
+settings.php
+
+$conf['apachesolr_environments']['my_env_id']['url'] = 'http://localhost:8983';
+
Configuration variables
---------------
+-----------------------
The module provides some (hidden) variables that can be used to tweak its
behavior:
@@ -130,8 +154,18 @@ behavior:
each Solr request, such as when making {apachesolr_search_node} consistent
with {node}.
+ - apachesolr_index_user: Define with which user you want the index process to
+ happen.
+
Troubleshooting
---------------
+---------------
+Problem:
+You use http basic auth to limit access to your Solr server.
+
+Solution:
+Set the Server URL to include the username and password like
+http://username:password@example.com:8080/solr
+
Problem:
Links to nodes appear in the search results with a different host name or
subdomain than is preferred. e.g. sometimes at http://example.com
View
39 sites/all/modules/contrib/apachesolr/Solr_Base_Query.php
@@ -278,13 +278,14 @@ class SolrBaseQuery extends SolrFilterSubQuery implements DrupalSolrQueryInterfa
protected $solrsort = array('#name' => 'score', '#direction' => 'desc');
// A flag to allow the search to be aborted.
public $abort_search = FALSE;
-
+
// A flag to check if need to retrieve another page of the result set
public $page = 0;
/**
- * @param $env_id
- * The environment where you are calling the query from. Typically the default environment.
+ * @param $name
+ * The search name, used for finding the correct blocks and other config.
+ * Typically "apachesolr".
*
* @param $solr
* An instantiated DrupalApacheSolrService Object.
@@ -444,7 +445,7 @@ public function addParam($name, $value) {
if (is_array($value)) {
$value = end($value);
}
- $this->params[$name] = trim($value);
+ $this->params[$name] = $this->normalizeParamValue($value);
return $this;
}
// We never actually populate $this->params['fq']. Instead
@@ -463,15 +464,28 @@ public function addParam($name, $value) {
$this->params[$name] = array();
}
- if (is_array($value)) {
- $this->params[$name] = array_merge($this->params[$name], array_values($value));
+ if (!is_array($value)) {
+ // Convert to array for array_map.
+ $param_values = array($value);
}
else {
- $this->params[$name][] = $value;
+ // Convert to a numerically keyed array.
+ $param_values = array_values($value);
}
+ $this->params[$name] = array_merge($this->params[$name], array_map(array($this, 'normalizeParamValue'), $param_values));
+
return $this;
}
+ protected function normalizeParamValue($value) {
+ // Convert boolean to string.
+ if (is_bool($value)) {
+ return $value ? 'true' : 'false';
+ }
+ // Convert to trimmed string.
+ return trim($value);
+ }
+
public function addParams(Array $params) {
foreach ($params as $name => $value) {
$this->addParam($name, $value);
@@ -572,7 +586,16 @@ public function getPath($new_keywords = NULL) {
if (isset($new_keywords)) {
return $this->base_path . '/' . $new_keywords;
}
- return $this->base_path . '/' . $this->getParam('q');
+ elseif ($this->getParam('q')) {
+ return $this->base_path . '/' . $this->getParam('q');
+ }
+ else {
+ // Return with empty query (the slash). The path for a facet
+ // becomes $this->base_path . '//facetinfo';
+ // We do this so we can have a consistent way of retrieving the query +
+ // additional parameters
+ return $this->base_path . '/';
+ }
}
public function getSolrsortUrlQuery() {
View
195 sites/all/modules/contrib/apachesolr/apachesolr.admin.inc
@@ -15,12 +15,18 @@ function apachesolr_environment_delete_form($form, &$form_state, $environment) {
'#type' => 'value',
'#value' => $environment['env_id'],
);
+ if (isset($environment['export_type']) && $environment['export_type'] == 3) {
+ $verb = t('Revert');
+ }
+ else {
+ $verb = t('Delete');
+ }
return confirm_form(
$form,
- t('Are you sure you want to delete search environment %name?', array('%name' => $environment['name'])),
+ t('Are you sure you want to !verb search environment %name?', array('%name' => $environment['name'], '!verb' => strtolower($verb))),
'admin/config/search/apachesolr',
t('This action cannot be undone.'),
- t('Delete'),
+ $verb,
t('Cancel')
);
}
@@ -149,6 +155,12 @@ function apachesolr_environment_edit_form($form, &$form_state, $environment = NU
'#submit' => array('apachesolr_environment_edit_submit'),
'#value' => t('Save'),
);
+ $form['actions']['save_edit'] = array(
+ '#type' => 'submit',
+ '#validate' => array('apachesolr_environment_edit_validate'),
+ '#submit' => array('apachesolr_environment_edit_submit'),
+ '#value' => t('Save and edit'),
+ );
$form['actions']['test'] = array(
'#type' => 'submit',
'#validate' => array('apachesolr_environment_edit_validate'),
@@ -168,7 +180,7 @@ function apachesolr_environment_edit_form($form, &$form_state, $environment = NU
$destination = $_GET['destination'];
}
else {
- $destination = 'admin/config/search/apachesolr';
+ $destination = 'admin/config/search/apachesolr/settings';
}
$form['actions']['cancel'] = array(
'#type' => 'link',
@@ -215,11 +227,20 @@ function apachesolr_environment_edit_validate($form, &$form_state) {
function apachesolr_environment_edit_submit($form, &$form_state) {
apachesolr_environment_save($form_state['values']);
if (!empty($form_state['values']['make_default'])) {
- variable_set('apachesolr_default_environment', $form_state['values']['env_id']);
+ apachesolr_set_default_environment($form_state['values']['env_id']);
}
cache_clear_all('apachesolr:environments', 'cache_apachesolr');
drupal_set_message(t('The %name search environment has been saved.', array('%name' => $form_state['values']['name'])));
- $form_state['redirect'] = 'admin/config/search/apachesolr/settings';
+ if ($form_state['values']['op'] == t('Save')) {
+ $form_state['redirect'] = 'admin/config/search/apachesolr/settings';
+ }
+ else {
+ $form_state['redirect'] = current_path();
+ }
+ // Regardlessly of the destination parameter we want to go to another page
+ unset($_GET['destination']);
+ drupal_static_reset('drupal_get_destination');
+ drupal_get_destination();
}
/**
@@ -305,26 +326,40 @@ function apachesolr_settings($form, &$form_state) {
),
);
$env_name = l($data['name'], 'admin/config/search/apachesolr/settings/' . $data['env_id'] . '/edit', array('query' => array('destination' => $_GET['q'])));
+
// Is this row our default environment?
if ($environment_id == $default_environment) {
$env_name = t('!environment <em>(Default)</em>', array('!environment' => $env_name));
$env_class_row = 'default-environment';
- $ops['delete'] = '';
}
else {
$env_class_row = '';
- // For every non-default we add a delete link
- $ops['delete'] = array(
- 'class' => 'operation',
- 'data' => l(t('Delete'), 'admin/config/search/apachesolr/settings/' . $data['env_id'] . '/delete'),
- );
}
+ // For every non-default we add a delete link
+ // Allow to revert a search environment or to delete it
+ $delete_value = '';
+ if (!isset($data['in_code_only'])) {
+ if ((isset($data['type']) && $data['type'] == 'Overridden')) {
+ $delete_value = array(
+ 'class' => 'operation',
+ 'data' => l(t('Revert'), 'admin/config/search/apachesolr/settings/' . $data['env_id'] . '/delete'),
+ );
+ }
+ // don't allow the deletion of the default environment
+ elseif ($environment_id != $default_environment) {
+ $delete_value = array(
+ 'class' => 'operation',
+ 'data' => l(t('Delete'), 'admin/config/search/apachesolr/settings/' . $data['env_id'] . '/delete'),
+ );
+ }
+ }
+ $ops['delete'] = $delete_value;
// When we are receiving a http POST (so the page does not show) we do not
// want to check the statusses of any environment
$class = '';
if (empty($form_state['input'])) {
- $class = apachesolr_server_status($data['url']) ? 'ok' : 'error';
+ $class = apachesolr_server_status($data['url'], $data['service_class']) ? 'ok' : 'error';
}
$headers = array(
@@ -431,7 +466,7 @@ function apachesolr_status_page($environment = NULL) {
}
// Check for availability
- if (!apachesolr_server_status($environment['url'])) {
+ if (!apachesolr_server_status($environment['url'], $environment['service_class'])) {
drupal_set_message(t('The server seems to be unavailable. Please verify the server settings at the <a href="!settings_page">settings page</a>', array('!settings_page' => url("admin/config/search/apachesolr/settings/{$environment['env_id']}/edit", array('query' => drupal_get_destination())))), 'warning');
return '';
}
@@ -457,40 +492,48 @@ function apachesolr_status_page($environment = NULL) {
$status = apachesolr_index_status($environment["env_id"]);
// We need a schema version greater than beta3. This is mostly to catch
// people using the Drupal 6 schema.
- if (version_compare($stats_summary['@schema_version'], 'drupal-3.0-beta3', '<=')) {
- drupal_set_message(t('Your schema.xml version is too old. You must update it and re-index your content.'), 'error');
+ if (preg_match('/^drupal-[13]/', $stats_summary['@schema_version'])) {
+ $minimum = 'drupal-3.0-beta4';
+ if (version_compare($stats_summary['@schema_version'], $minimum, '<')) {
+ drupal_set_message(t('Your schema.xml version is too old. You must update it to at least %minimum and re-index your content.', array('%minimum' => $minimum)), 'error');
+ }
}
$pending_msg = $stats_summary['@pending_docs'] ? t('(@pending_docs sent but not yet processed)', $stats_summary) : '';
- $indexed_message = t('@num Items !pending', array(
+ $index_msg = $stats_summary['@index_size'] ? t('(@index_size on disk)', $stats_summary) : '';
+ $indexed_message = t('@num Items !pending !index_msg', array(
'@num' => $data->index->numDocs,
'!pending' => $pending_msg,
+ '!index_msg' => $index_msg,
));
- $messages[] = array('Indexed', $indexed_message);
+ $messages[] = array(t('Indexed'), $indexed_message);
$remaining_message = t('@items (@percentage% has been sent to the server)', array(
'@items' => format_plural($status['remaining'], t('1 item'), t('@count items')),
'@percentage' => ((int)min(100, 100 * ($status['total'] - $status['remaining']) / max(1, $status['total']))),
)
);
- $messages[] = array('Remaining', $remaining_message);
+ $messages[] = array(t('Remaining'), $remaining_message);
- $messages[] = array('Schema', t('@schema_version', $stats_summary));
+ $messages[] = array(t('Schema'), t('@schema_version', $stats_summary));
if (!empty($stats_summary['@core_name'])) {
- $messages[] = array('Solr Core Name', t('@core_name', $stats_summary));
+ $messages[] = array(t('Solr Core Name'), t('@core_name', $stats_summary));
}
- $messages[] = array('Delay', t('@autocommit_time before updates are processed.', $stats_summary));
- $messages[] = array('Pending Deletions', t('@deletes_total', $stats_summary));
+ $messages[] = array(t('Delay'), t('@autocommit_time before updates are processed.', $stats_summary));
+ $messages[] = array(t('Pending Deletions'), t('@deletes_total', $stats_summary));
}
catch (Exception $e) {
watchdog('Apache Solr', nl2br(check_plain($e->getMessage())), NULL, WATCHDOG_ERROR);
}
}
+ if (empty($messages)) {
+ $messages[] = array(t('Error'), t('No data was returned from the server. Check your log messages.'));
+ }
// Initializes output with information about which server's setting we are
// editing, as it is otherwise not transparent to the end user.
$output['apachesolr_index_action_status'] = array(
'#prefix' => '<h3>' . t('@environment: Search Index Content', array('@environment' => $environment['name'])) . '</h3>',
'#theme' => 'table',
- '#header' => array('type', 'value'),
+ '#header' => array(t('Type'), t('Value')),
'#rows' => $messages,
);
@@ -510,16 +553,31 @@ function apachesolr_status_page($environment = NULL) {
return $output;
}
-function apachesolr_index_report($env_id) {
+function apachesolr_index_report($environment = NULL) {
+ if (!$environment) {
+ $env_id = apachesolr_default_environment();
+ drupal_goto('admin/reports/apachesolr/' . $env_id);
+ }
+ $environments = apachesolr_load_all_environments();
+ $environments_list = array();
+ foreach ($environments as $env) {
+ $var_status = array('!name' =>$env['name']);
+ $environments_list[] = l(t('Statistics for !name', $var_status), 'admin/reports/apachesolr/' . $env['env_id']);
+ }
+ $output['environments_list'] = array(
+ '#theme' => 'item_list',
+ '#items' => $environments_list,
+ );
+
try {
- $solr = apachesolr_get_solr($env_id);
+ $solr = apachesolr_get_solr($environment['env_id']);
$solr->clearCache();
$data = $solr->getLuke();
}
catch (Exception $e) {
watchdog('Apache Solr', nl2br(check_plain($e->getMessage())), NULL, WATCHDOG_ERROR);
drupal_set_message(nl2br(check_plain($e->getMessage())), "warning");
- return '';
+ return $output;
}
$messages = array();
@@ -533,7 +591,7 @@ function apachesolr_index_report($env_id) {
elseif (isset($data->index->numDocs)) {
$not_found = t('Not indexed');
try {
- $solr = apachesolr_get_solr($env_id);
+ $solr = apachesolr_get_solr($environment['env_id']);
// Note: we use 2 since 1 fails on Ubuntu Hardy.
$data = $solr->getLuke(2);
$messages[] = array(t('# of terms in index'), $data->index->numTerms);
@@ -592,10 +650,17 @@ function apachesolr_index_report($env_id) {
/**
* Page callback to show available conf files.
*/
-function apachesolr_config_files_overview() {
+function apachesolr_config_files_overview($environment = NULL) {
+ if (!$environment) {
+ $env_id = apachesolr_default_environment();
+ }
+ else {
+ $env_id = $environment['env_id'];
+ }
+
$xml = NULL;
try {
- $solr = apachesolr_get_solr();
+ $solr = apachesolr_get_solr($env_id);
$response = $solr->makeServletRequest('admin/file', array('wt' => 'xml'));
$xml = simplexml_load_string($response->data);
}
@@ -618,7 +683,7 @@ function apachesolr_config_files_overview() {
// Get data from the files.
$name = $item->attributes();
$name = ((string)$item->attributes()) ? (string)$item->attributes() : t('No name found');
- $files[$item_id]['name'] = l($name, 'admin/reports/apachesolr/conf/' . $name);
+ $files[$item_id]['name'] = l($name, 'admin/reports/apachesolr/' . $env_id . '/conf/' . $name);
// Retrieve the date attribute
if (isset($item->date)) {
@@ -658,10 +723,17 @@ function apachesolr_config_files_overview() {
/**
* Page callback to show one conf file.
*/
-function apachesolr_config_file($name) {
+function apachesolr_config_file($name, $environment = NULL) {
+ if (!$environment) {
+ $env_id = apachesolr_default_environment();
+ }
+ else {
+ $env_id = $environment['env_id'];
+ }
+
$output = '';
try {
- $solr = apachesolr_get_solr();
+ $solr = apachesolr_get_solr($env_id);
$response = $solr->makeServletRequest('admin/file', array('file' => $name));
$raw_file = $response->data;
$output = '<pre>' . check_plain($raw_file) . '</pre>';
@@ -927,6 +999,7 @@ function apachesolr_index_batch_index_entities($env_id, &$context) {
$status = apachesolr_index_status($env_id);
$context['sandbox']['progress'] = 0;
+ $context['sandbox']['submitted'] = 0;
$context['sandbox']['max'] = $status['remaining'];
}
@@ -934,8 +1007,20 @@ function apachesolr_index_batch_index_entities($env_id, &$context) {
// timeout or out of memory error.
$limit = variable_get('apachesolr_cron_limit', 50);
- $context['sandbox']['progress'] += apachesolr_index_entities($env_id, $limit);
- $context['message'] = t('Indexed @current of @total nodes', array('@current' => $context['sandbox']['progress'], '@total' => $context['sandbox']['max']));
+ if ($context['sandbox']['max'] >= $context['sandbox']['progress'] + $limit) {
+ $context['sandbox']['progress'] += $limit;
+ }
+ else {
+ $context['sandbox']['progress'] = $context['sandbox']['max'];
+ }
+ $context['sandbox']['submitted'] += apachesolr_index_entities($env_id, $limit);
+
+ $arguments = array(
+ '@current' => $context['sandbox']['progress'],
+ '@total' => $context['sandbox']['max'],
+ '@submitted' => $context['sandbox']['submitted'],
+ );
+ $context['message'] = t('Inspected @current of @total entities. Submitted @submitted documents to Solr', $arguments);
// Inform the batch engine that we are not finished, and provide an
// estimation of the completion level we reached.
@@ -945,6 +1030,7 @@ function apachesolr_index_batch_index_entities($env_id, &$context) {
// show it to the admin.
if ($context['finished']) {
$context['results']['count'] = $context['sandbox']['progress'];
+ $context['results']['submitted'] = $context['sandbox']['submitted'];
}
}
@@ -955,7 +1041,10 @@ function apachesolr_index_batch_index_finished($success, $results, $operations)
$message = '';
// $results['count'] will not be set if Solr is unavailable.
if (isset($results['count'])) {
- $message .= format_plural($results['count'], '1 item processed successfully.', '@count items successfully processed.');
+ $message .= format_plural($results['count'], '1 item processed successfully. ', '@count items successfully processed. ');
+ }
+ if (isset($results['submitted'])) {
+ $message .= format_plural($results['submitted'], '1 document successfully sent to Solr.', '@count documents successfully sent to Solr.');
}
if ($success) {
$type = 'status';
@@ -963,7 +1052,7 @@ function apachesolr_index_batch_index_finished($success, $results, $operations)
else {
// An error occurred. $operations contains the unprocessed operations.
$error_operation = reset($operations);
- $message .= ' ' . t('An error occurred while processing @num with arguments :', array('@num' => $error_operation[0])) . print_r($error_operation[0], TRUE);
+ $message .= ' ' . t('An error occurred while processing @num with arguments: @args', array('@num' => $error_operation[0], '@args' => print_r($error_operation[0], TRUE)));
$type = 'error';
}
drupal_set_message($message, $type);
@@ -1033,15 +1122,39 @@ function apachesolr_index_config_form_submit($form, &$form_state) {
// even if there is no change so the admin can remove
// bundles if there was an error.
$excluded_bundles = array_diff($all_bundles, $new_bundles);
- apachesolr_index_delete_bundles($env_id, $entity_type, $excluded_bundles);
- $callback = apachesolr_entity_get_callback($entity_type, 'bundles changed callback');
- if (!empty($callback)) {
- call_user_func($callback, $env_id, $existing_bundles, $new_bundles);
+ if (apachesolr_index_delete_bundles($env_id, $entity_type, $excluded_bundles)) {
+ $callback = apachesolr_entity_get_callback($entity_type, 'bundles changed callback');
+ if (!empty($callback)) {
+ call_user_func($callback, $env_id, $existing_bundles, $new_bundles);
+ }
+ }
+ else {
+ drupal_set_message(t('Search is temporarily unavailable. If the problem persists, please contact the site administrator.'), 'error');
}
}
// Clear the entity cache, since we will be changing its data.
entity_info_cache_clear();
-
+ cache_clear_all('apachesolr:environments', 'cache_apachesolr');
drupal_set_message(t('Your settings have been saved.'));
}
+
+/**
+ * Page callback for node/%node/devel/apachesolr.
+ */
+function apachesolr_devel($node) {
+ $item = new stdClass();
+ $item->entity_type = 'node';
+ $item->entity_id = $node->nid;
+ module_load_include('inc', 'apachesolr', 'apachesolr.index');
+ $documents = apachesolr_index_entity_to_documents($item, apachesolr_default_environment());
+ $output = ' ';
+ foreach ($documents as $document) {
+ $debug_data = array();
+ foreach ($document as $key => $value) {
+ $debug_data[$key] = $value;
+ }
+ $output .= kdevel_print_object($debug_data);
+ }
+ return $output;
+}
View
89 sites/all/modules/contrib/apachesolr/apachesolr.api.php 100644 → 100755
@@ -4,6 +4,16 @@
* Exposed Hooks in 7.x:
*/
+/**
+ * Lets modules know when the default environment is changed.
+ */
+function hook_apachesolr_default_environment($env_id, $old_env_id) {
+ $page = apachesolr_search_page_load('core_search');
+ if ($page && $page['env_id'] != $env_id) {
+ $page['env_id'] = $env_id;
+ apachesolr_search_page_save($page);
+ }
+}
/**
* Add index mappings for Field API types. The default mappings array
@@ -172,7 +182,13 @@ function hook_apachesolr_field_name_map_alter(&$map) {
* An object implementing DrupalSolrQueryInterface. No need for &.
*/
function hook_apachesolr_query_alter($query) {
- // I only want to see articles by the admin!
+ // I only want to see articles by the admin.
+ //
+ // NOTE: this "is_uid" filter does NOT refer to the English word "is"
+ // It is a combination of flags representing Integer-Single, which is
+ // abbreviated with the letters i and s.
+ //
+ // @see the <dynamicField> definitions in schema.xml or schema-solr3.xml
$query->addFilter("is_uid", 1);
// Only search titles.
@@ -196,14 +212,54 @@ function hook_apachesolr_delete_by_query_alter($query) {
}
/**
- * Add information to index other entities.
- * There are some modules in http://drupal.org that can give a good example of
- * custom entity indexing such as apachesolr_user_indexer, apachesolr_term
- *
* This is the place to look for the replacement to hook_apachesolr_node_exclude
* You should define a replacement for the status callback and return
- * 0 for nodes which you do not want to appear in the index and 1 for nodes
- * that you do. See http://drupal.org/node/1474906 for an example.
+ * FALSE for entities which you do not want to appear in the index and TRUE for
+ * those that you want to include
+ */
+
+/**
+ * This is invoked for each entity that is being inspected to be added to the
+ * index. if any module returns TRUE, the entity is skipped for indexing.
+ *
+ * @param integer $entity_id
+ * @param string $entity_type
+ * @param integer $row
+ * A complete set of data from the indexing table.
+ * @param string $env_id
+ * @return boolean
+ */
+function hook_apachesolr_exclude($entity_id, $entity_type, $row, $env_id) {
+ // Never index media entities to core_1
+ if ($entity_type == 'media' && $env_id == 'core_1') {
+ return TRUE;
+ }
+ return FALSE;
+}
+
+/**
+ * This is invoked for each entity from the type of ENTITY_TYPE that is being
+ * inspected to be added to the index. if any module returns TRUE, 
+ * the entity is skipped for indexing.
+ *
+ * @param integer $entity_id
+ * @param integer $row
+ * A complete set of data from the indexing table.
+ * @param string $env_id
+ * @return boolean
+ */
+function hook_apachesolr_ENTITY_TYPE_exclude($entity_id, $row, $env_id) {
+ // Never index ENTITY_TYPE to core_1
+ if ($env_id == 'core_1') {
+ return TRUE;
+ }
+ return FALSE;
+}
+
+/**
+ * Add information to index other entities.
+ * There are some modules in http://drupal.org that can give a good example of
+ * custom entity indexing such as apachesolr_user, apachesolr_term
*
* @param array $entity_info
*/
@@ -230,6 +286,13 @@ function hook_apachesolr_entity_info_alter(&$entity_info) {
$entity_info['node']['cron_check'] = 'apachesolr_index_node_check_table';
// Specific output processing for the results
$entity_info['node']['apachesolr']['result callback'] = 'apachesolr_search_node_result';
+
+ // BUNDLE SPECIFIC OVERRIDES
+ // The following can be overridden on a per-bundle basis.
+ // The bundle-specific settings will take precedence over the entity settings.
+ $entity_info['node']['bundles']['page']['apachesolr']['result callback'] = 'apachesolr_search_node_result';
+ $entity_info['node']['bundles']['page']['apachesolr']['status callback'][] = 'apachesolr_index_node_status_callback';
+ $entity_info['node']['bundles']['page']['apachesolr']['document callback'][] = 'apachesolr_index_node_solr_document';
}
@@ -325,12 +388,14 @@ function hook_apachesolr_index_document_build(ApacheSolrDocument $document, $ent
* @param $entity_type
*/
function hook_apachesolr_index_document_build_ENTITY_TYPE(ApacheSolrDocument $document, $entity, $env_id) {
- // Index book module data.
- if (!empty($entity->book['bid'])) {
- // Hard-coded - must change if apachesolr_index_key() changes.
- $document->is_book_bid = (int) $entity->book['bid'];
+ // Index field_main_image as a separate field
+ if ($entity->type == 'profile') {
+ $user = user_load(array('uid' => $entity->uid));
+ // Hard coded field, not recommended for inexperienced users.
+ $document->setMultiValue('sm_field_main_image', $user->picture);
}
}
+
/**
* Alter the prepared documents from one entity before sending them to Solr.
*
@@ -342,4 +407,4 @@ function hook_apachesolr_index_document_build_ENTITY_TYPE(ApacheSolrDocument $do
*/
function hook_apachesolr_index_documents_alter(array &$documents, $entity, $entity_type, $env_id) {
// Do whatever altering you need here
-}
+}
View
477 sites/all/modules/contrib/apachesolr/apachesolr.index.inc
@@ -2,33 +2,56 @@
/**
* @file
- * Functions used when indexing content to Apache Solr.
+ * Functions related to Apache Solr indexing operations.
*/
-
/**
- * Send up to $limit entities of each type into the index.
+ * Processes all index queues associated with the passed environment.
+ *
+ * An environment usually indexes one or more entity types. Each entity type
+ * stores its queue in a database table that is defined in the entity type's
+ * info array. This function processes N number of items in each queue table,
+ * where N is the limit passed as the second argument.
+ *
+ * The indexing routine allows developers to selectively bypass indexing on a
+ * per-entity basis by implementing the following hooks:
+ * - hook_apachesolr_exclude()
+ * - hook_apachesolr_ENTITY_TYPE_exclude()
+ *
+ * @param string $env_id
+ * The machine name of the environment.
+ * @param int $limit
+ * The number of items to process per queue table. For example, if there are
+ * two entities that are being indexed in this environment and they each have
+ * their own queue table, setting a limit of 50 will send a maximum number of
+ * 100 documents to the Apache Solr server.
+ *
+ * @return int
+ * The total numer of documents sent to the Apache Solr server for indexing.
+ *
+ * @see apachesolr_index_get_entities_to_index()
+ * @see apachesolr_index_entity_to_documents()
+ * @see apachesolr_index_send_to_solr()
*/
function apachesolr_index_entities($env_id, $limit) {
- $entities_processed = 0;
+ $documents_submitted = 0;
foreach (entity_get_info() as $entity_type => $info) {
// With each pass through the callback, retrieve the next group of nids.
$rows = apachesolr_index_get_entities_to_index($env_id, $entity_type, $limit);
$documents = array();
foreach ($rows as $row) {
- if (!empty($row)) {
- $documents = array_merge($documents, apachesolr_index_entity_to_documents($row, $env_id));
- }
+ $row_documents = apachesolr_index_entities_document($row, $entity_type, $env_id);
+ $documents = array_merge($documents, $row_documents);
}
$indexed = apachesolr_index_send_to_solr($env_id, $documents);
if ($indexed !== FALSE) {
- $entities_processed += count($rows);
+ $documents_submitted += count($documents);
$index_position = apachesolr_get_last_index_position($env_id, $entity_type);
$max_changed = $index_position['last_changed'];
$max_entity_id = $index_position['last_entity_id'];
foreach ($rows as $row) {
- if (!empty($row)) {
+ if (!empty($row->status)) {
if ($row->changed > $max_changed) {
$max_changed = $row->changed;
}
@@ -41,12 +64,53 @@ function apachesolr_index_entities($env_id, $limit) {
apachesolr_set_last_index_updated($env_id, REQUEST_TIME);
}
}
-
- return $entities_processed;
+ return $documents_submitted;
}
+function apachesolr_index_entities_document($row, $entity_type, $env_id) {
+ $documents = array();
+ if (!empty($row->status)) {
+ // Let any module exclude this entity from the index.
+ $build_document = TRUE;
+ foreach (module_implements('apachesolr_exclude') as $module) {
+ $exclude = module_invoke($module, 'apachesolr_exclude', $row->entity_id, $entity_type, $row, $env_id);
+ // If the hook returns TRUE we should exclude the entity
+ if (!empty($exclude)) {
+ $build_document = FALSE;
+ }
+ }
+ foreach (module_implements('apachesolr_' . $entity_type . '_exclude') as $module) {
+ $exclude = module_invoke($module, 'apachesolr_' . $entity_type . '_exclude', $row->entity_id, $row, $env_id);
+ // If the hook returns TRUE we should exclude the entity
+ if (!empty($exclude)) {
+ $build_document = FALSE;
+ }
+ }
+ if ($build_document) {
+ $documents = array_merge($documents, apachesolr_index_entity_to_documents($row, $env_id));
+ }
+ }
+ else {
+ // Delete the entity from our index if the status callback returned 0
+ apachesolr_remove_entity($env_id, $row->entity_type, $row->entity_id);
+ }
+ return $documents;
+}
/**
- * Helper function for modules implementing hook_search's 'status' op.
+ * Returns the total number of documents that are able to be indexed and the
+ * number of documents left to be indexed.
+ *
+ * This is a helper function for modules that implement hook_search_status().
+ *
+ * @param string $env_id
+ * The machine name of the environment.
+ *
+ * @return array
+ * An associative array with the key-value pairs:
+ * - remaining: The number of items left to index.
+ * - total: The total number of items to index.
+ *
+ * @see hook_search_status()
*/
function apachesolr_index_status($env_id) {
$remaining = 0;
@@ -68,8 +132,8 @@ function apachesolr_index_status($env_id) {
// for ordering we're grabbing the oldest first and then ordering by ID so
// that we get a definitive order.
$query = db_select($table, 'aie')
- ->condition('aie.status', 1)
->condition('aie.bundle', $bundles)
+ ->condition('aie.status', 1)
->condition(db_or()
->condition('aie.changed', $last_changed, '>')
->condition(db_and()
@@ -88,10 +152,43 @@ function apachesolr_index_status($env_id) {
}
/**
- * Worker callback for apachesolr_index_entities.
+ * Worker callback for apachesolr_index_entities().
+ *
+ * Loads and proccesses the entity queued for indexing and converts into one or
+ * more documents that are sent to the Apache Solr server for indexing.
+ *
+ * The entity is loaded as the user specified in the "apachesolr_index_user"
+ * system variable in order to prevent sentive data from being indexed and
+ * displayed to underprivileged users in search results. The index user defaults
+ * to a user ID of "0", which is the anonymous user.
+ *
+ * After the entity is loaded, it is converted to an array via the callback
+ * specified in the entity type's info array. The array that the entity is
+ * converted to is the model of the document sent to the Apache Solr server for
+ * indexing. This function allows develoeprs to modify the document by
+ * implementing the following hooks:
+ * - apachesolr_index_document_build()
+ * - apachesolr_index_document_build_ENTITY_TYPE()
+ * - apachesolr_index_documents_alter()
+ *
+ * @param stdClass $item
+ * The data returned by the queue table containing:
+ * - entity_id: An integer containing the unique identifier of the entity, for
+ * example a node ID or comment ID.
+ * - entity_type: The unique identifier for the entity, i.e. "node", "file".
+ * - bundle: The machine-readable name of the bundle the passed entity is
+ * associated with.
+ * - status: The "published" status of the entity. The status will also be set
+ * to "0" when entity is deleted but the Apache Solr server is unavailable.
+ * - changed: A timestamp flagging when the entity was last modified.
+ * @param string $env_id
+ * The machine name of the environment.
+ *
+ * @return array
+ * An associative array of documents that are sent to the Apache Solr server
+ * for indexing.
*
* @see apachesolr_index_nodes() for the old-skool version.
- * @return array of documents that need to be sent to the index of solr
*/
function apachesolr_index_entity_to_documents($item, $env_id) {
@@ -104,7 +201,7 @@ function apachesolr_index_entity_to_documents($item, $env_id) {
// Should indexing take place using anon ( default )
// or as another user
$uid = variable_get('apachesolr_index_user', 0);
- if($uid == 0) {
+ if ($uid == 0) {
$user = drupal_anonymous_user();
}
else {
@@ -128,7 +225,7 @@ function apachesolr_index_entity_to_documents($item, $env_id) {
$document = _apachesolr_index_process_entity_get_document($entity, $entity_type);
//Get the callback array to add stuff to the document
- $callbacks = apachesolr_entity_get_callback($entity_type, 'document callback');
+ $callbacks = apachesolr_entity_get_callback($entity_type, 'document callback', $bundle);
$documents = array();
foreach ($callbacks as $callback) {
// Call a type-specific callback to add stuff to the document.
@@ -167,7 +264,12 @@ function apachesolr_index_entity_to_documents($item, $env_id) {
}
// Now allow modules to alter each other's additions for maximum flexibility.
- drupal_alter('apachesolr_index_documents', $documents, $entity, $entity_type, $env_id);
+
+ // Hook to allow modifications of the retrieved results
+ foreach (module_implements('apachesolr_index_documents_alter') as $module) {
+ $function = $module . '_apachesolr_index_documents_alter';
+ $function($documents, $entity, $entity_type, $env_id);
+ }
// Restore the user.
$user = $saved_user;
@@ -182,11 +284,6 @@ function apachesolr_index_entity_to_documents($item, $env_id) {
* @return number indexed, or FALSE on failure.
*/
function apachesolr_index_send_to_solr($env_id, $documents) {
- // Do not index when we do not have any documents to send
- if (empty($documents)) {
- return FALSE;
- }
-
try {
// Get the $solr object
$solr = apachesolr_get_solr($env_id);
@@ -199,7 +296,11 @@ function apachesolr_index_send_to_solr($env_id, $documents) {
watchdog('Apache Solr', nl2br(check_plain($e->getMessage())), NULL, WATCHDOG_ERROR);
return FALSE;
}
-
+ // Do not index when we do not have any documents to send
+ // Send TRUE because this is not an error
+ if (empty($documents)) {
+ return TRUE;
+ }
// Send the document off to Solr.
watchdog('Apache Solr', 'Adding @count documents.', array('@count' => count($documents)));
try {
@@ -231,7 +332,7 @@ function apachesolr_index_send_to_solr($env_id, $documents) {
*
* $text must be stripped of control characters before hand.
*/
-function apachesolr_index_add_tags_to_document(&$document, $text) {
+function apachesolr_index_add_tags_to_document($document, $text) {
$tags_to_index = variable_get('apachesolr_tags_to_index', array(
'h1' => 'tags_h1',
'h2' => 'tags_h2_h3',
@@ -281,8 +382,16 @@ function _apachesolr_index_process_entity_get_document($entity, $entity_type) {
$document = new ApacheSolrDocument();
+ // Define our url options in advance. This differs depending on the
+ // language
+ $languages = language_list();
+ $url_options = array('absolute' => TRUE);
+ if (isset($entity->language) && isset($languages[$entity->language])) {
+ $url_options = $url_options + array('language' => $languages[$entity->language]);
+ }
+
$document->id = apachesolr_document_id($entity_id, $entity_type);
- $document->site = url(NULL, array('absolute' => TRUE));
+ $document->site = url(NULL, $url_options);
$document->hash = apachesolr_site_hash();
$document->entity_id = $entity_id;
@@ -290,12 +399,6 @@ function _apachesolr_index_process_entity_get_document($entity, $entity_type) {
$document->bundle = $bundle;
$document->bundle_name = entity_bundle_label($entity_type, $bundle);
- $path = entity_uri($entity_type, $entity);
- // A path is not a requirement of an entity
- if (!empty($path)) {
- $document->path = $path['path'];
- $document->url = url($path['path'], $path['options'] + array('absolute' => TRUE));
- }
if (empty($entity->language)) {
// 'und' is the language-neutral code in Drupal 7.
$document->language = LANGUAGE_NONE;
@@ -304,14 +407,20 @@ function _apachesolr_index_process_entity_get_document($entity, $entity_type) {
$document->language = $entity->language;
}
- // Path aliases can have important information about the content.
- // Add them to the index as well.
- if (function_exists('drupal_get_path_alias')) {
- // Add any path alias to the index, looking first for language specific
- // aliases but using language neutral aliases otherwise.
- $output = drupal_get_path_alias($document->path, $document->language);
- if ($output && $output != $document->path) {
- $document->path_alias = $output;
+ $path = entity_uri($entity_type, $entity);
+ // A path is not a requirement of an entity
+ if (!empty($path)) {
+ $document->path = $path['path'];
+ $document->url = url($path['path'], $path['options'] + $url_options);
+ // Path aliases can have important information about the content.
+ // Add them to the index as well.
+ if (function_exists('drupal_get_path_alias')) {
+ // Add any path alias to the index, looking first for language specific
+ // aliases but using language neutral aliases otherwise.
+ $output = drupal_get_path_alias($document->path, $document->language);
+ if ($output && $output != $document->path) {
+ $document->path_alias = $output;
+ }
}
}
return $document;
@@ -337,9 +446,9 @@ function apachesolr_index_get_entities_to_index($env_id, $entity_type, $limit) {
// Find the next batch of entities to index for this entity type. Note that
// for ordering we're grabbing the oldest first and then ordering by ID so
// that we get a definitive order.
+ // Also note that we fetch ALL fields from the indexer table
$query = db_select($table, 'aie')
- ->fields('aie', array('entity_type', 'entity_id', 'changed'))
- ->condition('aie.status', 1)
+ ->fields('aie')
->condition('aie.bundle', $bundles)
->condition(db_or()
->condition('aie.changed', $last_changed, '>')
@@ -357,55 +466,62 @@ function apachesolr_index_get_entities_to_index($env_id, $entity_type, $limit) {
$status_callbacks = apachesolr_entity_get_callback($entity_type, 'status callback');
foreach ($records as $record) {
- // Check status callback before sending to the index
- $status = TRUE;
- foreach($status_callbacks as $status_callback) {
- if (is_callable($status_callback)) {
- // by placing $status in front we prevent calling any other callback
- // after one status callback returned false
- $status = $status && $status_callback($record->entity_id, $record->entity_type);
- }
- }
-
- // Delete the entity from our index if the status callback returns 0
- if ($status == TRUE) {
- $rows[] = $record;
- }
- else {
- // Discard this entry
- $rows[] = NULL;
- // Delete if from the index - Do not use apachesolr_entity_delete because
- // the module does not want to load a complete entity
- if (apachesolr_index_delete_entity_from_index($env_id, $record->entity_type, $record->entity_id)) {
- // There was no exception, so delete from the table.
- db_delete($table)
- ->condition('entity_type', $entity_type)
- ->condition('entity_id', $id)
- ->execute();
+ // Check status and status callbacks before sending to the index
+ if (is_array($status_callbacks)) {
+ foreach($status_callbacks as $status_callback) {
+ if (is_callable($status_callback)) {
+ // by placing $status in front we prevent calling any other callback
+ // after one status callback returned false
+ $record->status = $record->status && $status_callback($record->entity_id, $record->entity_type);
+ }
}
}
+ $rows[] = $record;
}
return $rows;
}
/**
- * Delete an index from an environment and/or a specific entity type
- * @param String $env_id
- * @param String $type
+ * Delete the whole index for an environment.
+ *
+ * @param string $env_id
+ * The solr environment indentifier.
+ * @param string $entity_type
+ * (optional) specify to remove just this entity_type from the index.
+ * @param string $bundle
+ * (optional) also specify a bundle to remove just the bundle from
+ * the index.
*/
-function apachesolr_index_delete_index($env_id) {
+function apachesolr_index_delete_index($env_id, $entity_type = NULL, $bundle = NULL) {
// Instantiate a new Solr object.
try {
$solr = apachesolr_get_solr($env_id);
$query = '*:*';
+ if (!empty($entity_type) && !empty($bundle)) {
+ $query = "(bundle:$bundle AND entity_type:$entity_type) OR sm_parent_entity_bundle:{$entity_type}-{$bundle}";
+ }
+ elseif (!empty($bundle)) {
+ $query = "(bundle:$bundle)";
+ }
+
// Allow other modules to modify the delete query.
// For example, use the site hash so that you only delete this site's
// content: $query = 'hash:' . apachesolr_site_hash()
drupal_alter('apachesolr_delete_by_query', $query);
$solr->deleteByQuery($query);
$solr->commit();
- apachesolr_clear_last_index_position($env_id);
+
+ if (!empty($entity_type)) {
+ $rebuild_callback = apachesolr_entity_get_callback($entity_type, 'reindex callback');
+ if (is_callable($rebuild_callback)) {
+ $rebuild_callback($env_id, $bundle);
+ }
+ }
+ else {
+ apachesolr_index_mark_for_reindex($env_id);
+ }
+
apachesolr_set_last_index_updated($env_id, REQUEST_TIME);
}
catch (Exception $e) {
@@ -413,12 +529,24 @@ function apachesolr_index_delete_index($env_id) {
}
}
-function apachesolr_index_delete_bundles($env_id, $entity_type, $excluded_bundles) {
+/**
+ * Delete from the index documents with the entity type and any of the excluded bundles.
+ *
+ * Also deletes all documents that have the entity type and bundle as a parent.
+ *
+ * @param string $env_id
+ * @param string $entity_type
+ * @param array $excluded_bundles
+ *
+ * @return TRUE on success, FALSE on failure.
+ */
+function apachesolr_index_delete_bundles($env_id, $entity_type, array $excluded_bundles) {
// Remove newly omitted bundles.
try {
$solr = apachesolr_get_solr($env_id);
foreach ($excluded_bundles as $bundle) {
- $query = "bundle:$bundle AND entity_type:$entity_type";
+ $query = "(bundle:$bundle AND entity_type:$entity_type) OR sm_parent_entity_bundle:{$entity_type}-{$bundle}";
+
// Allow other modules to modify the delete query.
// For example, use the site hash so that you only delete this site's
// content: $query = 'hash:' . apachesolr_site_hash()
@@ -428,15 +556,24 @@ function apachesolr_index_delete_bundles($env_id, $entity_type, $excluded_bundle
if ($excluded_bundles) {
$solr->commit();
}
+ return TRUE;
}
catch (Exception $e) {
watchdog('Apache Solr', nl2br(check_plain($e->getMessage())), NULL, WATCHDOG_ERROR);
- drupal_set_message(t('The Apache Solr search engine is not available. Please contact your site administrator.'), 'error');
+ return FALSE;
}
}
/**
- * Delete an entity from the indexer.
+ * Delete an entity from the index.
+ *
+ * Also deletes all documents that have the deleted document as a parent.
+ *
+ * @param string $env_id
+ * @param string $entity_type
+ * @param string $entity_id
+ *
+ * @return TRUE on success, FALSE on failure.
*/
function apachesolr_index_delete_entity_from_index($env_id, $entity_type, $entity_id) {
static $failed = FALSE;
@@ -445,7 +582,9 @@ function apachesolr_index_delete_entity_from_index($env_id, $entity_type, $entit
}
try {
$solr = apachesolr_get_solr($env_id);
- $solr->deleteById(apachesolr_document_id($entity_id, $entity_type));
+ $document_id = apachesolr_document_id($entity_id, $entity_type);
+ $query = "id:$document_id OR sm_parent_document_id:$document_id";
+ $solr->deleteByQuery($query);
apachesolr_set_last_index_updated($env_id, REQUEST_TIME);
return TRUE;
}
@@ -458,7 +597,7 @@ function apachesolr_index_delete_entity_from_index($env_id, $entity_type, $entit
}
/**
- * @param $entity_id
+ * @param $entity_type
*
* @throws Exception
*/
@@ -477,7 +616,6 @@ function apachesolr_index_mark_for_reindex($env_id, $entity_type = NULL) {
cache_clear_all('*', 'cache_apachesolr', TRUE);
}
-
/**
* Sets what bundles on the specified entity type should be indexed.
*
@@ -571,17 +709,20 @@ function apachesolr_index_node_solr_document(ApacheSolrDocument $document, $node
$document->label = apachesolr_clean_text($node->title);
// Build the node body.
- $build = node_view($node, 'search_index');
- // Why do we need this?
+ $build = node_view($node, 'search_index', !empty($node->language) ? $node->language : LANGUAGE_NONE);
+ // Remove useless html crap out of the render.
unset($build['#theme']);
$text = drupal_render($build);
$document->content = apachesolr_clean_text($text);
+
+ // Adding the teaser
if (isset($node->teaser)) {
$document->teaser = apachesolr_clean_text($node->teaser);
}
else {
$document->teaser = truncate_utf8($document->content, 300, TRUE);
}
+
// Path aliases can have important information about the content.
// Add them to the index as well.
if (function_exists('drupal_get_path_alias')) {
@@ -595,6 +736,7 @@ function apachesolr_index_node_solr_document(ApacheSolrDocument $document, $node
}
}
+ // Author information
$document->ss_name = $node->name;
// We want the name to be searchable for keywords.
$document->tos_name = $node->name;
@@ -604,14 +746,14 @@ function apachesolr_index_node_solr_document(ApacheSolrDocument $document, $node
$username = format_username($account);
$document->ss_name_formatted = $username;
$document->tos_name_formatted = $username;
-
- // Everything else uses dynamic fields
$document->is_uid = $node->uid;
$document->bs_status = $node->status;
$document->bs_sticky = $node->sticky;
$document->bs_promote = $node->promote;
$document->is_tnid = $node->tnid;
$document->bs_translate = $node->translate;
+
+ // Language specific checks
if (empty($node->language)) {
// 'und' is the language-neutral code in Drupal 7.
$document->ss_language = LANGUAGE_NONE;
@@ -619,46 +761,29 @@ function apachesolr_index_node_solr_document(ApacheSolrDocument $document, $node
else {
$document->ss_language = $node->language;
}
+
+ // Timestamp of the node
$document->ds_created = apachesolr_date_iso($node->created);
$document->ds_changed = apachesolr_date_iso($node->changed);
+
+ // Comment counts + time
if (isset($node->last_comment_timestamp) && !empty($node->comment_count)) {
$document->ds_last_comment_timestamp = apachesolr_date_iso($node->last_comment_timestamp);
$document->ds_last_comment_or_change = apachesolr_date_iso(max($node->last_comment_timestamp, $node->changed));
+ $document->is_comment_count = $node->comment_count;
}
else {
$document->ds_last_comment_or_change = apachesolr_date_iso($node->changed);
}
- $document->is_comment_count = isset($node->comment_count) ? $node->comment_count : 0;
// Fetch extra data normally not visible, including comments.
// We do this manually (with module_implements instead of node_invoke_nodeapi)
// because we want a keyed array to come back. Only in this way can we decide
// whether to index comments or not.
$extra = array();
- $exclude_comments = in_array($node->type, variable_get('apachesolr_exclude_comments_types', array()), TRUE);
-
- if (!empty($extra)) {
- // Use an omit-norms text field since this is generally going to be short; not
- // really a full-text field.
- $document->tos_content_extra = apachesolr_clean_text(implode(' ', $extra));
- }
-
- $document->type_name = node_type_get_name($node);
- $document->created = apachesolr_date_iso($node->created);
- $document->changed = apachesolr_date_iso($node->changed);
- $last_change = (isset($node->last_comment_timestamp) && $node->last_comment_timestamp > $node->changed) ? $node->last_comment_timestamp : $node->changed;
- $document->last_comment_or_change = apachesolr_date_iso($last_change);
- $document->comment_count = isset($node->comment_count) ? $node->comment_count : 0;
+ $excludes = variable_get('apachesolr_exclude_nodeapi_types', array());
+ $exclude_nodeapi = isset($excludes[$node->type]) ? $excludes[$node->type] : array();
- // We need to get the real username here, since it needs a full user object.
- // That means we can't do the format_username() call on the display side.
- $document->name = format_username(user_load($node->uid));
-
- // Fetch extra data normally not visible, including comments.
- // We do this manually (with module_implements instead of node_invoke_nodeapi)
- // because we want a keyed array to come back. Only in this way can we decide
- // whether to index comments or not.
- $extra = array();
foreach (module_implements('node_update_index') as $module) {
// Invoke nodeapi if this module has not been excluded, for example,
// exclude 'comment' for a type to skip indexing its comments.
@@ -667,17 +792,23 @@ function apachesolr_index_node_solr_document(ApacheSolrDocument $document, $node
if ($output = $function($node)) {
$extra[$module] = $output;
}
+ }
}
- }
+
+ // Adding the text of the comments
if (isset($extra['comment'])) {
$comments = $extra['comment'];
+ // Remove comments from the extra fields
unset($extra['comment']);
$document->ts_comments = apachesolr_clean_text($comments);
// @todo: do we want to reproduce apachesolr_add_tags_to_document() for comments?
}
- // Use an omit-norms text field since this is generally going to be short; not
- // really a full-text field.
- $document->tos_content_extra = apachesolr_clean_text(implode(' ', $extra));
+ // If there are other extra fields, add them to the document
+ if (!empty($extra)) {
+ // Use an omit-norms text field since this is generally going to be short; not
+ // really a full-text field.
+ $document->tos_content_extra = apachesolr_clean_text(implode(' ', $extra));
+ }
// Generic usecase for future reference. Callbacks can
// allow you to send back multiple documents
@@ -688,59 +819,59 @@ function apachesolr_index_node_solr_document(ApacheSolrDocument $document, $node
function apachesolr_index_node_bundles_changed($env_id, $existing_bundles, $new_bundles) {
- $removed_bundles = array_diff($existing_bundles, $new_bundles);
- $added_bundles = array_diff($new_bundles, $existing_bundles);
- $indexer_table = apachesolr_get_indexer_table('node');
- $transaction = db_transaction();
- try {
- if ($removed_bundles) {
- db_delete($indexer_table)
- ->condition('entity_type', 'node')
- ->condition('bundle', $removed_bundles)
- ->execute();
- }
- if ($added_bundles) {
- $select = db_select('node', 'n');
- $select->addExpression("'node'", 'entity_type');
- $select->condition('type', $added_bundles);
- $select->addField('n', 'nid', 'entity_id');
- $select->addField('n', 'type', 'bundle');
- $select->addField('n', 'status', 'status');
- $select->addExpression(REQUEST_TIME, 'changed');
-
- $insert = db_insert($indexer_table)
- ->fields(array('entity_id', 'bundle', 'status', 'entity_type', 'changed'))
- ->from($select)
- ->execute();
- }
- }
- catch (Exception $e) {
- $transaction->rollback();
- throw $e;
- }
+ // Nothing to do for now.
}
/**
* Reindexing callback for ApacheSolr, for nodes.
*
- * @param String $env_id
+ * @param string $env_id
+ * The solr environment
+ * @param string|null $bundle
+ * (optional) The bundle type to reindex. If not used
+ * all bundles will be reindexed.
*
* @throws Exception
*/
-function apachesolr_index_node_solr_reindex($env_id) {
+function apachesolr_index_node_solr_reindex($env_id, $bundle = NULL) {
$indexer_table = apachesolr_get_indexer_table('node');
$transaction = db_transaction();
try {
- db_delete($indexer_table)
- ->condition('entity_type', 'node')
- ->execute();
+ // Leave status 0 rows - those need to be
+ // removed from the index later.
+ $delete = db_delete($indexer_table);
+ $delete->condition('status', 1);
+
+ if (!empty($bundle)) {
+ $delete->condition('bundle', $bundle);
+ }
+
+ $delete->execute();
+
+ $indexable_bundles = apachesolr_get_index_bundles($env_id, 'node');
+
+ if ($bundle && !empty($indexable_bundles) && !in_array($bundle, $indexable_bundles)) {
+ // The bundle specified is not in the indexable bundles list.
+ return;
+ }
+
$select = db_select('node', 'n');
+ $select->condition('status', 1);
$select->addExpression("'node'", 'entity_type');
$select->addField('n', 'nid', 'entity_id');
$select->addField('n', 'type', 'bundle');
$select->addField('n', 'status', 'status');
$select->addExpression(REQUEST_TIME, 'changed');
+ if ($bundle) {
+ // Mark all nodes of the specified content type for reindexing.
+ $select->condition('n.type', $bundle);
+ }
+ elseif (!empty($indexable_bundles)) {
+ // Restrict reindex to content types in the indexable bundles list.
+ $select->condition('n.type', $indexable_bundles, 'IN');
+ }
+
$insert = db_insert($indexer_table)
->fields(array('entity_id', 'bundle', 'status', 'entity_type', 'changed'))
->from($select)
@@ -880,17 +1011,16 @@ function apachesolr_fields_default_indexing_callback($entity, $field_name, $inde
'key' => $index_key,
'value' => $function($values[$i]['value']),
);
-
- // Only store the first value of the field in a singular index
- if ($numeric && ($i == 0)) {
- $singular_field_info = $field_info;
- $singular_field_info['multiple'] = FALSE;
- $single_key = apachesolr_index_key($singular_field_info);
- $fields[] = array(
- 'key' => $single_key,
- 'value' => $function($values[$i]['value']),
- );
- }
+ }
+ // Also store the first value of the field in a singular index for multi value fields
+ if ($field_info['multiple'] && $numeric && !empty($values[0])) {
+ $singular_field_info = $field_info;
+ $singular_field_info['multiple'] = FALSE;
+ $single_key = apachesolr_index_key($singular_field_info);
+ $fields[] = array(
+ 'key' => $single_key,
+ 'value' => $function($values[0]['value']),
+ );
}
}
return $fields;
@@ -1018,6 +1148,33 @@ function apachesolr_userreference_indexing_callback($entity, $field_name, $index
}
/**
+ * Indexing callback for entityreference fields.
+ */
+function apachesolr_entityreference_indexing_callback($entity, $field_name, $index_key, $field_info) {
+ $fields = array();
+ if (!empty($entity->{$field_name})) {
+
+ // Gets entity type and index key. We need to prefix the ID with the entity
+ // type so we know what entity we are dealing with in the mapping callback.
+ $entity_type = $field_info['field']['settings']['target_type'];
+ $index_key = apachesolr_index_key($field_info);
+
+ // Iterates over all references and adds them to the fields.
+ foreach ($entity->$field_name as $entity_references) {
+ foreach ($entity_references as $reference) {
+ if ($id = (!empty($reference['target_id'])) ? $reference['target_id'] : FALSE) {
+ $fields[] = array(
+ 'key' => $index_key,
+ 'value' => $entity_type . ':' . $id,
+ );
+ }
+ }
+ }
+ }
+ return $fields;
+}
+
+/**
* Extract HTML tag contents from $text and add to boost fields.
*
* $text must be stripped of control characters before hand.
@@ -1076,7 +1233,7 @@ function apachesolr_index_node_check_table() {
$node_lists = array_chunk($nodes, $limit, TRUE);
foreach ($node_lists as $nodes) {
- watchdog('Apache Solr', 'On cron running apachesolr_nodeapi_mass_update() on nids @nids', array('@nids' => implode(',', array_keys($nodes))), WATCHDOG_WARNING);
+ watchdog('Apache Solr', 'On cron running apachesolr_nodeapi_mass_update() on nids @nids', array('@nids' => implode(',', array_keys($nodes))), WATCHDOG_NOTICE);
if (!apachesolr_index_nodeapi_mass_update($nodes, $table)) {
// Solr query failed - so stop trying.
break;
@@ -1085,15 +1242,15 @@ function apachesolr_index_node_check_table() {
// Check for deleted content that wasn't deleted from the index.
$query = db_select($table, 'aien')
- ->fields('aien', array('entity_id'))
->isNull('n.nid')
->range(0, ($limit*2));
+ $query->addExpression('aien.entity_id', 'nid');
$query->leftJoin('node', 'n', 'n.nid = aien.entity_id');
$nodes = $query->execute()->fetchAllAssoc('nid');
$node_lists = array_chunk($nodes, $limit, TRUE);
foreach ($node_lists as $nodes) {
- watchdog('Apache Solr', 'On cron running apachesolr_nodeapi_mass_delete() on nids @nids', array('@nids' => implode(',', array_keys($nodes))), WATCHDOG_WARNING);
+ watchdog('Apache Solr', 'On cron running apachesolr_nodeapi_mass_delete() on nids @nids', array('@nids' => implode(',', array_keys($nodes))), WATCHDOG_NOTICE);
if (!apachesolr_index_nodeapi_mass_delete($nodes, $table)) {
// Solr query failed - so stop trying.
break;
View
7 sites/all/modules/contrib/apachesolr/apachesolr.info
@@ -16,6 +16,7 @@ files[] = plugins/facetapi/adapter.inc
files[] = plugins/facetapi/query_type_date.inc
files[] = plugins/facetapi/query_type_term.inc
files[] = plugins/facetapi/query_type_numeric_range.inc
+files[] = plugins/facetapi/query_type_geo.inc
files[] = tests/Dummy_Solr.php
files[] = tests/apachesolr_base.test
files[] = tests/solr_index_and_search.test
@@ -23,9 +24,9 @@ files[] = tests/solr_base_query.test
files[] = tests/solr_base_subquery.test
files[] = tests/solr_document.test
-; Information added by drupal.org packaging script on 2012-04-05
-version = "7.x-1.0-beta19"
+; Information added by drupal.org packaging script on 2012-10-16
+version = "7.x-1.1"
core = "7.x"
project = "apachesolr"
-datestamp = "1333625738"
+datestamp = "1350356169"
View
30 sites/all/modules/contrib/apachesolr/apachesolr.install
@@ -106,8 +106,6 @@ function apachesolr_schema() {
'can disable' => FALSE,
// Variable name to use in exported code.
'identifier' => 'environment',
- // Use the environment load callback directly.
- 'load callback' => 'apachesolr_environment_load',
// Thin wrapper for the environment save callback.
'save callback' => 'apachesolr_ctools_environment_save',
// Thin wrapper for the environment delete callback.
@@ -214,7 +212,7 @@ function apachesolr_schema() {
'not null' => TRUE,
),
'status' => array(
- 'description' => 'Boolean indicating whether the entity is visible to non-administrators (eg, published for nodes).',
+ 'description' => 'Boolean indicating whether the entity should be in the index.',
'type' => 'int',
'not null' => TRUE,
'default' => 1,
@@ -227,7 +225,7 @@ function apachesolr_schema() {
),
),
'indexes' => array(
- 'changed' => array('bundle', 'status', 'changed'),
+ 'bundle_changed' => array('bundle', 'changed'),
),
'primary key' => array('entity_id'),
);
@@ -280,6 +278,7 @@ function apachesolr_uninstall() {
variable_del('apachesolr_index_updated');
variable_del('apachesolr_read_only');
variable_del('apachesolr_set_nodeapi_messages');
+ variable_del('apachesolr_last_optimize');
// Remove blocks.
db_delete('block')->condition('module', 'apachesolr')->execute();
}
@@ -801,3 +800,26 @@ function apachesolr_update_7013() {
db_add_primary_key('apachesolr_index_bundles', array('env_id', 'entity_type', 'bundle'));
}
+/**
+ * Remove status from the key.
+ */
+function apachesolr_update_7014() {
+ $types = array(
+ 'other' => 'apachesolr_index_entities',
+ 'node' => 'apachesolr_index_entities_node',
+ );
+ foreach ($types as $type => $table) {
+ db_drop_index($table, 'changed');
+ db_add_index($table, 'bundle_changed', array('bundle', 'changed'));
+ }
+}
+
+
+/**
+ * Fix primary key schema mismatch for those who cleanly installed with beta16.
+ */
+function apachesolr_update_7015() {
+ // Brand new installations since update_7013 have the wrong primary key.
+ db_drop_primary_key('apachesolr_index_entities');
+ db_add_primary_key('apachesolr_index_entities', array('entity_id', 'entity_type'));
+}
View
200 sites/all/modules/contrib/apachesolr/apachesolr.interface.inc
@@ -311,3 +311,203 @@ interface DrupalSolrQueryInterface {
function solr($method);
}
+/**
+ * The interface for all 'Service' objects.
+ */
+interface DrupalApacheSolrServiceInterface {
+ /**
+ * Call the /admin/ping servlet, to test the connection to the server.
+ *
+ * @param $timeout
+ * maximum time to wait for ping in seconds, -1 for unlimited (default 2).
+ * @return
+ * (float) seconds taken to ping the server, FALSE if timeout occurs.
+ */
+ function ping($timeout = 2);
+
+ /**
+ * Get information about the Solr Core.
+ *
+ * @return
+ * (string) system info encoded in json
+ */
+ function getSystemInfo();
+
+ /**
+ * Get just the field meta-data about the index.
+ */
+ function getFields($num_terms = 0);
+
+ /**
+ * Get meta-data about the index.
+ */
+ function getLuke($num_terms = 0);
+
+ /**
+ * Get information about the Solr Core.
+ *
+ * Returns a Simple XMl document
+ */
+ function getStats();
+
+ /**
+ * Get summary information about the Solr Core.
+ */
+ function getStatsSummary();
+
+ /**
+ * Clear cached Solr data.
+ */
+ function clearCache();
+
+ /**
+ * Constructor
+ *
+ * @param $url
+ * The URL to the Solr server, possibly including a core name. E.g. http://localhost:8983/solr/
+ * or https://search.example.com/solr/core99/
+ * @param $env_id
+ * The machine name of a corresponding saved configuration used for loading
+ * data like which facets are enabled.
+ */
+ function __construct($url, $env_id = NULL);
+
+ function getId();
+
+ /**
+ * Make a request to a servlet (a path) that's not a standard path.
+ *
+ * @param string $servlet
+ * A path to be added to the base Solr path. e.g. 'extract/tika'
+ *
+ * @param array $params
+ * Any request parameters when constructing the URL.
+ *
+ * @param array $options
+ * @see drupal_http_request() $options.
+ *
+ * @return
+ * response object
+ *
+ * @thows Exception
+ */
+ function makeServletRequest($servlet, $params = array(), $options = array());
+
+ /**
+ * Get the Solr url
+ *
+ * @return string
+ */
+ function getUrl();
+
+ /**
+ * Set the Solr url.
+ *
+ * @param $url
+ *
+ * @return $this
+ */
+ function setUrl($url);
+
+ /**
+ * Raw update Method. Takes a raw post body and sends it to the update service. Post body
+ * should be a complete and well formed xml document.
+ *
+ * @param string $rawPost
+ * @param float $timeout Maximum expected duration (in seconds)
+ *
+ * @return response object
+ *
+ * @throws Exception If an error occurs during the service call
+ */
+ function update($rawPost, $timeout = FALSE);
+
+ /**
+ * Add an array of Solr Documents to the index all at once
+ *
+ * @param array $documents Should be an array of ApacheSolrDocument instances
+ * @param boolean $allowDups
+ * @param boolean $overwritePending
+ * @param boolean $overwriteCommitted
+ *
+ * @return response objecte
+ *
+ * @throws Exception If an error occurs during the service call
+ */
+ function addDocuments($documents, $overwrite = NULL, $commitWithin = NULL);
+
+ /**
+ * Send a commit command. Will be synchronous unless both wait parameters are set to false.
+ *
+ * @param boolean $optimize Defaults to true
+ * @param boolean $waitFlush Defaults to true
+ * @param boolean $waitSearcher Defaults to true
+ * @param float $timeout Maximum expected duration (in seconds) of the commit operation on the server (otherwise, will throw a communication exception). Defaults to 1 hour
+ *
+ * @return response object
+ *
+ * @throws Exception If an error occurs during the service call
+ */
+ function commit($optimize = true, $waitFlush = true, $waitSearcher = true, $timeout = 3600);
+
+ /**
+ * Create a delete document based on document ID
+ *
+ * @param string $id Expected to be utf-8 encoded
+ * @param float $timeout Maximum expected duration of the delete operation on the server (otherwise, will throw a communication exception)
+ *
+ * @return response object
+ *
+ * @throws Exception If an error occurs during the service call
+ */
+ function deleteById($id, $timeout = 3600);
+
+ /**
+ * Create and post a delete document based on multiple document IDs.
+ *
+ * @param array $ids Expected to be utf-8 encoded strings
+ * @param float $timeout Maximum expected duration of the delete operation on the server (otherwise, will throw a communication exception)
+ *
+ * @return response object
+ *
+ * @throws Exception If an error occurs during the service call
+ */
+ function deleteByMultipleIds($ids, $timeout = 3600);
+
+ /**
+ * Create a delete document based on a query and submit it
+ *
+ * @param string $rawQuery Expected to be utf-8 encoded
+ * @param float $timeout Maximum expected duration of the delete operation on the server (otherwise, will throw a communication exception)
+ * @return stdClass response object
+ *
+ * @throws Exception If an error occurs during the service call
+ */
+ function deleteByQuery($rawQuery, $timeout = 3600);
+
+ /**
+ * Send an optimize command. Will be synchronous unless both wait parameters are set
+ * to false.
+ *
+ * @param boolean $waitFlush
+ * @param boolean $waitSearcher
+ * @param float $timeout Maximum expected duration of the commit operation on the server (otherwise, will throw a communication exception)
+ *
+ * @return response object
+ *
+ * @throws Exception If an error occurs during the service call
+ */
+ function optimize($waitFlush = true, $waitSearcher = true, $timeout = 3600);
+
+ /**
+ * Simple Search interface
+ *
+ * @param string $query The raw query string
+ * @param array $params key / value pairs for other query parameters (see Solr documentation), use arrays for parameter keys used more than once (e.g. facet.field)
+ *
+ * @return response object
+ *
+ * @throws Exception If an error occurs during the service call
+ */
+ function search($query = '', array $params = array(), $method = 'GET');
+}
View
559 sites/all/modules/contrib/apachesolr/apachesolr.module
@@ -60,7 +60,7 @@ function apachesolr_menu() {
'access arguments' => array('administer search'),
'weight' => 0,
'file' => 'apachesolr.admin.inc',
- 'type' => MENU_DEFAULT_LOCAL_TASK,
+ 'type' => MENU_LOCAL_TASK,
);
$items[$settings_path . '%apachesolr_environment/index/remaining'] = array(
'title' => 'Remaining',
@@ -152,21 +152,29 @@ function apachesolr_menu() {
'file' => 'apachesolr.admin.inc',
'type' => MENU_CALLBACK,
);
- $env_id = apachesolr_default_environment();
- $items['admin/reports/apachesolr'] = array(
+
+ $reports_path = 'admin/reports/apachesolr';
+ $items[$reports_path] = array(
'title' => 'Apache Solr search index',
'description' => 'Information about the contents of the index the server',
'page callback' => 'apachesolr_index_report',
- 'page arguments' => array($env_id),
'access arguments' => array('access site reports'),
'file' => 'apachesolr.admin.inc',
);
- $items['admin/reports/apachesolr/index'] = array(
+ $items[$reports_path . '/%apachesolr_environment'] = array(
+ 'title' => 'Apache Solr search index',
+ 'description' => 'Information about the contents of the index the server',
+ 'page callback' => 'apachesolr_index_report',
+ 'page arguments' => array(3),
+ 'access arguments' => array('access site reports'),
+ 'file' => 'apachesolr.admin.inc',
+ );
+ $items[$reports_path . '/%apachesolr_environment/index'] = array(
'title' => 'Search index',
'file' => 'apachesolr.admin.inc',
'type' => MENU_DEFAULT_LOCAL_TASK,
);
- $items['admin/reports/apachesolr/conf'] = array(
+ $items[$reports_path . '/%apachesolr_environment/conf'] = array(
'title' => 'Configuration files',
'page callback' => 'apachesolr_config_files_overview',
'access arguments' => array('access site reports'),
@@ -174,14 +182,24 @@ function apachesolr_menu() {
'weight' => 5,
'type' => MENU_LOCAL_TASK,
);
- $items['admin/reports/apachesolr/conf/%'] = array(
+ $items[$reports_path . '/%apachesolr_environment/conf/%'] = array(
'title' => 'Configuration file',
'page callback' => 'apachesolr_config_file',
- 'page arguments' => array(4),
+ 'page arguments' => array(5, 3),
'access arguments' => array('access site reports'),
'file' => 'apachesolr.admin.inc',
'type' => MENU_CALLBACK,
);
+ if (module_exists('devel')) {
+ $items['node/%node/devel/apachesolr'] = array(
+ 'title' => 'Apache Solr',
+ 'page callback' => 'apachesolr_devel',
+ 'page arguments' => array(1),
+ 'access arguments' => array('access devel information'),
+ 'file' => 'apachesolr.admin.inc',
+ 'type' => MENU_LOCAL_TASK,
+ );
+ }
// We handle our own menu paths for facets
if (module_exists('facetapi')) {
@@ -300,6 +318,12 @@ function apachesolr_facetapi_query_types() {
'adapter' => 'apachesolr',
),
),
+ 'apachesolr_geo' => array(
+ 'handler' => array(
+ 'class' => 'ApacheSolrFacetapiGeo',
+ 'adapter' => 'apachesolr',
+ ),
+ ),
);
}
@@ -329,41 +353,46 @@ function apachesolr_facetapi_facet_info($searcher_info) {
function apachesolr_default_node_facet_info($searcher_info) {
$facets = apachesolr_common_node_facets();
- foreach (apachesolr_entity_fields('node') as $field_nm => $field_info) {
- if (!empty($field_info['facets'])) {
- $field = apachesolr_index_key($field_info);
- $facets[$field] = array(
- 'label' => check_plain($field_info['display_name']),
- 'dependency plugins' => $field_info['dependency plugins'],
- 'field api name' => $field_info['field']['field_name'],
- 'description' => t('Filter by field of type @type.', array('@type' => $field_info['field']['type'])),
- 'map callback' => $field_info['map callback'],
- 'map options' => $field_info,
- 'hierarchy callback' => $field_info['hierarchy callback'],
- );
- if (!empty($field_info['facet mincount allowed'])) {
- $facets[$field]['facet mincount allowed'] = $field_info['facet mincount allowed'];
- }
- if (!empty($field_info['facet missing allowed'])) {
- $facets[$field]['facet missing allowed'] = $field_info['facet missing allowed'];
- }
- if (!empty($field_info['query types'])) {
- $facets[$field]['query types'] = $field_info['query types'];
- }
- // TODO : This is actually deprecated but we should still support
- // older versions of facetapi. We should remove once facetapi has RC1
- // For reference : http://drupal.org/node/1161444
- if (!empty($field_info['query type'])) {
- $facets[$field]['query type'] = $field_info['query type'];
- }
- if (!empty($field_info['min callback'])) {
- $facets[$field]['min callback'] = $field_info['min callback'];
- }
- if (!empty($field_info['max callback'])) {
- $facets[$field]['max callback'] = $field_info['max callback'];
- }
- if (!empty($field_info['map callback'])) {
- $facets[$field]['map callback'] = $field_info['map callback'];
+ foreach (apachesolr_entity_fields('node') as $field_nm => $nodefields) {
+ foreach ($nodefields as $field_info) {
+ if (!empty($field_info['facets'])) {
+ $field = apachesolr_index_key($field_info);
+ $facets[$field] = array(
+ 'label' => check_plain($field_info['display_name']),
+ 'dependency plugins' => $field_info['dependency plugins'],
+ 'field api name' => $field_info['field']['field_name'],
+ 'description' => t('Filter by field of type @type.', array('@type' => $field_info['field']['type'])),
+ 'map callback' => $field_info['map callback'],
+ 'map options' => $field_info,
+ 'hierarchy callback' => $field_info['hierarchy callback'],
+ );
+ if (!empty($field_info['facet mincount allowed'])) {
+ $facets[$field]['facet mincount allowed'] = $field_info['facet mincount allowed'];
+ }
+ if (!empty($field_info['facet missing allowed'])) {
+ $facets[$field]['facet missing allowed'] = $field_info['facet missing allowed'];
+ }
+ if (!empty($field_info['query types'])) {
+ $facets[$field]['query types'] = $field_info['query types'];
+ }
+ if (!empty($field_info['allowed operators'])) {
+ $facets[$field]['allowed operators'] = $field_info['allowed operators'];
+ }
+ // TODO : This is actually deprecated but we should still support
+ // older versions of facetapi. We should remove once facetapi has RC1
+ // For reference : http://drupal.org/node/1161444
+ if (!empty($field_info['query type'])) {
+ $facets[$field]['query type'] = $field_info['query type'];
+ }
+ if (!empty($field_info['min callback'])) {
+ $facets[$field]['min callback'] = $field_info['min callback'];
+ }
+ if (!empty($field_info['max callback'])) {
+ $facets[$field]['max callback'] = $field_info['max callback'];
+ }
+ if (!empty($field_info['map callback'])) {
+ $facets[$field]['map callback'] = $field_info['map callback'];
+ }
}
}
}
@@ -456,7 +485,7 @@ function apachesolr_failure($search_name, $querystring) {
switch ($fail_rule) {
case 'apachesolr:show_error':
- drupal_set_message(t('The Apache Solr search engine is not available. Please contact your site administrator.'), 'error');
+ drupal_set_message(t('Search is temporarily unavailable. If the problem persists, please contact the site administrator.'), 'error');
break;
case 'apachesolr:show_no_results':
// Do nothing.