diff --git a/docs/administration/project_organization/architecture.md b/docs/administration/project_organization/architecture.md index adb022d7d4..ef857e6fb6 100644 --- a/docs/administration/project_organization/architecture.md +++ b/docs/administration/project_organization/architecture.md @@ -28,7 +28,7 @@ The architecture of [[= product_name =]] is layered and uses clearly defined API |[Persistence cache](persistence_cache.md)|The implementation of SPI\Persistence that decorates the main backend implementation.| |[Search](search.md)|Search API that allows both full-text search and querying the content.| |[SQL Storage Engine](search.md#legacy-search-engine)|Legacy search engine is SQL-based and uses Doctrine's database connection.| -|[Solr Storage Engine](solr_search_engine.md)|Transparent drop-in replacement for the SQL-based Legacy search engine.| +|[Solr Storage Engine](solr_overview.md)|Transparent drop-in replacement for the SQL-based Legacy search engine.| |[IO](file_management.md#native-io-handler)|The IO API is organized around two types of handlers, both used by the IOService.| |[IO Handler](clustering.md#dfs-io-handler)|The IO Handler manipulates metadata, making up for the potential inconsistency of network-based filesystems.| |[Recommendation](enable_personalization.md)|Recommendation API.| diff --git a/docs/content_management/field_types/field_type_reference/field_type_reference.md b/docs/content_management/field_types/field_type_reference/field_type_reference.md index 4551bc0a80..07ed14acf1 100644 --- a/docs/content_management/field_types/field_type_reference/field_type_reference.md +++ b/docs/content_management/field_types/field_type_reference/field_type_reference.md @@ -64,4 +64,4 @@ Legacy Search/Storage Engine index is limited to 255 characters in database desi so formatted and unformatted text blocks will only index the first part. In case of multiple selection Field Types like Keyword, Selection, Country, etc., only the first choices are indexed. They are indexed only as a text blob separated by string separator. -Proper indexing of these Field Types is done with [Solr Search engine](solr_search_engine.md). +Proper indexing of these Field Types is done with [Solr Search engine](solr_overview.md). diff --git a/docs/content_management/field_types/field_type_search.md b/docs/content_management/field_types/field_type_search.md index 7f422c8bc5..2b5601eede 100644 --- a/docs/content_management/field_types/field_type_search.md +++ b/docs/content_management/field_types/field_type_search.md @@ -100,4 +100,4 @@ You could also define a custom Field definition for certain Fields, like for the !!! note - If you want to learn more about the Solr implementation and detailed information about configuring it, check out the [Solr Search Bundle](solr_search_engine.md). + If you want to learn more about the Solr implementation and detailed information about configuring it, check out the [Solr Search Bundle](solr_overview.md). diff --git a/docs/content_management/url_management/url_api.md b/docs/content_management/url_management/url_api.md index ad6c8cfcd9..3edd2cbe97 100644 --- a/docs/content_management/url_management/url_api.md +++ b/docs/content_management/url_management/url_api.md @@ -27,4 +27,4 @@ in which you need to specify: ## URL search reference For the reference of Search Criteria and Sort Clauses you can use in URL search, -see [URL search reference](url_search_reference.md). +see [URL Search Criteria](url_search_criteria.md) and [URL Sort Clauses](url_search_sort_clauses.md). diff --git a/docs/getting_started/install_ibexa_dxp.md b/docs/getting_started/install_ibexa_dxp.md index 0a487d89c8..9ef2c15bfc 100644 --- a/docs/getting_started/install_ibexa_dxp.md +++ b/docs/getting_started/install_ibexa_dxp.md @@ -259,17 +259,17 @@ You may choose to replace the [default search engine](search.md#legacy-search-en === "Solr" - Follow [How to set up Solr search engine](solr_search_engine.md#how-to-set-up-solr-search-engine) to install Solr. + Follow [How to set up Solr search engine](install_solr.md) to install Solr. === "Elasticsearch" Do the following steps to enable Elasticsearch: - 1. [Download and install Elasticsearch](elasticsearch_search_engine.md#step-1-download-and-install-elasticsearch) - 2. [Verify that the Elasticsearch instance is up](elasticsearch_search_engine.md#step-2-verify-that-the-elasticsearch-instance-is-up) - 3. [Set the default search engine](elasticsearch_search_engine.md#step-3-set-the-default-search-engine) - 4. [Configure the search engine](elasticsearch_search_engine.md#step-4-configure-the-search-engine) - 5. [Push the templates](elasticsearch_search_engine.md#step-5-push-the-templates) + 1. [Download and install Elasticsearch](install_elastic_search.md) + 2. [Verify that the Elasticsearch instance is up](install_elastic_search.md#verify-the-instance) + 3. [Set the default search engine](install_elastic_search.md#set-the-default-search-engine) + 4. [Configure the search engine](configure_elastic_search.md) + 5. [Push the templates](install_elastic_search.md#push-the-templates) Configure the following parameter in the `.env` file: diff --git a/docs/infrastructure_and_maintenance/cache/persistence_cache.md b/docs/infrastructure_and_maintenance/cache/persistence_cache.md index 51312fa84d..4667138254 100644 --- a/docs/infrastructure_and_maintenance/cache/persistence_cache.md +++ b/docs/infrastructure_and_maintenance/cache/persistence_cache.md @@ -34,7 +34,7 @@ Notes: For instance tree operations or changes to Content Types are examples of operations that also need to invalidate content cache by tags. - Search is not defined as persistence and the queries themselves are not planned to be cached as they are too complex by design (for example, full text). - Use [Solr](solr_search_engine.md) which caches this for you to improve scale/performance, and to offload your database. + Use [Solr](solr_overview.md) which caches this for you to improve scale/performance, and to offload your database. For further details on which calls are cached or not, see details in the [Symfony Web Debug Toolbar](devops.md#web-debug-toolbar) which has info on cache use in two places: diff --git a/docs/infrastructure_and_maintenance/clustering/clustering.md b/docs/infrastructure_and_maintenance/clustering/clustering.md index c418d1a36f..22aea3fa86 100644 --- a/docs/infrastructure_and_maintenance/clustering/clustering.md +++ b/docs/infrastructure_and_maintenance/clustering/clustering.md @@ -28,7 +28,7 @@ For further details on requirements, see [Requirements page](requirements.md). It is also recommended to use: -- [Solr](solr_search_engine.md) or [Elasticsearch](elasticsearch_search_engine.md) for better search and performance +- [Solr](solr_overview.md) or [Elasticsearch](elastic_search_overview.md) for better search and performance - a CDN for improved performance and faster ping time worldwide - you can use Fastly, which has native support as HTTP cache and CDN. - active/passive database for failover diff --git a/docs/infrastructure_and_maintenance/performance.md b/docs/infrastructure_and_maintenance/performance.md index fc047b25ca..208870408a 100644 --- a/docs/infrastructure_and_maintenance/performance.md +++ b/docs/infrastructure_and_maintenance/performance.md @@ -16,7 +16,7 @@ If you are in a hurry, the most important recommendations on this page are: - Dump optimized Composer autoload classmap - Use a full web (Nginx/Apache) server with vhost - Avoid shared filesystems for code (Docker for Mac/Win, VirtualBox/*, Vagrant, etc.), or find ways to optimize or work around the issues. -- For clustering (mainly relevant for production/staging), reduce latency to Redis/Memcached, use Varnish and [Solr](solr_search_engine.md). +- For clustering (mainly relevant for production/staging), reduce latency to Redis/Memcached, use Varnish and [Solr](solr_overview.md). ## Client @@ -78,12 +78,12 @@ In production setups: ### Search -- Use [Solr Bundle and Solr](solr_search_engine.md) to greatly offload your database and get more stable performance on your installation. +- Use [Solr Bundle and Solr](solr_overview.md) to greatly offload your database and get more stable performance on your installation. ## Long-running console commands Executing long-running console commands can result in running out of memory. -Two examples of such commands are a custom import command and the indexing command provided by the [Solr Bundle](solr_search_engine.md). +Two examples of such commands are a custom import command and the indexing command provided by the [Solr Bundle](solr_overview.md). ### Reducing memory usage diff --git a/docs/search/extensibility/create_custom_sort_clause.md b/docs/search/extensibility/create_custom_sort_clause.md index 62e844054b..a372b43872 100644 --- a/docs/search/extensibility/create_custom_sort_clause.md +++ b/docs/search/extensibility/create_custom_sort_clause.md @@ -2,7 +2,7 @@ description: Create custom Sort Clause to use with Solr and Elasticsearch search engines. --- -# Create custom Search Criterion +# Create custom Sort Clause To create a custom Sort Clause, do the following. diff --git a/docs/search/extensibility/index_custom_elasticsearch_data.md b/docs/search/extensibility/index_custom_elasticsearch_data.md index 2f62e27b5d..f0461cf0a3 100644 --- a/docs/search/extensibility/index_custom_elasticsearch_data.md +++ b/docs/search/extensibility/index_custom_elasticsearch_data.md @@ -4,7 +4,7 @@ description: Index custom data when using the Elasticsearch search engine. # Index custom Elasticsearch data -[Elasticsearch](elasticsearch_search_engine.md) indexes content and Location data out of the box. +[Elasticsearch](elastic_search_overview.md) indexes content and Location data out of the box. Besides what is indexed automatically, you can add additional data to the Elasticsearch index. To do so, subscribe to one of the following events: diff --git a/docs/search/search_reindexing.md b/docs/search/reindex_search.md similarity index 97% rename from docs/search/search_reindexing.md rename to docs/search/reindex_search.md index 3e96d073ce..81c64b041c 100644 --- a/docs/search/search_reindexing.md +++ b/docs/search/reindex_search.md @@ -2,7 +2,7 @@ description: Reindexing lets you create or refresh the search engine index. --- -# Search reindexing +# Reindex search To (re)create or refresh the search engine index for configured search engines (per SiteAccess repository), use the `php bin/console ibexa:reindex` command. diff --git a/docs/search/search.md b/docs/search/search.md index 3ac0ae91b0..c0162d2ca2 100644 --- a/docs/search/search.md +++ b/docs/search/search.md @@ -8,9 +8,9 @@ page_type: landing_page [[= product_name =]] exposes a very powerful [Search API](search_api.md), allowing both full-text search and querying the content Repository using several built-in Search Criteria and Sort Clauses. These are supported across different search engines, allowing you to plug in another search engine without changing your code. [[= cards([ - "search/search_engines", - "search/elasticsearch_search_engine", - "search/solr_search_engine", + "search/search_engines/search_engines", + "search/search_engines/elastic_search/elastic_search_overview", + "search/search_engines/solr_search_engine/solr_overview", "search/search_api", "search/search_criteria_and_sort_clauses", "search/extensibility/create_custom_search_criterion", diff --git a/docs/search/search_engines.md b/docs/search/search_engines.md deleted file mode 100644 index 59532a263f..0000000000 --- a/docs/search/search_engines.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: Ibexa DXP enables you to use Solr or Elasticsearch search engines, in addition to the built-in legacy search engine which has limited functionalities. ---- - -# Search engines - -Currently, the following search engines exist in their own [[= product_name =]] Bundles: - -1. [Legacy search engine](#legacy-search-engine), a database-powered search engine for basic needs. -1. [Solr](solr_search_engine.md), an integration providing better overall performance, much better scalability and support for more advanced search capabilities. -1. [Elasticsearch](elasticsearch_search_engine.md), available for [[= product_name_exp =]] customers, a document-oriented engine providing even better performance and scalability. - -## Legacy search engine - -Legacy search engine is the default search engine, it is SQL-based and uses Doctrine's database connection. -Its connections are defined in the same way as for storage engine, and no further specific configuration is needed. - -!!! tip - - The features and performance of Legacy search engine are limited. - If you have specific search or performance needs you should look towards using [Solr](solr_search_engine.md) - or [Elasticsearch](elasticsearch_search_engine.md). - - Using the Legacy search engine disables most shop features, such as product search. - -### Configuring Repository with Legacy search engine - -Search can be configured independently from storage, and the following configuration example shows both the default values, and how you configure legacy as the search engine: - -``` yaml -ibexa: - repositories: - main: - storage: - engine: legacy - connection: default - search: - engine: legacy - connection: default -``` - -## Search engine comparison - -| Feature | Elasticsearch | Apache Solr | Legacy Search Engine (SQL) | -| --- | --- | --- | --- | -| Filtering | Yes | Yes | Yes, limited\* | -| Query (filter with scoring) | Yes | Yes | Only filters, no scoring | -| Fulltext search | Yes, limited | Yes | Yes, limited\*\* | -| Index-time boosting | \*\*\* | No | No | -| Aggregations | Yes | Yes | No | - - -\* Usage of Criteria and Sort Clauses for Fields does not perform well on medium to larger -amount of data with Legacy Search Engine (SQL), use Solr for this. - -\*\* For more information about fulltext search syntax support, see [Fulltext Criterion](fulltext_criterion.md). - -\*\*\* Elasticsearch offers query-time boosting instead. diff --git a/docs/search/elasticsearch_search_engine.md b/docs/search/search_engines/elastic_search/configure_elastic_search.md similarity index 73% rename from docs/search/elasticsearch_search_engine.md rename to docs/search/search_engines/elastic_search/configure_elastic_search.md index 16b272c922..e2a556e19e 100644 --- a/docs/search/elasticsearch_search_engine.md +++ b/docs/search/search_engines/elastic_search/configure_elastic_search.md @@ -1,113 +1,19 @@ --- -description: Configure Elasticsearch to use with Ibexa DXP. +description: Configure Elasticsearch to use it with Ibexa DXP. --- -# Elasticsearch search engine +# Configure Elasticsearch -Elasticsearch is an open-source, distributed, Java-based search engine that responds to queries -in real-time and is easily scalable in reaction to changing processing needs. +## Configure connections -For a detailed description of advanced settings that you might require in a specific -production environment, see the documentation provided by Elastic, beginning with the [Set up Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/setup.html) section. +To configure Elasticsearch, first, you need to configure the connections. -**Prerequisite** +There are two possibilities of connection: +- using [cluster of Elasticsearch nodes](#cluster) +- using [Elasticsearch Cloud](#elasticsearch-cloud) -To proceed you need to be familiar with how indexing, filtering and queries work. - -## Step 1: Download and install Elasticsearch - -[Install Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/install-elasticsearch.html) on your server. -For example, use the following [Docker](https://docs.docker.com/get-started/overview/) command: - -``` -docker run -d --name ibexa-dxp-elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.16.2 -``` - -!!! note - - [[= product_name =]] supports Elasticsearch in version 7.16.2 or higher. - -## Step 2: Verify that the Elasticsearch instance is up - -To make sure that the Elasticsearch instance operates properly, access the instance (for example, with `curl http://localhost:9200/`). - -If Elasticsearch operates properly, an object with cluster details is displayed. -It should be similar to the following example: - -``` json -{ - "name" : "doej-MacPro-mTkBe", - "cluster_name" : "elasticsearch", - "cluster_uuid" : "WLYqnQ_lSZGbX-vDIe_vZQ", - "version" : { - "number" : "7.7.0", - "build_flavor" : "default", - "build_type" : "tar", - "build_hash" : "5b1fea5", - "build_date" : "2020-05-10T02:35:59.208Z", - "build_snapshot" : false, - "lucene_version" : "8.5.1", - "minimum_wire_compatibility_version" : "6.8.0", - "minimum_index_compatibility_version" : "6.0.0-beta1" - }, - "tagline" : "You Know, for Search" -} -``` - -## Step 3: Set the default search engine - -Set the following environment variable (for example, in the `.env` or `.env.local` file): - -``` yaml -SEARCH_ENGINE=elasticsearch -``` - -## Step 4: Configure the search engine - -[[= product_name =]] comes pre-configured to work with an Elasticsearch cluster that uses default settings, and you can use this initial setup for testing purposes. -However, to effectively search through actual data, you must provide specific settings. - -!!! note - - All the settings, their order and meaning, correspond to the settings that are described in - the Elasticsearch documentation. - -First, decide how [[= product_name =]] connects to Elasticsearch and configure other connection settings. -For more information, see [Configuring connections](#connections). - -Then, define a field type mappings template that instructs Elasticsearch to interpret [[= product_name =]] fields as specific types. For more information, see [Configuring field type mappings](#field-type-mapping-templates). - -## Step 5: Push the templates - -For each of your defined connections, push the templates to the Elasticsearch engine by running the following command: - -``` bash -php bin/console ibexa:elasticsearch:put-index-template -``` - -You can modify the behavior of the command with a number of switches. Use the `-h` switch to display a complete list of available options. - -## Step 6: Reindex the database - -After creating index templates, run the following command to reindex your data: - -``` bash -php bin/console ibexa:reindex -``` - -!!! caution "Risks of premature indexing" - - Do not reindex your data before you create index templates. - Otherwise Elasticsearch attempts to use its [dynamic field mapping](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/dynamic-field-mapping.html) feature to create type mappings automatically. - -## Elasticsearch configuration - -### Connections - -When you configure the Elasticsearch integration, you must first configure the connections. -You either connect to a [cluster of Elasticsearch nodes](#cluster) or the [Elasticsearch Cloud](#elasticsearch-cloud). - -Define the connection settings under the `connections` [configuration key](configuration.md#configuration-files). First, set a name of the connection: +No matter which option you choose, you have to define the connection settings under the `connections` key. +Set a name of the connection: ``` yaml ibexa_elasticsearch: @@ -115,11 +21,6 @@ ibexa_elasticsearch: : ``` -Then decide whether to add a cluster that you administer and manage yourself, or use a cloud -solution from Elastic, as well as configure additional parameters. -You can then decide how the cluster [handles communication with individual nodes](#multi-node-cluster-behavior), -and configure the [security settings](#security). - !!! tip "A default connection" If you define more than one connection, for example, to create a separate connection for @@ -131,28 +32,30 @@ and configure the [security settings](#security). default_connection: ``` -### Cluster +Now, you need to decide whether to add a cluster that you administer and manage yourself, or use a cloud +solution from Elastic, as well as configure additional parameters. + +If you want to connect by using a cluster, follow the instructions below in the [Cluster](#cluster) section. +If you want to use Elasticsearch Cloud, skip to [Elasticsearch Cloud](#elasticsearch-cloud) section. + +## Configure clustering -A cluster consists of a number of nodes. -You might start with just one node and add more nodes if you need more processing power. +A cluster consists of nodes. +You might start with one node and then add more nodes if you need more processing power. -When you configure a node, set the following parameters: +When you configure a node, you need to set the following parameters: -- `host` - An IP address or domain name of the host. -The default value is `localhost`. -- `port` - A port to connect to. -The default value is `9200`. +- `host` - an IP address or domain name of the host. Default value: `localhost`. +- `port` - a port to connect to. Default value: `9200`. If you have several Elasticsearch instances that run on the same host, and want to make them distinct, you can change the default number. -- `scheme` - A protocol to be used to access the node. Default value is `http`. -- `path` - By default, path is not used. -The default value is `null`. +- `scheme` - a protocol used to access the node. Default value: `http`. +- `path` - by default, path is not used. Default value: `null`. If you have several Elasticsearch instances that run on the same host, and want to make them distinct, you can define a path for each instance. -- `user`/`pass` - Credentials, if needed to log in to the host. -Default values are `null`. +- `user`/`pass` - credentials, if needed to log in to the host. Default values: `null`. -List the addresses of cluster nodes under the `hosts` key: +Next, list the addresses of cluster nodes under the `hosts` key: ``` yaml ibexa_elasticsearch: @@ -203,13 +106,13 @@ ibexa_elasticsearch: default_connection: simple ``` -#### Multi-node cluster behavior +### Multi-node cluster behavior When you configure a cluster-based connection, and the cluster consists of many nodes, you can choose strategies that govern how the cluster reacts to changing operating conditions, or how workload is distributed among the nodes. -##### Connection pool +#### Connection pool With this setting you decide how a list of hosts that form a cluster is managed. The list of active hosts tends to change in time, due to different reasons, such as connectivity @@ -232,7 +135,7 @@ For more information and a list of available choices, see [Connection pool](http If you change the connection pool setting, it is recommended that you to perform load tests to check whether the change does not negatively impact the performance of your environment. -##### Connection selector +#### Connection selector When the cluster consists of many hosts, the `connection_selector` setting decides what strategy is used to pick a node to send query requests to. @@ -265,14 +168,14 @@ number of retries might differ. For more information, see [Set retries](https://www.elastic.co/guide/en/elasticsearch/client/php-api/7.x/configuration.html#_set_retries). -### Elasticsearch Cloud +## Configure Elasticsearch Cloud As an alternative to using your own cluster, you can use Elasticsearch Cloud, a commercial SaaS solution. With Elasticsearch Cloud you do not have to build or manage your own Elasticsearch cluster. Also, you do all the configuration and administration in a graphical user interface. To connect to a cloud solution with [[= product_name =]], you must set the `elastic_cloud_id` parameter by -providing an alphanumerical ID string that you obtain from the cloud's user interface, for example: +providing an alphanumerical ID string that you get from the cloud's user interface, for example: ``` yaml : @@ -281,7 +184,7 @@ providing an alphanumerical ID string that you obtain from the cloud's user inte With the ID set, you must configure authentication to be able to access the remote environment. -### Security +## Configure security Elasticsearch instances support `basic` and `api_key` authentication methods. You select authentication type and configure the settings under the `authentication` key. By default, authentication is disabled: @@ -295,10 +198,10 @@ You select authentication type and configure the settings under the `authenticat If you connect to Elasticsearch hosts outside of your local network, you might also need to configure SSL encryption. -#### Basic authentication +### Basic authentication If your Elasticsearch server is protected by HTTP authentication, you must provide [[= product_name =]] with the credentials. -When using basic authentication, you must pass the following parameters: +In the basic authentication, you must pass the following parameters: ``` yaml @@ -321,7 +224,7 @@ ibexa_elasticsearch: credentials: ['elastic', '1htFY83VvX2JRDw88MOkOejk'] ``` -#### API key authentication +### API key authentication If your Elasticsearch cluster is protected by API keys, you must provide the key and secret in authentication configuration to connect [[= product_name =]] with the cluster. With API key authentication you can define different @@ -354,7 +257,7 @@ ibexa_elasticsearch: credentials: ['8Ek5f3IBGQlWj6v4M7zG', 'rmI6IechSnSJymWJ4LZqUw'] ``` -#### SSL +### SSL When you need to protect your communication with the Elasticsearch server, you can use SSL encryption. When configuring SSL for your internal infrastructure, you can use your own client certificates signed by a public CA. @@ -407,7 +310,7 @@ verification: false For more information, see [Elasticsearch: SSL Encyption](https://www.elastic.co/guide/en/elasticsearch/client/php-api/7.x/security.html#_ssl_encryption_2). -#### Enabling debugging +### Enable debugging In a staging environment, you can log messages about the status of communication with Elasticsearch. You can then use Symfony Profiler to review the logs. @@ -428,17 +331,17 @@ By default, debugging is disabled. To enable debugging, you can toggle either of Make sure that you disable debugging in a production environment. -### Field Type mapping templates +## Define Field Type mapping templates Before you can re-index the [[= product_name =]] data, so that Elasticsearch can search through its contents, you must define an index template. Templates instruct Elasticsearch to recognize [[= product_name =]] Fields as specific data types, based on, for example, a field name. They help you prevent Elasticsearch from using the dynamic field mapping feature to create type mappings automatically. -You can create several field type mapping templates for each index, for example, to define settings that are specific for different languages. +You can create several Field Type mapping templates for each index, for example, to define settings that are specific for different languages. When you establish a relationship between a field mapping template and a connection, you can apply several templates, too. -#### Defining a template +### Define a template -To define a field mapping template, you must provide a number of settings under the `index_templates` key. +To define a field mapping template, you must provide settings under the `index_templates` key. The structure of the template is as follows: ``` yaml @@ -513,7 +416,7 @@ For more information about mappings, see [Elasticsearch documentation](https://w For an example of default configuration with a list of searchable fields. To see the default configuration, go to `vendor/ibexa/elasticsearch/src/bundle/Resources/config/` and open the `default-config.yaml` file. -#### Fine-tuning the search results +### Fine-tune the search results Your search results can be adjusted by configuring additional parameters. For a list of available mapping parameters and their usage, see [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/mapping-params.html). @@ -572,7 +475,7 @@ ibexa_elasticsearch: # ... ``` -### Binding templates with connections +## Bind templates with connections Once you have created the field mapping template(s), you must establish a relationship between the templates and a connection. You do this by adding the "index_templates" key to a connection definition. @@ -589,7 +492,7 @@ If you have several index templates, you can apply different combinations of tem For more information about how Elasticsearch handles settings and mappings from multiple templates that match the same index, see [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/indices-templates-v1.html#multiple-templates-v1). -## Extending Elasticsearch +# Extend Elasticsearch To learn how you can create document field mappers, custom Search Criteria, -custom Sort Clauses and Aggregations, see [Search extensibility](create_custom_search_criterion.md). +custom Sort Clauses and Aggregations, see [Create custom Search Criterion](create_custom_search_criterion.md). diff --git a/docs/search/search_engines/elastic_search/elastic_search_overview.md b/docs/search/search_engines/elastic_search/elastic_search_overview.md new file mode 100644 index 0000000000..20470a8520 --- /dev/null +++ b/docs/search/search_engines/elastic_search/elastic_search_overview.md @@ -0,0 +1,22 @@ +--- +description: Elasticsearch search engine overview. +--- + +# Elasticsearch search engine + +Elasticsearch is an open-source, distributed, Java-based search engine that responds to queries +in real-time and is easily scalable in reaction to changing processing needs. + +Elasticsearch enables you to use filtering, query, query-time boosting, fulltext search and aggregations. +It organizes data into documents, that then are grouped into indices. + +As a result of having distributed architecture, Elasticsearch can analyze massive amounts of data with almost real-time performance. +Instead of searching text directly, it searches and index. Thanks to this mechanism, it is able to achieve fast response. + +For a detailed description of advanced settings that you might require in a specific production environment, +see the documentation provided by Elastic. +Start with the [Set up Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/setup.html) section. + +**Prerequisite** + +To proceed you need to be familiar with how indexing, filtering and queries work. \ No newline at end of file diff --git a/docs/search/search_engines/elastic_search/install_elastic_search.md b/docs/search/search_engines/elastic_search/install_elastic_search.md new file mode 100644 index 0000000000..099df55fa7 --- /dev/null +++ b/docs/search/search_engines/elastic_search/install_elastic_search.md @@ -0,0 +1,92 @@ +--- +description: Install Elasticsearch to use it with Ibexa DXP. +--- + +# Install Elasticsearch + +## Download and install Elasticsearch + +[Install Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/install-elasticsearch.html) on your server. +As an example, use the following [Docker](https://docs.docker.com/get-started/overview/) command: + +```yml +docker run -d --name ibexa-dxp-elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.16.2 +``` + +!!! note + + [[= product_name =]] supports Elasticsearch in version 7.16.2 or higher. + +## Verify the instance + +To make sure that the Elasticsearch instance operates properly, access the instance (for example, with `curl http://localhost:9200/`). + +If Elasticsearch operates properly, an object with cluster details is displayed. +It should be similar to the following example: + +``` json +{ + "name" : "doej-MacPro-mTkBe", + "cluster_name" : "elasticsearch", + "cluster_uuid" : "WLYqnQ_lSZGbX-vDIe_vZQ", + "version" : { + "number" : "7.7.0", + "build_flavor" : "default", + "build_type" : "tar", + "build_hash" : "5b1fea5", + "build_date" : "2020-05-10T02:35:59.208Z", + "build_snapshot" : false, + "lucene_version" : "8.5.1", + "minimum_wire_compatibility_version" : "6.8.0", + "minimum_index_compatibility_version" : "6.0.0-beta1" + }, + "tagline" : "You Know, for Search" +} +``` + +## Set the default search engine + +Set the following environment variable (for example, in the `.env` or `.env.local` file): + +``` yaml +SEARCH_ENGINE=elasticsearch +``` + +## Configure the search engine + +[[= product_name =]] comes pre-configured to work with an Elasticsearch cluster that uses default settings, and you can use this initial setup for testing purposes. +However, to effectively search through actual data, you must provide specific settings. +All configuration is made in the `/config/packages/ibexa_elasticsearch.yaml` file. + +!!! note + + All the settings, their order and meaning, correspond to the settings that are described in + the Elasticsearch documentation. + +First, decide how [[= product_name =]] connects to Elasticsearch and configure other connection settings. +For more information, see [Configuring connections](configure_elastic_search.md#connections). + +Then, define a Field Type mappings template that instructs Elasticsearch to interpret [[= product_name =]] fields as specific types. For more information, see [Configuring Field Type mappings](configure_elastic_search.md#field-type-mapping-templates). + +## Push the templates + +For each of your defined connections, push the templates to the Elasticsearch engine by running the following command: + +``` bash +php bin/console ibexa:elasticsearch:put-index-template +``` + +You can modify the behavior of the command with a number of switches. Use the `-h` switch to display a complete list of available options. + +## Reindex the database + +After creating index templates, run the following command to reindex your data: + +``` bash +php bin/console ibexa:reindex +``` + +!!! caution "Risks of premature indexing" + + Do not reindex your data before you create index templates. + Otherwise Elasticsearch attempts to use its [dynamic field mapping](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/dynamic-field-mapping.html) feature to create type mappings automatically. \ No newline at end of file diff --git a/docs/search/search_engines/legacy_search_engine/configure_repository.md b/docs/search/search_engines/legacy_search_engine/configure_repository.md new file mode 100644 index 0000000000..2f7de7dc66 --- /dev/null +++ b/docs/search/search_engines/legacy_search_engine/configure_repository.md @@ -0,0 +1,20 @@ +--- +description: Configure Legacy search engine to use it with Ibexa DXP. +--- + +# Configure repository with Legacy search engine + +Search can be configured independently from storage, and the following configuration example shows both the default values, +and how you configure legacy as the search engine: + +``` yaml +ibexa: + repositories: + main: + storage: + engine: legacy + connection: default + search: + engine: legacy + connection: default +``` diff --git a/docs/search/search_engines/legacy_search_engine/legacy_search_overview.md b/docs/search/search_engines/legacy_search_engine/legacy_search_overview.md new file mode 100644 index 0000000000..6618c5b330 --- /dev/null +++ b/docs/search/search_engines/legacy_search_engine/legacy_search_overview.md @@ -0,0 +1,19 @@ +--- +description: Legacy search engine overview. +--- + +# Legacy search engine + +Legacy search engine is the default search engine. It is SQL-based and uses Doctrine's database connection. +The connections are defined in the same way as for storage engine, and no further specific configuration is needed. +Legacy search engine is recommended for basic needs and is not intended in production. +It allows you to use filtering and fulltext search, but with some limitations. +For more information, check [search engine comparison](search_engines/#search-engine-comparison) + +!!! tip + + The features and performance of Legacy search engine are limited. + If you have specific search or performance needs, it is recommended to use [Solr](solr_overview.md) + or [Elasticsearch](elastic_search_overview.md) instead. + + Using the Legacy search engine disables most shop features, such as product search. diff --git a/docs/search/search_engines/search_engines.md b/docs/search/search_engines/search_engines.md new file mode 100644 index 0000000000..a710ddf405 --- /dev/null +++ b/docs/search/search_engines/search_engines.md @@ -0,0 +1,29 @@ +--- +description: Learn about different search engines that are supported by Ibexa DXP. +--- + +# Search engines + +[[= product_name =]] enables you to use different search engines. +Currently, they exist in their own [[= product_name =]] Bundles: + +1. [Legacy search engine](legacy_search_overview.md) - a database-powered search engine for basic needs. +1. [Solr](solr_overview.md) - an integration providing better overall performance, better scalability and support for more advanced search capabilities. +1. [Elasticsearch](elastic_search_overview.md) - a document-oriented engine providing even better performance and scalability. + +## Search engines comparison + +| Feature | Legacy Search Engine (SQL) | Solr | Elasticsearch | +| --- | --- | --- | --- | +| Filtering | Yes, limited\* | Yes | Yes | +| Query (filter with scoring) | Only filters, no scoring | Yes | Yes | +| Fulltext search | Yes, limited\*\* | Yes | Yes, limited | +| Index-time boosting | No | No | Query-time boosting\*\*\* | +| Aggregations | No | Yes | Yes | + +\* Usage of Criteria and Sort Clauses for Fields does not perform well on medium to larger +amount of data with Legacy Search Engine (SQL). + +\*\* For more information about fulltext search syntax support, see [Fulltext Criterion](fulltext_criterion.md). + +\*\*\* Elasticsearch offers query-time boosting instead. \ No newline at end of file diff --git a/docs/search/search_engines/solr_search_engine/configure_solr.md b/docs/search/search_engines/solr_search_engine/configure_solr.md new file mode 100644 index 0000000000..b1f58d206f --- /dev/null +++ b/docs/search/search_engines/solr_search_engine/configure_solr.md @@ -0,0 +1,244 @@ +--- +description: Configure Solr search engine to use it with Ibexa DXP. +--- + +# Configure Solr + +## Configure boosting + +!!! caution "Index time boosting" + + Index time boosting was deprecated in Solr 6.5 and removed in Solr 7.0. + Until query time boosting is implemented, there is no way to boost in the bundle out of the box. + +!!! tip "How boosting interacts with Search API" + + Boosting of fields or documents will affect the score (relevance) of your search result hits + when using Search API for any Criteria you specify on `$query->query`, or in REST by using `Query` element. + When you don't specify anything to sort on, the result will be sorted by this relevance. + Anything set on `$query->filter`, or in REST using `Filter` element, will *not* affect scoring and only works + as a pure filter for the result. Thus make sure to place Criteria you want to affect scoring on `query`. + +Boosting currently happens when indexing, so if you change your configuration you will need to re-index. + +Boosting tells the search engine which parts of the content model have more importance when searching, and is an important part of tuning your search results relevance. Importance is defined using a numeric value, where `1.0` is default, values higher than that are more important, and values lower (down to `0.0`) are less important. + +Boosting is configured per connection that you configure to use for a given Repository, like in this `config/packages/ibexa_solr.yaml` example: + +``` yaml +ibexa_solr: + connections: + default: + boost_factors: + content_type: + # Boost a whole Content Type + article: 2.0 + meta_field: + # Boost a meta Field (name, text) system wide, or for a given Content Type + name: 10.0 + article: + # Boost the meta full text Field for article more than 2.0 set above + text: 5.0 +``` + +The configuration above will result in the following boosting (Content Type / Field): + +- `article/title: 2.0` +- `news/description: 1.0` (default) +- `article/text (meta): 5.0` +- `blog_post/name (meta): 10.0` +- `article/name (meta): 2.0` + +!!! tip "How to configure boosting on specific fields" + + Currently, boosting on particular fields is missing. + However, it could be configured using 3rd party [Novactive/NovaeZSolrSearchExtraBundle](https://github.com/Novactive/NovaeZSolrSearchExtraBundle) in case of custom search implementation, e.g. to handle your front-end search form. + Unfortunately, this doesn't affect search performed in the administration interface. + + The following example presents boosting configuration for Folder's `name` and `description` fields. + First, in `ibexa_solr.yaml` configure [custom fulltext fields.](https://github.com/Novactive/NovaeZSolrSearchExtraBundle/blob/master/doc/custom_fields.md) + + ```yaml + ez_solr_search_extra: + system: + default: + fulltext_fields: + custom_folder_name: + - folder/name + custom_folder_description: + - folder/description + ``` + + The second step requires you to use `\Novactive\EzSolrSearchExtra\Query\Content\Criterion\MultipleFieldsFullText` instead of default `\Ibexa\Contracts\Core\Repository\Values\Content\Query\Criterion\FullText`. + The following example shows custom query which benefits from the custom fields created in the previous example. + + ```php + searchService = $searchService; + } + + public function searchAction(Request $request): Response + { + $queryString = $request->get('query'); + + $query = new Query(); + $query->query = new \Novactive\EzSolrSearchExtra\Query\Content\Criterion\MultipleFieldsFullText( + $queryString, + [ + 'metaBoost' => [ + 'custom_folder_name' => 20.0, + 'custom_folder_description' => 10.0, + ] + ] + ); + + $searchResult = $this->searchService->findContent($query); + + ... + } + } + ``` + + Remember to clear the cache and perform search engine reindex afterwords. + + The above configuration will result in the following boosting (Content Type / Field): + + - `folder/name: 20.0` + - `folder/description: 10.0` + +## Index related objects + +You can use indexation of related objects to search through text of related content. +Indexing is disabled by default. +To set it up you need to define the maximum indexing depth using the following YAML configuration: + +```yaml +ibexa_solr: + # ... + connections: + default: + # ... + indexing_depth: + # Default value: 0 - no relation indexing, 1 - direct relations, 2nd level relations, 3rd level relations (maximum value). + default: 1 + content_type: + # Index depth defined for specific content type + article: 2 +``` + +## Configure Solr Replication (master/slave) + +!!! note + + The configuration below has been tested on Solr 7.7. + +### Configure Master for replication + +First you need to change the core configuration in `solrconfig.xml` (for example `*/opt/solr/server/ibexa/collection1/conf/solrconfig.xml`). +You can copy and paste the code below before any other `requestHandler` section. + +```xml + + + optimize + optimize + schema.xml,stopwords.txt,elevate.xml + 00:00:10 + + 2 + + 16 + + +solrconfig_slave.xml:solrconfig.xml,x.xml,y.xml +``` + +Then restart the master with: + +```bash +sudo su - solr -c "/opt/solr/bin/solr restart" +``` + +### Configure Slave for replication + +You have to edit the same file on the slave server, and use the code below: + +```xml + + + + + http://123.456.789.0:8983/solr/collection1/replication + + + 00:00:20 + + + + internal + + + 5000 + 10000 + + + username + password + + +``` + +Next, restart Solr slave. + +Connect to the Solr slave interface (http://localhost:8983/solr), go to your core and check the replication status: + +![Solr Slave](solr.png) + +# Configure HTTP Client for Solr queries + +Ibexa Solr Bundle uses Symfony HTTP Client to fetch and update Solr index. +You can configure timeout and maximum number of retries for that client using Solr Bundle's Semantic configuration: + +```yaml +ibexa_solr: + # ... + http_client: + # ... + timeout: 30 + max_retries: 5 +``` + +# Extend Solr + +To learn how you can create document field mappers, custom Search Criteria, +custom Sort Clauses and Aggregations, see [Search extensibility](create_custom_search_criterion.md). diff --git a/docs/search/search_engines/solr_search_engine/install_solr.md b/docs/search/search_engines/solr_search_engine/install_solr.md new file mode 100644 index 0000000000..feda28b954 --- /dev/null +++ b/docs/search/search_engines/solr_search_engine/install_solr.md @@ -0,0 +1,295 @@ +--- +description: Install Solr search engine to use it with Ibexa DXP. +--- + +# Install Solr search engine + +## Configure and start Solr + +The example presents a configuration with a single core. +For configuring Solr in other ways, including examples, see [Solr Cores and `solr.xml`](https://cwiki.apache.org/confluence/display/solr/Solr+Cores+and+solr.xml) and [core administration](https://wiki.apache.org/solr/CoreAdmin). + +### Download Solr files + +!!! note "Solr versions" + + Supported Solr versions are Solr 7 and 8. Using most recent version of Solr 7.7 or 8.11 is recommended. + +Download and extract Solr: + +- [solr-7.7.2.tgz](http://archive.apache.org/dist/lucene/solr/7.7.2/solr-7.7.2.tgz) or [solr-7.7.2.zip](http://archive.apache.org/dist/lucene/solr/7.7.2/solr-7.7.2.zip) +- [solr-8.11.2.tgz](https://www.apache.org/dyn/closer.lua/lucene/solr/8.11.2/solr-8.11.2.tgz) or [solr-8.11.2.zip](https://www.apache.org/dyn/closer.lua/lucene/solr/8.11.2/solr-8.11.2.zip) + +Copy the necessary configuration files. In the example below from the root of your project to the place you extracted Solr: + +``` bash +# Make sure to replace the /opt/solr/ path with where you have placed Solr +cd /opt/solr +mkdir -p server/ibexa/template +cp -R /vendor/ibexa/solr/src/lib/Resources/config/solr/* server/ibexa/template +cp server/solr/configsets/_default/conf/{solrconfig.xml,stopwords.txt,synonyms.txt} server/ibexa/template +cp server/solr/solr.xml server/ibexa + +# If you are using Ibexa Commerce, additionally copy commerce-specific configuration files: +cat /vendor/ibexa/commerce-shop/src/bundle/Search/Resources/config/solr/custom-fields-types.xml >> server/ibexa/template/custom-fields-types.xml +cat /vendor/ibexa/commerce-shop/src/bundle/Search/Resources/config/solr/language-fieldtypes.xml >> server/ibexa/template/language-fieldtypes.xml + +# Modify solrconfig.xml to remove the section that doesn't agree with your schema +sed -i.bak '//d' server/ibexa/template/solrconfig.xml + +# Start Solr (but apply autocommit settings below first if you need to) +bin/solr -s ibexa +bin/solr create_core -c collection1 -d server/ibexa/template +``` + +#### Set up SolrCloud + +SolrCloud is a cluster of Solr servers. It enables you to: + +- centralize configuration +- automatically load balance and fail-over for queries +- integrate ZooKeeper for cluster coordination and configuration + +To set SolrCloud up follow [SolrCloud reference guide.](https://lucene.apache.org/solr/guide/7_7/solrcloud.html) + +### Continue Solr configuration + +The bundle does not commit Solr index changes directly on Repository updates, leaving it up to you to tune this using `solrconfig.xml` as best practice suggests. + +This setting is **required** if you want to see the changes after publish. It is strongly recommended to set-up `solrconfig.xml` like this: + +``` xml + + + + ${solr.autoCommit.maxTime:15000} + false + + + + + ${solr.autoSoftCommit.maxTime:20} + +``` + +### Generate Solr configuration automatically + +The command line tool `bin/generate-solr-config.sh` generates Solr 7 configuration automatically. +It can be used for deploying to Ibexa Cloud (Platform.sh) and on-premise installs. + +Execute the script from the [[= product_name =]] root directory for further information: + +``` bash +./vendor/ibexa/solr/bin/generate-solr-config.sh --help +``` + +## Configure the bundle + +The Solr Search Engine Bundle can be configured in many ways. The config further below assumes you have parameters set up for Solr DSN and search engine *(however both are optional)*, for example: + +``` yaml + env(SEARCH_ENGINE): solr + env(SOLR_DSN): 'http://localhost:8983/solr' + env(SOLR_CORE): collection1 +``` + +### Single-core example (default) + +Out of the box in [[= product_name =]] the following is enabled for a simple setup: + +``` yaml +ibexa_solr: + endpoints: + endpoint0: + dsn: '%solr_dsn%' + core: '%solr_core%' + connections: + default: + entry_endpoints: + - endpoint0 + mapping: + default: endpoint0 +``` + +### Shared-core example + +The following example separates one language. The installation contains several similar languages, +and one very different language that should receive proper language analysis for proper stemming and sorting behavior by Solr: + +``` yaml +ibexa_solr: + endpoints: + endpoint0: + dsn: '%solr_dsn%' + core: core0 + endpoint1: + dsn: '%solr_dsn%' + core: core1 + connections: + default: + entry_endpoints: + - endpoint0 + - endpoint1 + mapping: + translations: + jpn-JP: endpoint1 + # Other languages, for instance eng-US and other western languages are sharing core + default: endpoint0 +``` + +### Multi-core example + +If full language analysis features are preferred, then each language can be configured with separate cores. + +!!! note + + Make sure to test this setup against a single-core setup, as it might perform worse than single-core if your project uses a lot of language fallbacks per SiteAccess, as queries will then be performed across several cores at once. + +``` yaml +ibexa_solr: + endpoints: + endpoint0: + dsn: '%solr_dsn%' + core: core0 + endpoint1: + dsn: '%solr_dsn%' + core: core1 + endpoint2: + dsn: '%solr_dsn%' + core: core2 + endpoint3: + dsn: '%solr_dsn%' + core: core3 + endpoint4: + dsn: '%solr_dsn%' + core: core4 + endpoint5: + dsn: '%solr_dsn%' + core: core5 + endpoint6: + dsn: '%solr_dsn%' + core: core6 + connections: + default: + entry_endpoints: + - endpoint0 + - endpoint1 + - endpoint2 + - endpoint3 + - endpoint4 + - endpoint5 + - endpoint6 + mapping: + translations: + jpn-JP: endpoint1 + eng-US: endpoint2 + fre-FR: endpoint3 + ger-DE: endpoint4 + esp-ES: endpoint5 + # Not really used, but specified here for fallback if more languages are suddenly added by content admins + default: endpoint0 + # Also use separate core for main languages (differs from content object to content object) + # This is useful to reduce number of cores queried for always available language fallbacks + main_translations: endpoint6 +``` + +### SolrCloud example + +To use SolrCloud you need to specify data distribution strategy for connection via the `distribution_strategy` option to [`cloud`.](https://lucene.apache.org/solr/guide/7_7/solrcloud.html) + +The example is based on multi-core setup so any specific language analysis options could be specified on the collection level. + +``` yaml +ibexa_solr: + endpoints: + main: + dsn: '%solr_dsn%' + core: '%solr_main_core%' + en: + dsn: '%solr_dsn%' + core: '%solr_en_core%' + fr: + dsn: '%solr_dsn%' + core: '%solr_fr_core%' + # ... + connections: + default: + distribution_strategy: cloud + entry_endpoints: + - main + - en + - fr + # - ... + mapping: + translations: + eng-GB: en + fre-FR: fr + # ... + main_translations: main +``` + +This solution uses the default SolrCloud [document routing strategy: `compositeId`.](https://lucene.apache.org/solr/guide/7_7/shards-and-indexing-data-in-solrcloud.html#ShardsandIndexingDatainSolrCloud-DocumentRouting) + +### Solr Basic HTTP Authorization +Solr core can be secured with Basic HTTP Authorization. See more information here: [Solr Basic Authentication Plugin.](https://cwiki.apache.org/confluence/display/solr/Basic+Authentication+Plugin) +In the example below we configured Solr Bundle to work with secured Solr core. + +``` yaml +ibexa_solr: + endpoints: + endpoint0: + dsn: '%solr_dsn%' + core: core0 + user: example + pass: password +``` + +Obviously, you should pass credentials for every configured and HTTP Basic secured Solr core. Configuration for multi core setup is exactly the same. + +## Configure repository with the specific search engine + +The following is an example of configuring Solr search engine, where `connection` name is same as in the example above, and engine is set to `solr`: + +``` yaml +ibexa: + repositories: + default: + storage: ~ + search: + engine: '%search_engine%' + connection: default +``` + +`%search_engine%` is a parameter that is configured in `config/packages/ibexa.yaml`, and should be changed from its default value `legacy` to `solr` to activate Solr as the search engine. + +## Clear prod cache + +While Symfony `dev` environment keeps track of changes to YAML files, `prod` does not, so clear the cache to make sure Symfony reads the new config: + +``` bash +php bin/console --env=prod cache:clear +``` + +## Run CLI indexing command + +The last step is to execute the initial indexation of data: + +``` bash +php bin/console --env=prod --siteaccess= ibexa:reindex +``` + +### Possible exceptions + +If you have not configured your setup correctly, some exceptions might happen on indexing. +Here are the most common issues you may encounter: + +- Exception if Binary files in database have an invalid path prefix + - Make sure `var_dir` is configured properly in `ibexa.yaml` configuration. + - If your database is inconsistent in regards to file paths, try to update entries to be correct *(make sure to make a backup first)*. +- Exception on unsupported Field Types + - Make sure to implement all Field Types in your installation, or to configure missing ones as [NullType](nullfield.md) if implementation is not needed. +- Content is not immediately available  + - Solr Bundle on purpose does not commit changes directly on Repository updates *(on indexing)*, but lets you control this using Solr configuration. Adjust Solr's `autoSoftCommit` (visibility of changes to search index) and/or `autoCommit` (hard commit, for durability and replication) to balance performance and load on your Solr instance against needs you have for "[NRT](https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching)". +- Running out of memory during indexing + - In general make sure to run indexing using the prod environment to avoid debuggers and loggers from filling up memory. + - Flysystem: You can find further info in https://jira.ez.no/browse/EZP-25325. \ No newline at end of file diff --git a/docs/search/search_engines/solr_search_engine/solr_overview.md b/docs/search/search_engines/solr_search_engine/solr_overview.md new file mode 100644 index 0000000000..42e8244d9a --- /dev/null +++ b/docs/search/search_engines/solr_search_engine/solr_overview.md @@ -0,0 +1,13 @@ +--- +description: Solr search engine overview. +--- + +# Solr search engine + +[Solr search engine](https://github.com/ibexa/solr) allows you to use advanced search features: filtering, query, fulltext search and aggregations. + +When you enable Solr and re-index your content, all your existing Search queries using `SearchService` will be powered by Solr automatically. +This allows you to scale up your [[= product_name =]] installation and be able to continue development locally against SQL engine, and have a test infrastructure, Staging and Prod powered by Solr. +By this, it also removes considerable load from your database. + +For further information on the architecture of Ibexa DXP, see [Architecture](architecture.md). \ No newline at end of file diff --git a/docs/search/solr_search_engine.md b/docs/search/solr_search_engine.md deleted file mode 100644 index ca1f2c48fc..0000000000 --- a/docs/search/solr_search_engine.md +++ /dev/null @@ -1,539 +0,0 @@ ---- -description: Configure Solr search engine to use with Ibexa DXP. ---- - -# Solr search engine - -[ibexa/solr](https://github.com/ibexa/solr) aims to be a transparent drop-in replacement for the SQL-based Legacy search engine powering [[= product_name =]] Search API by default. When you enable Solr and re-index your content, all your existing Search queries using `SearchService` will be powered by Solr automatically. This allows you to scale up your [[= product_name =]] installation and be able to continue development locally against SQL engine, and have a test infrastructure, Staging and Prod powered by Solr. This removes considerable load from your database. See [further information on the architecture of Ibexa DXP](architecture.md). - -## Set up Solr search engine - -### Step 1: Configure and start Solr - -The example presents a configuration with a single core. For configuring Solr in other ways, including examples, see [Solr Cores and `solr.xml`](https://cwiki.apache.org/confluence/display/solr/Solr+Cores+and+solr.xml) and [core administration](https://wiki.apache.org/solr/CoreAdmin). - -#### Download and configure - -!!! note "Solr versions" - - Supported Solr versions are Solr 7 and 8. Using most recent version of Solr 7.7 or 8.11 is recommended. - -Download and extract Solr: - -- [solr-7.7.2.tgz](http://archive.apache.org/dist/lucene/solr/7.7.2/solr-7.7.2.tgz) or [solr-7.7.2.zip](http://archive.apache.org/dist/lucene/solr/7.7.2/solr-7.7.2.zip) -- [solr-8.11.2.tgz](https://www.apache.org/dyn/closer.lua/lucene/solr/8.11.2/solr-8.11.2.tgz) or [solr-8.11.2.zip](https://www.apache.org/dyn/closer.lua/lucene/solr/8.11.2/solr-8.11.2.zip) - -Copy the necessary configuration files. In the example below from the root of your project to the place you extracted Solr: - -``` bash -# Make sure to replace the /opt/solr/ path with where you have placed Solr -cd /opt/solr -mkdir -p server/ibexa/template -cp -R /vendor/ibexa/solr/src/lib/Resources/config/solr/* server/ibexa/template -cp server/solr/configsets/_default/conf/{solrconfig.xml,stopwords.txt,synonyms.txt} server/ibexa/template -cp server/solr/solr.xml server/ibexa - -# If you are using Ibexa Commerce, additionally copy commerce-specific configuration files: -cat /vendor/ibexa/commerce-shop/src/bundle/Search/Resources/config/solr/custom-fields-types.xml >> server/ibexa/template/custom-fields-types.xml -cat /vendor/ibexa/commerce-shop/src/bundle/Search/Resources/config/solr/language-fieldtypes.xml >> server/ibexa/template/language-fieldtypes.xml - -# Modify solrconfig.xml to remove the section that doesn't agree with your schema -sed -i.bak '//d' server/ibexa/template/solrconfig.xml - -# Start Solr (but apply autocommit settings below first if you need to) -bin/solr -s ibexa -bin/solr create_core -c collection1 -d server/ibexa/template -``` - -##### SolrCloud - -SolrCloud is a cluster of Solr servers. It enables you to: - -- centralize configuration -- automatically load balance and fail-over for queries -- integrate ZooKeeper for cluster coordination and configuration - -To set SolrCloud up follow [SolrCloud reference guide.](https://lucene.apache.org/solr/guide/7_7/solrcloud.html) - -#### Further configuration - -The bundle does not commit Solr index changes directly on Repository updates, leaving it up to you to tune this using `solrconfig.xml` as best practice suggests. - -This setting is **required** if you want to see the changes after publish. It is strongly recommended to set-up `solrconfig.xml` like this: - -``` xml - - - - ${solr.autoCommit.maxTime:15000} - false - - - - - ${solr.autoSoftCommit.maxTime:20} - -``` - -#### Generating configuration - -The command line tool `bin/generate-solr-config.sh` generates Solr 7 configuration automatically. -It can be used for deploying to Ibexa Cloud (Platform.sh) and on-premise installs. - -Execute the script from the [[= product_name =]] root directory for further information: - -``` bash -./vendor/ibexa/solr/bin/generate-solr-config.sh --help -``` - -### Step 2: Configure the bundle - -The Solr Search Engine Bundle can be configured in many ways. The config further below assumes you have parameters set up for Solr DSN and search engine *(however both are optional)*, for example: - -``` yaml - env(SEARCH_ENGINE): solr - env(SOLR_DSN): 'http://localhost:8983/solr' - env(SOLR_CORE): collection1 -``` - -#### Single-core example (default) - -Out of the box in [[= product_name =]] the following is enabled for a simple setup under the `ibexa_solr` [configuration key](configuration.md#configuration-files): - -``` yaml -ibexa_solr: - endpoints: - endpoint0: - dsn: '%solr_dsn%' - core: '%solr_core%' - connections: - default: - entry_endpoints: - - endpoint0 - mapping: - default: endpoint0 -``` - -#### Shared-core example - -The following example separates one language. The installation contains several similar languages, -and one very different language that should receive proper language analysis for proper stemming and sorting behavior by Solr: - -``` yaml -ibexa_solr: - endpoints: - endpoint0: - dsn: '%solr_dsn%' - core: core0 - endpoint1: - dsn: '%solr_dsn%' - core: core1 - connections: - default: - entry_endpoints: - - endpoint0 - - endpoint1 - mapping: - translations: - jpn-JP: endpoint1 - # Other languages, for instance eng-US and other western languages are sharing core - default: endpoint0 -``` - -#### Multi-core example - -If full language analysis features are preferred, then each language can be configured with separate cores. - -!!! note - - Make sure to test this setup against a single-core setup, as it might perform worse than single-core if your project uses a lot of language fallbacks per SiteAccess, as queries will then be performed across several cores at once. - -``` yaml -ibexa_solr: - endpoints: - endpoint0: - dsn: '%solr_dsn%' - core: core0 - endpoint1: - dsn: '%solr_dsn%' - core: core1 - endpoint2: - dsn: '%solr_dsn%' - core: core2 - endpoint3: - dsn: '%solr_dsn%' - core: core3 - endpoint4: - dsn: '%solr_dsn%' - core: core4 - endpoint5: - dsn: '%solr_dsn%' - core: core5 - endpoint6: - dsn: '%solr_dsn%' - core: core6 - connections: - default: - entry_endpoints: - - endpoint0 - - endpoint1 - - endpoint2 - - endpoint3 - - endpoint4 - - endpoint5 - - endpoint6 - mapping: - translations: - jpn-JP: endpoint1 - eng-US: endpoint2 - fre-FR: endpoint3 - ger-DE: endpoint4 - esp-ES: endpoint5 - # Not really used, but specified here for fallback if more languages are suddenly added by content admins - default: endpoint0 - # Also use separate core for main languages (differs from content object to content object) - # This is useful to reduce number of cores queried for always available language fallbacks - main_translations: endpoint6 -``` - -#### SolrCloud example - -To use SolrCloud you need to specify data distribution strategy for connection via the `distribution_strategy` option to [`cloud`.](https://lucene.apache.org/solr/guide/7_7/solrcloud.html) - -The example is based on multi-core setup so any specific language analysis options could be specified on the collection level. - -``` yaml -ibexa_solr: - endpoints: - main: - dsn: '%solr_dsn%' - core: '%solr_main_core%' - en: - dsn: '%solr_dsn%' - core: '%solr_en_core%' - fr: - dsn: '%solr_dsn%' - core: '%solr_fr_core%' - # ... - connections: - default: - distribution_strategy: cloud - entry_endpoints: - - main - - en - - fr - # - ... - mapping: - translations: - eng-GB: en - fre-FR: fr - # ... - main_translations: main -``` - -This solution uses the default SolrCloud [document routing strategy: `compositeId`.](https://lucene.apache.org/solr/guide/7_7/shards-and-indexing-data-in-solrcloud.html#ShardsandIndexingDatainSolrCloud-DocumentRouting) - -#### Solr Basic HTTP Authorization -Solr core can be secured with Basic HTTP Authorization. See more information here: [Solr Basic Authentication Plugin.](https://cwiki.apache.org/confluence/display/solr/Basic+Authentication+Plugin) -In the example below we configured Solr Bundle to work with secured Solr core. - -``` yaml -ibexa_solr: - endpoints: - endpoint0: - dsn: '%solr_dsn%' - core: core0 - user: example - pass: password -``` - -Obviously, you should pass credentials for every configured and HTTP Basic secured Solr core. Configuration for multi core setup is exactly the same. - -### Step 3: Configure repository with the specific search engine - -The following is an example of configuring Solr search engine, where `connection` name is same as in the example above, and engine is set to `solr`: - -``` yaml -ibexa: - repositories: - default: - storage: ~ - search: - engine: '%search_engine%' - connection: default -``` - -`%search_engine%` is a parameter that is configured in `config/packages/ibexa.yaml`, and should be changed from its default value `legacy` to `solr` to activate Solr as the search engine. - -### Step 4: Clear prod cache - -While Symfony `dev` environment keeps track of changes to YAML files, `prod` does not, so clear the cache to make sure Symfony reads the new config: - -``` bash -php bin/console --env=prod cache:clear -``` - -### Step 5: Run CLI indexing command - -The last step is to execute the initial indexation of data: - -``` bash -php bin/console --env=prod --siteaccess= ibexa:reindex -``` - -#### Possible exceptions - -If you have not configured your setup correctly, some exceptions might happen on indexing. -Here are the most common issues you may encounter: - -- Exception if Binary files in database have an invalid path prefix - - Make sure `var_dir` is configured properly in `ibexa.yaml` configuration. - - If your database is inconsistent in regards to file paths, try to update entries to be correct *(make sure to make a backup first)*. -- Exception on unsupported Field Types - - Make sure to implement all Field Types in your installation, or to configure missing ones as [NullType](nullfield.md) if implementation is not needed. -- Content is not immediately available  - - Solr Bundle on purpose does not commit changes directly on Repository updates *(on indexing)*, but lets you control this using Solr configuration. Adjust Solr's `autoSoftCommit` (visibility of changes to search index) and/or `autoCommit` (hard commit, for durability and replication) to balance performance and load on your Solr instance against needs you have for "[NRT](https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching)". -- Running out of memory during indexing - - In general make sure to run indexing using the prod environment to avoid debuggers and loggers from filling up memory. - - Flysystem: You can find further info in https://jira.ez.no/browse/EZP-25325. - -## Solr configuration - -### Boost configuration - -!!! caution "Index time boosting" - - Index time boosting was deprecated in Solr 6.5 and removed in Solr 7.0. - Until query time boosting is implemented, there is no way to boost in the bundle out of the box. - -!!! tip "How boosting interacts with Search API" - - Boosting of fields or documents will affect the score (relevance) of your search result hits - when using Search API for any Criteria you specify on `$query->query`, or in REST by using `Query` element. - When you don't specify anything to sort on, the result will be sorted by this relevance. - Anything set on `$query->filter`, or in REST using `Filter` element, will *not* affect scoring and only works - as a pure filter for the result. Thus make sure to place Criteria you want to affect scoring on `query`. - -Boosting currently happens when indexing, so if you change your configuration you will need to re-index. - -Boosting tells the search engine which parts of the content model have more importance when searching, and is an important part of tuning your search results relevance. Importance is defined using a numeric value, where `1.0` is default, values higher than that are more important, and values lower (down to `0.0`) are less important. - -Boosting is configured per connection that you configure to use for a given Repository, like in this `config/packages/ibexa_solr.yaml` example: - -``` yaml -ibexa_solr: - connections: - default: - boost_factors: - content_type: - # Boost a whole Content Type - article: 2.0 - meta_field: - # Boost a meta Field (name, text) system wide, or for a given Content Type - name: 10.0 - article: - # Boost the meta full text Field for article more than 2.0 set above - text: 5.0 -``` - -The configuration above will result in the following boosting (Content Type / Field): - -- `article/title: 2.0` -- `news/description: 1.0` (default) -- `article/text (meta): 5.0` -- `blog_post/name (meta): 10.0` -- `article/name (meta): 2.0` - -!!! tip "How to configure boosting on specific fields" - - Currently, boosting on particular fields is missing. - However, it could be configured using 3rd party [Novactive/NovaeZSolrSearchExtraBundle](https://github.com/Novactive/NovaeZSolrSearchExtraBundle) in case of custom search implementation, e.g. to handle your front-end search form. - Unfortunately, this doesn't affect search performed in the administration interface. - - The following example presents boosting configuration for Folder's `name` and `description` fields. - First, in `ibexa_solr.yaml` configure [custom fulltext fields.](https://github.com/Novactive/NovaeZSolrSearchExtraBundle/blob/master/doc/custom_fields.md) - - ```yaml - ez_solr_search_extra: - system: - default: - fulltext_fields: - custom_folder_name: - - folder/name - custom_folder_description: - - folder/description - ``` - - The second step requires you to use `\Novactive\EzSolrSearchExtra\Query\Content\Criterion\MultipleFieldsFullText` instead of default `\Ibexa\Contracts\Core\Repository\Values\Content\Query\Criterion\FullText`. - The following example shows custom query which benefits from the custom fields created in the previous example. - - ```php - searchService = $searchService; - } - - public function searchAction(Request $request): Response - { - $queryString = $request->get('query'); - - $query = new Query(); - $query->query = new \Novactive\EzSolrSearchExtra\Query\Content\Criterion\MultipleFieldsFullText( - $queryString, - [ - 'metaBoost' => [ - 'custom_folder_name' => 20.0, - 'custom_folder_description' => 10.0, - ] - ] - ); - - $searchResult = $this->searchService->findContent($query); - - ... - } - } - ``` - - Remember to clear the cache and perform search engine reindex afterwords. - - The above configuration will result in the following boosting (Content Type / Field): - - - `folder/name: 20.0` - - `folder/description: 10.0` - -### Indexing related objects - -You can use indexation of related objects to search through text of related content. -Indexing is disabled by default. -To set it up you need to define the maximum indexing depth using the following YAML configuration: - -```yaml -ibexa_solr: - # ... - connections: - default: - # ... - indexing_depth: - # Default value: 0 - no relation indexing, 1 - direct relations, 2nd level relations, 3rd level relations (maximum value). - default: 1 - content_type: - # Index depth defined for specific content type - article: 2 -``` - -### Configuring Solr Replication (master/slave) - -!!! note - - The configuration below has been tested on Solr 7.7. - -#### Configuring Master for replication - -First you need to change the core configuration in `solrconfig.xml` (for example `*/opt/solr/server/ibexa/collection1/conf/solrconfig.xml`). -You can copy and paste the code below before any other `requestHandler` section. - -```xml - - - optimize - optimize - schema.xml,stopwords.txt,elevate.xml - 00:00:10 - - 2 - - 16 - - -solrconfig_slave.xml:solrconfig.xml,x.xml,y.xml -``` - -Then restart the master with: - -```bash -sudo su - solr -c "/opt/solr/bin/solr restart" -``` - -#### Configuring Slave for replication - -You have to edit the same file on the slave server, and use the code below: - -```xml - - - - - http://123.456.789.0:8983/solr/collection1/replication - - - 00:00:20 - - - - internal - - - 5000 - 10000 - - - username - password - - -``` - -Next, restart Solr slave. - -Connect to the Solr slave interface (http://localhost:8983/solr), go to your core and check the replication status: - -![Solr Slave](solr.png) - -## Configuring HTTP Client for Solr queries - -Ibexa Solr Bundle uses Symfony HTTP Client to fetch and update Solr index. -You can configure timeout and maximum number of retries for that client using Solr Bundle's Semantic configuration: - -```yaml -ibexa_solr: - # ... - http_client: - # ... - timeout: 30 - max_retries: 5 -``` - -## Extending Solr - -To learn how you can create document field mappers, custom Search Criteria, -custom Sort Clauses and Aggregations, see [Search extensibility](create_custom_search_criterion.md). diff --git a/docs/search/url_search_reference/url_search_reference.md b/docs/search/url_search_reference/url_search_criteria.md similarity index 67% rename from docs/search/url_search_reference/url_search_reference.md rename to docs/search/url_search_reference/url_search_criteria.md index 7b5a85d8f6..769b74c0c9 100644 --- a/docs/search/url_search_reference/url_search_reference.md +++ b/docs/search/url_search_reference/url_search_criteria.md @@ -3,9 +3,9 @@ description: URL Search Criteria help define and fine-tune search queries for UR page_type: reference --- -# URL search reference +# URL Search Criteria reference -## URL criteria reference +URL Search Criteria are part of Search Criteria and they are supported only by URL Search. |URL criteria|URL based on| |------------|------------| @@ -18,17 +18,4 @@ page_type: reference |[SectionId](sectionid_url_criterion.md)|Matches URLs from content placed in the Section with the specified ID.| |[SectionIdentifier](sectionidentifier_url_criterion.md)|Matches URLs from content placed in Sections with the specified identifiers.| |[Validity](validity_url_criterion.md)|Matches URLs based on validity flag.| -|[VisibleOnly](visibleonly_url_criterion.md)|Matches URLs from published content.| - -## URL Sort Clauses reference - -Sort Clauses are the sorting options for URLs. - -All Sort Clauses can take the following optional argument: - -- `sortDirection` - the direction of the sorting, either `\Ibexa\Contracts\Core\Repository\Values\URL\Query\SortClause::SORT_ASC` (default) or `\Ibexa\Contracts\Core\Repository\Values\URL\Query\SortClause::SORT_DESC` - -| Sort Clause | Sorting based on | -|-----|-----| -|[Id](id_url_sort_clause.md)|URL ID| -|[URL](url_url_sort_clause.md)|URL address| +|[VisibleOnly](visibleonly_url_criterion.md)|Matches URLs from published content.| \ No newline at end of file diff --git a/docs/search/url_search_reference/url_search_sort_clauses.md b/docs/search/url_search_reference/url_search_sort_clauses.md new file mode 100644 index 0000000000..c2e4fec50d --- /dev/null +++ b/docs/search/url_search_reference/url_search_sort_clauses.md @@ -0,0 +1,17 @@ +--- +description: URL Sort Clauses +page_type: reference +--- + +# URL Sort Clauses + +URL Sort Clauses are the sorting options for URLs. + +All URL Sort Clauses can take the following optional argument: + +- `sortDirection` - the direction of the sorting, either `\Ibexa\Contracts\Core\Repository\Values\URL\Query\SortClause::SORT_ASC` (default) or `\Ibexa\Contracts\Core\Repository\Values\URL\Query\SortClause::SORT_DESC` + +| Sort Clause | Sorting based on | +|-----|-----| +|[Id](id_url_sort_clause.md)|URL ID| +|[URL](url_url_sort_clause.md)|URL address| diff --git a/docs/users/login_and_registration.md b/docs/users/login_and_registration.md deleted file mode 100644 index 2201044e8c..0000000000 --- a/docs/users/login_and_registration.md +++ /dev/null @@ -1,91 +0,0 @@ ---- -description: Configure registering and activating customers. ---- - -# Login and registration - -## Login - -In [[= product_name =]], users can log in with their user name or email. - -## Registration - -[[= product_name =]] provides different registration options for private and business customers. - -### Private customers - -A private customer can register directly via the`/register/private` route. - -A double opt-in process checks the email address, and then creates and activates the user. - -#### Activation link recipient - -You can decide that emails with the activation link are not sent to the customer, but to a different email address. - -``` yaml -ibexa.commerce.site_access.config.core.default.user_activation_receiver: supervisor@example.com -``` - -The email message contains the name of the user and a link to the user in the Back Office. -To generate a correct link to the Back Office, configure `related_admin_site_access`: - -``` yaml -ibexa.commerce.site_access.config.core.default.related_admin_site_access: 'admin' -``` - -To adapt the success message of the private registration, modify the text module with the identifier `success_register_private`. - -### Business customers - -A business customer has two options to register: - -1. Apply for a new account - fill the business form and apply for an account via the `/registration/choice` route. - -1. Activate an account - a business customer who already has a customer number can register using a customer number and an invoice number. -The shop checks this data. There are two options: - - - activate a business account - the customer is created using their customer number and can immediately see their special discounts in the shop. - - in [[= product_name_com =]], create the main contact in Customer Center. If Customer Center is enabled, the company is created in the shop, and the account is created as the main contact. - -## Configuration - -You can specify the default Location ID for Users per SiteAccess: - -``` yaml -ibexa.commerce.site_access.config.core.default.user_group_location: 106 -ibexa.commerce.site_access.config.core.default.user_group_location.business: 106 -ibexa.commerce.site_access.config.core.default.user_group_location.private: 106 -ibexa.commerce.site_access.config.core.default.user_group_location.editor: 14 -``` - -`redirect_homepage` configures the default list of URLs from which the user is redirected after login. - -``` yaml -ibexa.commerce.site_access.config.core.default.redirect_homepage: - - /login - - /register - - /registration - - /password - - /token - - /change_password -``` - -### Account activation email - -When the `info_email_after_user_activation` parameter is set to true, -the customer receives an email when the account is enabled using the activation link. - -``` yaml -ibexa.commerce.site_access.config.core.default.info_email_after_user_activation: false -``` - -## Template list - -| Path | Description | -| -------- | ------------------ | -| `Eshop/Resources/views/Forms/register_private.html.twig` | Form for private customer registration | -| `Eshop/Resources/views/Forms/register_business.html.twig` | Form for business customer registration | -| `Eshop/Resources/views/Forms/register_choice.html.twig` | Overview page for registration, which offers buttons for the different registration types (and activation of existing customers) | -| `Eshop/Resources/views/Forms/activate_business.html.twig` | Form for activating existing customers | -| `Eshop/Resources/views/Checkout/checkout_login.html.twig` | Login form in checkout | -| `Eshop/Resources/views/Security/login.html.twig` | Login form | diff --git a/mkdocs.yml b/mkdocs.yml index a8376f2b8f..fcc05a2dd9 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -399,11 +399,20 @@ nav: - CDP activation: cdp/cdp_activation.md - Search: - Search: search/search.md - - Search engines: search/search_engines.md - - Elasticsearch search engine: search/elasticsearch_search_engine.md - - Solr search engine: search/solr_search_engine.md + - Search engines: + - Search engines: search/search_engines/search_engines.md + - Elasticsearch search engine: + - Overview: search/search_engines/elastic_search/elastic_search_overview.md + - Install Elasticsearch: search/search_engines/elastic_search/install_elastic_search.md + - Configure Elasticsearch: search/search_engines/elastic_search/configure_elastic_search.md + - Solr search engine: + - Overview: search/search_engines/solr_search_engine/solr_overview.md + - Install Solr: search/search_engines/solr_search_engine/install_solr.md + - Configure Solr: search/search_engines/solr_search_engine/configure_solr.md + - Legacy search engine: + - Overview: search/search_engines/legacy_search_engine/legacy_search_overview.md + - Configure repository: search/search_engines/legacy_search_engine/configure_repository.md - Search Criteria and Sort Clauses: search/search_criteria_and_sort_clauses.md - - Search API: search/search_api.md - Search Criteria reference: - Search Criteria reference: search/criteria_reference/search_criteria_reference.md - Ancestor: search/criteria_reference/ancestor_criterion.md @@ -513,64 +522,84 @@ nav: - ShippingMethod: search/criteria_reference/shipment_shipping_method_criterion.md - StatusCriterion: search/criteria_reference/shipment_status_criterion.md - UpdatedAtCriterion: search/criteria_reference/shipment_updatedat_criterion.md + - URL Search Criteria: + - URL Search Criteria: search/url_search_reference/url_search_criteria.md + - MatchAll Criterion: search/url_search_reference/matchall_url_criterion.md + - MatchNone Criterion: search/url_search_reference/matchnone_url_criterion.md + - Pattern Criterion: search/url_search_reference/pattern_url_criterion.md + - SectionId Criterion: search/url_search_reference/sectionid_url_criterion.md + - SectionIdentifier Criterion: search/url_search_reference/sectionidentifier_url_criterion.md + - Validity Criterion: search/url_search_reference/validity_url_criterion.md + - VisibleOnly Criterion: search/url_search_reference/visibleonly_url_criterion.md + - LogicalAnd Criterion: search/url_search_reference/logicaland_url_criterion.md + - LogicalNot Criterion: search/url_search_reference/logicalnot_url_criterion.md + - LogicalOr Criterion: search/url_search_reference/logicalor_url_criterion.md - Sort Clause reference: - - Sort Clause reference: search/sort_clause_reference/sort_clause_reference.md - - ContentId: search/sort_clause_reference/contentid_sort_clause.md - - ContentName: search/sort_clause_reference/contentname_sort_clause.md - - ContentTranslatedName: search/sort_clause_reference/contenttranslatedname_sort_clause.md - - ContentTypeName: search/sort_clause_reference/contenttypename_sort_clause.md - - CustomField: search/sort_clause_reference/customfield_sort_clause.md - - DateModified: search/sort_clause_reference/datemodified_sort_clause.md - - DatePublished: search/sort_clause_reference/datepublished_sort_clause.md - - DateTrashed: search/sort_clause_reference/datetrashed_sort_clause.md - - Depth: search/sort_clause_reference/depth_sort_clause.md - - Field: search/sort_clause_reference/field_sort_clause.md - - Id: search/sort_clause_reference/id_sort_clause.md - - IsMainLocation: search/sort_clause_reference/ismainlocation_sort_clause.md - - MapLocationDistance: search/sort_clause_reference/maplocationdistance_sort_clause.md - - Path: search/sort_clause_reference/path_sort_clause.md - - Priority: search/sort_clause_reference/priority_sort_clause.md - - Random: search/sort_clause_reference/random_sort_clause.md - - Score: search/sort_clause_reference/score_sort_clause.md - - SectionIdentifier: search/sort_clause_reference/sectionidentifier_sort_clause.md - - SectionName: search/sort_clause_reference/sectionname_sort_clause.md - - UserLogin: search/sort_clause_reference/userlogin_sort_clause.md - - Visibility: search/sort_clause_reference/visibility_sort_clause.md - - Product Sort Clauses: - - Product Sort Clauses: search/sort_clause_reference/product_sort_clauses.md - - BasePrice: search/sort_clause_reference/baseprice_sort_clause.md - - CreatedAt: search/sort_clause_reference/createdat_sort_clause.md - - CustomPrice: search/sort_clause_reference/customprice_sort_clause.md - - ProductAvailability: search/sort_clause_reference/productavailability_sort_clause.md - - ProductCode: search/sort_clause_reference/productcode_sort_clause.md - - ProductName: search/sort_clause_reference/productname_sort_clause.md - - Order Sort Clauses: - - Order Sort Clauses: search/sort_clause_reference/order_sort_clauses.md - - Id: search/sort_clause_reference/order_id_sort_clause.md - - Created: search/sort_clause_reference/order_created_sort_clause.md - - Updated: search/sort_clause_reference/order_updated_sort_clause.md - - Status: search/sort_clause_reference/order_status_sort_clause.md - - Payment Sort Clauses: - - Payment Sort Clauses: search/sort_clause_reference/payment_sort_clauses.md - - Id: search/sort_clause_reference/payment_id_sort_clause.md - - Identifier: search/sort_clause_reference/payment_identifier_sort_clause.md - - CreatedAt: search/sort_clause_reference/payment_createdat_sort_clause.md - - UpdatedAt: search/sort_clause_reference/payment_updatedat_sort_clause.md - - Status: search/sort_clause_reference/payment_status_sort_clause.md - - Payment Method Sort Clauses: - - Payment Method Sort Clauses: search/sort_clause_reference/payment_method_sort_clauses.md - - CreatedAt: search/sort_clause_reference/payment_method_createdat_sort_clause.md - - Enabled: search/sort_clause_reference/payment_method_enabled_sort_clause.md - - Id: search/sort_clause_reference/payment_method_id_sort_clause.md - - Identifier: search/sort_clause_reference/payment_method_identifier_sort_clause.md - - UpdatedAt: search/sort_clause_reference/payment_method_updatedat_sort_clause.md - - Shipment Sort Clauses: - - Shipment Sort Clauses: search/sort_clause_reference/shipment_sort_clauses.md - - Id: search/sort_clause_reference/shipment_id_sort_clause.md - - Identifier: search/sort_clause_reference/shipment_identifier_sort_clause.md - - CreatedAt: search/sort_clause_reference/shipment_createdat_sort_clause.md - - UpdatedAt: search/sort_clause_reference/shipment_updatedat_sort_clause.md - - Status: search/sort_clause_reference/shipment_status_sort_clause.md + - General Sort Clauses: + - General Sort Clause reference: search/sort_clause_reference/sort_clause_reference.md + - ContentId: search/sort_clause_reference/contentid_sort_clause.md + - ContentName: search/sort_clause_reference/contentname_sort_clause.md + - ContentTranslatedName: search/sort_clause_reference/contenttranslatedname_sort_clause.md + - ContentTypeName: search/sort_clause_reference/contenttypename_sort_clause.md + - CustomField: search/sort_clause_reference/customfield_sort_clause.md + - DateModified: search/sort_clause_reference/datemodified_sort_clause.md + - DatePublished: search/sort_clause_reference/datepublished_sort_clause.md + - DateTrashed: search/sort_clause_reference/datetrashed_sort_clause.md + - Depth: search/sort_clause_reference/depth_sort_clause.md + - Field: search/sort_clause_reference/field_sort_clause.md + - Id: search/sort_clause_reference/id_sort_clause.md + - IsMainLocation: search/sort_clause_reference/ismainlocation_sort_clause.md + - MapLocationDistance: search/sort_clause_reference/maplocationdistance_sort_clause.md + - Path: search/sort_clause_reference/path_sort_clause.md + - Priority: search/sort_clause_reference/priority_sort_clause.md + - Random: search/sort_clause_reference/random_sort_clause.md + - Score: search/sort_clause_reference/score_sort_clause.md + - SectionIdentifier: search/sort_clause_reference/sectionidentifier_sort_clause.md + - SectionName: search/sort_clause_reference/sectionname_sort_clause.md + - UserLogin: search/sort_clause_reference/userlogin_sort_clause.md + - Visibility: search/sort_clause_reference/visibility_sort_clause.md + - Product Sort Clauses: + - Product Sort Clauses: search/sort_clause_reference/product_sort_clauses.md + - BasePrice: search/sort_clause_reference/baseprice_sort_clause.md + - CreatedAt: search/sort_clause_reference/createdat_sort_clause.md + - CustomPrice: search/sort_clause_reference/customprice_sort_clause.md + - ProductAvailability: search/sort_clause_reference/productavailability_sort_clause.md + - ProductStock: search/criteria_reference/productstock_criterion.md + - ProductStockRange: search/criteria_reference/productstockrange_criterion.md + - ProductCode: search/sort_clause_reference/productcode_sort_clause.md + - ProductName: search/sort_clause_reference/productname_sort_clause.md + - Order Sort Clauses: + - Order Sort Clauses: search/sort_clause_reference/order_sort_clauses.md + - Id: search/sort_clause_reference/order_id_sort_clause.md + - Created: search/sort_clause_reference/order_created_sort_clause.md + - Updated: search/sort_clause_reference/order_updated_sort_clause.md + - Status: search/sort_clause_reference/order_status_sort_clause.md + - Payment Sort Clauses: + - Payment Sort Clauses: search/sort_clause_reference/payment_sort_clauses.md + - Id: search/sort_clause_reference/payment_id_sort_clause.md + - Identifier: search/sort_clause_reference/payment_identifier_sort_clause.md + - CreatedAt: search/sort_clause_reference/payment_createdat_sort_clause.md + - UpdatedAt: search/sort_clause_reference/payment_updatedat_sort_clause.md + - Status: search/sort_clause_reference/payment_status_sort_clause.md + - Payment Method Sort Clauses: + - Payment Method Sort Clauses: search/sort_clause_reference/payment_method_sort_clauses.md + - CreatedAt: search/sort_clause_reference/payment_method_createdat_sort_clause.md + - Enabled: search/sort_clause_reference/payment_method_enabled_sort_clause.md + - Id: search/sort_clause_reference/payment_method_id_sort_clause.md + - Identifier: search/sort_clause_reference/payment_method_identifier_sort_clause.md + - UpdatedAt: search/sort_clause_reference/payment_method_updatedat_sort_clause.md + - Shipment Sort Clauses: + - Shipment Sort Clauses: search/sort_clause_reference/shipment_sort_clauses.md + - Id: search/sort_clause_reference/shipment_id_sort_clause.md + - Identifier: search/sort_clause_reference/shipment_identifier_sort_clause.md + - CreatedAt: search/sort_clause_reference/shipment_createdat_sort_clause.md + - UpdatedAt: search/sort_clause_reference/shipment_updatedat_sort_clause.md + - Status: search/sort_clause_reference/shipment_status_sort_clause.md + - URL Sort Clauses: + - URL Sort Clauses: search/url_search_reference/url_search_sort_clauses.md + - Id Sort Clause: search/url_search_reference/id_url_sort_clause.md + - Url Sort Clause: search/url_search_reference/url_url_sort_clause.md + - Search API: search/search_api.md - Aggregation reference: - Aggregation reference: search/aggregation_reference/aggregation_reference.md - ContentTypeTermAggregation: search/aggregation_reference/contenttypeterm_aggregation.md @@ -599,27 +628,13 @@ nav: - Product attribute aggregations: search/aggregation_reference/product_attribute_aggregations.md - ProductAvailabilityTermAggregation: search/aggregation_reference/productavailabilityterm_aggregation.md - ProductStockRangeAggregation: search/aggregation_reference/productstockrange_aggregation.md + - ProductStockRangeAggregation: search/aggregation_reference/productstockrange_aggregation.md - ProductPriceRangeAggregation: search/aggregation_reference/productpricerange_aggregation.md - ProductTypeTermAggregation: search/aggregation_reference/producttypeterm_aggregation.md - SelectionTermAggregation: search/aggregation_reference/selectionterm_aggregation.md - TaxonomyEntryIdAggregation: search/aggregation_reference/taxonomyentryid_aggregation.md - TimeRangeAggregation: search/aggregation_reference/timerange_aggregation.md - - URL search reference: - - URL search reference: search/url_search_reference/url_search_reference.md - - Id Sort Clause: search/url_search_reference/id_url_sort_clause.md - - Url Sort Clause: search/url_search_reference/url_url_sort_clause.md - - MatchAll Criterion: search/url_search_reference/matchall_url_criterion.md - - MatchNone Criterion: search/url_search_reference/matchnone_url_criterion.md - - Pattern Criterion: search/url_search_reference/pattern_url_criterion.md - - SectionId Criterion: search/url_search_reference/sectionid_url_criterion.md - - SectionIdentifier Criterion: search/url_search_reference/sectionidentifier_url_criterion.md - - Validity Criterion: search/url_search_reference/validity_url_criterion.md - - VisibleOnly Criterion: search/url_search_reference/visibleonly_url_criterion.md - - LogicalAnd Criterion: search/url_search_reference/logicaland_url_criterion.md - - LogicalNot Criterion: search/url_search_reference/logicalnot_url_criterion.md - - LogicalOr Criterion: search/url_search_reference/logicalor_url_criterion.md - Search in trash reference: search/search_in_trash_reference.md - - Search reindexing: search/search_reindexing.md - Extend search: - Create custom Search Criterion: search/extensibility/create_custom_search_criterion.md - Create custom Sort Clause: search/extensibility/create_custom_sort_clause.md @@ -628,6 +643,7 @@ nav: - Index custom Elasticsearch data: search/extensibility/index_custom_elasticsearch_data.md - Customize Elasticsearch index structure: search/extensibility/customize_elasticsearch_index_structure.md - Manipulate Elasticsearch query: search/extensibility/manipulate_elasticsearch_query.md + - Reindex search: search/reindex_search.md - Infrastructure and maintenance: - Infrastructure and maintenance: infrastructure_and_maintenance/infrastructure_and_maintenance.md - Request lifecycle: infrastructure_and_maintenance/request_lifecycle.md diff --git a/plugins.yml b/plugins.yml index e91826d469..69a6a42af9 100644 --- a/plugins.yml +++ b/plugins.yml @@ -22,8 +22,8 @@ plugins: 'guide/shop_configuration.md': 'commerce/commerce.md' 'commerce/basket.md': 'commerce/commerce.md' 'templating/layout/customize_basket.md': 'commerce/commerce.md' - 'guide/search/extend_solr.md': 'search/solr_search_engine.md' - 'guide/search/extend_elasticsearch.md': 'search/elasticsearch_search_engine.md' + 'guide/search/extend_solr.md': 'search/search_engines/solr_search_engine/solr_overview.md' + 'guide/search/extend_elasticsearch.md': 'search/search_engines/elastic_search/elastic_search_overview.md' 'api/public_php_api.md': 'api/php_api/php_api.md' 'api/rest_api_usage.md': 'api/rest_api/rest_api_usage/rest_api_usage.md' @@ -294,8 +294,8 @@ plugins: 'guide/personalization/best_practices/recommendation_integration.md': 'personalization/recommendation_integration.md' 'guide/search/search.md': 'search/search.md' - 'guide/search/elastic.md': 'search/elasticsearch_search_engine.md' - 'guide/search/solr.md': 'search/solr_search_engine.md' + 'guide/search/elastic.md': 'search/search_engines/elastic_search/elastic_search_overview.md' + 'guide/search/solr.md': 'search/search_engines/solr_search_engine/solr_overview.md' 'api/public_php_api_search.md': 'search/search_api.md' 'guide/search/criteria_reference/search_criteria_reference.md': 'search/criteria_reference/search_criteria_reference.md' 'guide/search/criteria_reference/ancestor_criterion.md': 'search/criteria_reference/ancestor_criterion.md' @@ -405,7 +405,8 @@ plugins: 'guide/search/aggregation_reference/keywordterm_aggregation.md': 'search/aggregation_reference/keywordterm_aggregation.md' 'guide/search/aggregation_reference/selectionterm_aggregation.md': 'search/aggregation_reference/selectionterm_aggregation.md' 'guide/search/aggregation_reference/timerange_aggregation.md': 'search/aggregation_reference/timerange_aggregation.md' - 'guide/search/url_search_reference/url_search_reference.md': 'search/url_search_reference/url_search_reference.md' + 'guide/search/url_search_reference/url_search_reference.md': 'search/url_search_reference/url_search_criteria.md' + 'search/url_search_reference/url_search_reference.md': 'search/url_search_reference/url_search_criteria.md' 'guide/search/url_search_reference/id_url_sort_clause.md': 'search/url_search_reference/id_url_sort_clause.md' 'guide/search/url_search_reference/url_url_sort_clause.md': 'search/url_search_reference/url_url_sort_clause.md' 'guide/search/url_search_reference/matchall_url_criterion.md': 'search/url_search_reference/matchall_url_criterion.md'