diff --git a/3.5/aql/functions-arangosearch.md b/3.5/aql/functions-arangosearch.md index 391d75ffb0..5e64b8639a 100644 --- a/3.5/aql/functions-arangosearch.md +++ b/3.5/aql/functions-arangosearch.md @@ -260,6 +260,14 @@ Match documents where the attribute at **path** is greater than (or equal to) *low* and *high* can be numbers or strings (technically also `null`, `true` and `false`), but the data type must be the same for both. +{% hint 'warning' %} +The alphabetical order of characters is not taken into account by ArangoSearch, +i.e. range queries in SEARCH operations against Views will not follow the +language rules as per the defined Analyzer locale nor the server language +(startup option `--default-language`)! +Also see [Known Issues](../release-notes-known-issues35.html#arangosearch). +{% endhint %} + - **path** (attribute path expression): the path of the attribute to test in the document - **low** (number\|string): minimum value of the desired range @@ -406,6 +414,14 @@ is processed by a tokenizing Analyzer (type `"text"` or `"delimiter"`) or if it is an array, then a single token/element starting with the prefix is sufficient to match the document. +{% hint 'warning' %} +The alphabetical order of characters is not taken into account by ArangoSearch, +i.e. range queries in SEARCH operations against Views will not follow the +language rules as per the defined Analyzer locale nor the server language +(startup option `--default-language`)! +Also see [Known Issues](../release-notes-known-issues35.html#arangosearch). +{% endhint %} + - **path** (attribute path expression): the path of the attribute to compare against in the document - **prefix** (string): a string to search at the start of the text diff --git a/3.5/aql/operations-search.md b/3.5/aql/operations-search.md index 364d009aa7..9e8c147414 100644 --- a/3.5/aql/operations-search.md +++ b/3.5/aql/operations-search.md @@ -64,6 +64,14 @@ are supported: - `!=` - `IN` (array or range), also `NOT IN` +{% hint 'warning' %} +The alphabetical order of characters is not taken into account by ArangoSearch, +i.e. range queries in SEARCH operations against Views will not follow the +language rules as per the defined Analyzer locale nor the server language +(startup option `--default-language`)! +Also see [Known Issues](../release-notes-known-issues35.html#arangosearch). +{% endhint %} + ```js FOR doc IN viewName SEARCH ANALYZER(doc.text == "quick" OR doc.text == "brown", "text_en") diff --git a/3.5/arangosearch-analyzers.md b/3.5/arangosearch-analyzers.md index 1339779706..d6f5c97559 100644 --- a/3.5/arangosearch-analyzers.md +++ b/3.5/arangosearch-analyzers.md @@ -134,7 +134,7 @@ attributes: - `locale` (string): a locale in the format `language[_COUNTRY][.encoding][@variant]` (square brackets denote optional parts), e.g. `"de.utf-8"` or `"en_US.utf-8"`. Only UTF-8 encoding is - meaningful in ArangoDB. + meaningful in ArangoDB. Also see [Supported Languages](#supported-languages). ### Norm @@ -147,7 +147,7 @@ attributes: - `locale` (string): a locale in the format `language[_COUNTRY][.encoding][@variant]` (square brackets denote optional parts), e.g. `"de.utf-8"` or `"en_US.utf-8"`. Only UTF-8 encoding is - meaningful in ArangoDB. + meaningful in ArangoDB. Also see [Supported Languages](#supported-languages). - `accent` (boolean, _optional_): - `true` to preserve accented characters (default) - `false` to convert accented characters to their base characters @@ -194,16 +194,13 @@ An Analyzer capable of breaking up strings into individual words while also optionally filtering out stop-words, extracting word stems, applying case conversion and accent removal. -Stemming support is provided by -[Snowball](https://snowballstem.org/){:target="_blank"}. - The *properties* allowed for this Analyzer are an object with the following attributes: - `locale` (string): a locale in the format `language[_COUNTRY][.encoding][@variant]` (square brackets denote optional parts), e.g. `"de.utf-8"` or `"en_US.utf-8"`. Only UTF-8 encoding is - meaningful in ArangoDB. + meaningful in ArangoDB. Also see [Supported Languages](#supported-languages). - `accent` (boolean, _optional_): - `true` to preserve accented characters - `false` to convert accented characters to their base characters (default) @@ -281,3 +278,37 @@ Name | Type | Language `text_ru` | `text` | Russian `text_sv` | `text` | Swedish `text_zh` | `text` | Chinese + +Supported Languages +------------------- + +Analyzers rely on [ICU](http://site.icu-project.org/){:target="_blank"} for +language-dependent tokenization and normalization. The ICU data file +`icudtl.dat` that ArangoDB ships with contains information for a lot of +languages, which are technically all supported. + +{% hint 'warning' %} +The alphabetical order of characters is not taken into account by ArangoSearch, +i.e. range queries in SEARCH operations against Views will not follow the +language rules as per the defined Analyzer locale nor the server language +(startup option `--default-language`)! +Also see [Known Issues](release-notes-known-issues35.html#arangosearch). +{% endhint %} + +Stemming support is provided by [Snowball](https://snowballstem.org/){:target="_blank"}, +which supports the following languages: + +Code | Language +------|----------- +`de` | German +`en` | English +`es` | Spanish +`fi` | Finnish +`fr` | French +`it` | Italian +`nl` | Dutch +`no` | Norwegian +`pt` | Portuguese +`ru` | Russian +`sv` | Swedish +`zh` | Chinese diff --git a/3.5/release-notes-known-issues35.md b/3.5/release-notes-known-issues35.md index fb5432c009..51ff510aff 100644 --- a/3.5/release-notes-known-issues35.md +++ b/3.5/release-notes-known-issues35.md @@ -23,6 +23,7 @@ ArangoSearch | **Date Added:** 2018-12-03
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** Using a loop variable in expressions within a corresponding SEARCH condition is not supported
**Affected Versions:** 3.4.x, 3.5.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#318](https://github.com/arangodb/backlog/issues/318){:target="_blank"} (internal) | | **Date Added:** 2019-06-25
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** The `primarySort` attribute in ArangoSearch View definitions can not be set via the web interface. The option is immutable, but the web interface does not allow to set any View properties upfront (it creates a View with default parameters before the user has a chance to configure it).
**Affected Versions:** 3.5.x
**Fixed in Versions:** -
**Reference:** N/A | | **Date Added:** 2019-11-06
**Component:** ArangoSearch
**Deployment Mode:** Cluster
**Description:** There is a possibility to get into deadlocks during Coordinator execution if a custom Analyzer was created (and is present in the `_analyzers` system collection). It is recommended not to use custom Analyzers in production environments in affected versions.
**Affected Versions:** 3.5.x
**Fixed in Versions:** 3.5.3
**Reference:** [arangodb/backlog#651](https://github.com/arangodb/backlog/issues/651){:target="_blank"} (internal) | +| **Date Added:** 2020-03-19
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** Operators and functions in `SEARCH` clauses of AQL queries which compare values such as `>`, `>=`, `<`, `<=`, `IN_RANGE()` and `STARTS_WITH()` neither take the server language (`--default-language`) nor the Analyzer locale into account. The alphabetical order of characters as defined by a language is thus not honored and can lead to unexpected results in range queries.
**Affected Versions:** 3.5.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#679](https://github.com/arangodb/backlog/issues/679){:target="_blank"} (internal) | AQL --- diff --git a/3.6/aql/functions-arangosearch.md b/3.6/aql/functions-arangosearch.md index 119c10847a..f79b479713 100644 --- a/3.6/aql/functions-arangosearch.md +++ b/3.6/aql/functions-arangosearch.md @@ -260,6 +260,14 @@ Match documents where the attribute at **path** is greater than (or equal to) *low* and *high* can be numbers or strings (technically also `null`, `true` and `false`), but the data type must be the same for both. +{% hint 'warning' %} +The alphabetical order of characters is not taken into account by ArangoSearch, +i.e. range queries in SEARCH operations against Views will not follow the +language rules as per the defined Analyzer locale nor the server language +(startup option `--default-language`)! +Also see [Known Issues](../release-notes-known-issues35.html#arangosearch). +{% endhint %} + - **path** (attribute path expression): the path of the attribute to test in the document - **low** (number\|string): minimum value of the desired range @@ -438,6 +446,14 @@ is processed by a tokenizing Analyzer (type `"text"` or `"delimiter"`) or if it is an array, then a single token/element starting with the prefix is sufficient to match the document. +{% hint 'warning' %} +The alphabetical order of characters is not taken into account by ArangoSearch, +i.e. range queries in SEARCH operations against Views will not follow the +language rules as per the defined Analyzer locale nor the server language +(startup option `--default-language`)! +Also see [Known Issues](../release-notes-known-issues35.html#arangosearch). +{% endhint %} + - **path** (attribute path expression): the path of the attribute to compare against in the document - **prefix** (string): a string to search at the start of the text diff --git a/3.6/aql/operations-search.md b/3.6/aql/operations-search.md index 57f182b12b..8c03118d02 100644 --- a/3.6/aql/operations-search.md +++ b/3.6/aql/operations-search.md @@ -64,6 +64,14 @@ are supported: - `!=` - `IN` (array or range), also `NOT IN` +{% hint 'warning' %} +The alphabetical order of characters is not taken into account by ArangoSearch, +i.e. range queries in SEARCH operations against Views will not follow the +language rules as per the defined Analyzer locale nor the server language +(startup option `--default-language`)! +Also see [Known Issues](../release-notes-known-issues35.html#arangosearch). +{% endhint %} + ```js FOR doc IN viewName SEARCH ANALYZER(doc.text == "quick" OR doc.text == "brown", "text_en") diff --git a/3.6/arangosearch-analyzers.md b/3.6/arangosearch-analyzers.md index 277ce18f85..9bdbcf9a71 100644 --- a/3.6/arangosearch-analyzers.md +++ b/3.6/arangosearch-analyzers.md @@ -135,7 +135,7 @@ attributes: - `locale` (string): a locale in the format `language[_COUNTRY][.encoding][@variant]` (square brackets denote optional parts), e.g. `"de.utf-8"` or `"en_US.utf-8"`. Only UTF-8 encoding is - meaningful in ArangoDB. + meaningful in ArangoDB. Also see [Supported Languages](#supported-languages). ### Norm @@ -148,7 +148,7 @@ attributes: - `locale` (string): a locale in the format `language[_COUNTRY][.encoding][@variant]` (square brackets denote optional parts), e.g. `"de.utf-8"` or `"en_US.utf-8"`. Only UTF-8 encoding is - meaningful in ArangoDB. + meaningful in ArangoDB. Also see [Supported Languages](#supported-languages). - `accent` (boolean, _optional_): - `true` to preserve accented characters (default) - `false` to convert accented characters to their base characters @@ -215,16 +215,13 @@ An Analyzer capable of breaking up strings into individual words while also optionally filtering out stop-words, extracting word stems, applying case conversion and accent removal. -Stemming support is provided by -[Snowball](https://snowballstem.org/){:target="_blank"}. - The *properties* allowed for this Analyzer are an object with the following attributes: - `locale` (string): a locale in the format `language[_COUNTRY][.encoding][@variant]` (square brackets denote optional parts), e.g. `"de.utf-8"` or `"en_US.utf-8"`. Only UTF-8 encoding is - meaningful in ArangoDB. + meaningful in ArangoDB. Also see [Supported Languages](#supported-languages). - `accent` (boolean, _optional_): - `true` to preserve accented characters - `false` to convert accented characters to their base characters (default) @@ -367,3 +364,36 @@ Name | Type | Language `text_ru` | `text` | Russian `text_sv` | `text` | Swedish `text_zh` | `text` | Chinese + +Supported Languages +------------------- + +Analyzers rely on [ICU](http://site.icu-project.org/){:target="_blank"} for +language-dependent tokenization and normalization. The ICU data file +`icudtl.dat` that ArangoDB ships with contains information for a lot of +languages, which are technically all supported. + +{% hint 'warning' %} +The alphabetical order of characters is not taken into account by ArangoSearch, +i.e. range queries in SEARCH operations against Views will not follow the +language rules as per the defined Analyzer locale nor the server language +(startup option `--default-language`)! +Also see [Known Issues](release-notes-known-issues36.html#arangosearch). +{% endhint %} + +Stemming support is provided by [Snowball](https://snowballstem.org/){:target="_blank"}, +which supports the following languages: + +Code | Language +------|----------- +`de` | German +`en` | English +`es` | Spanish +`fi` | Finnish +`fr` | French +`it` | Italian +`nl` | Dutch +`no` | Norwegian +`pt` | Portuguese +`ru` | Russian +`sv` | Swedish diff --git a/3.6/release-notes-known-issues35.md b/3.6/release-notes-known-issues35.md index fb5432c009..51ff510aff 100644 --- a/3.6/release-notes-known-issues35.md +++ b/3.6/release-notes-known-issues35.md @@ -23,6 +23,7 @@ ArangoSearch | **Date Added:** 2018-12-03
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** Using a loop variable in expressions within a corresponding SEARCH condition is not supported
**Affected Versions:** 3.4.x, 3.5.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#318](https://github.com/arangodb/backlog/issues/318){:target="_blank"} (internal) | | **Date Added:** 2019-06-25
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** The `primarySort` attribute in ArangoSearch View definitions can not be set via the web interface. The option is immutable, but the web interface does not allow to set any View properties upfront (it creates a View with default parameters before the user has a chance to configure it).
**Affected Versions:** 3.5.x
**Fixed in Versions:** -
**Reference:** N/A | | **Date Added:** 2019-11-06
**Component:** ArangoSearch
**Deployment Mode:** Cluster
**Description:** There is a possibility to get into deadlocks during Coordinator execution if a custom Analyzer was created (and is present in the `_analyzers` system collection). It is recommended not to use custom Analyzers in production environments in affected versions.
**Affected Versions:** 3.5.x
**Fixed in Versions:** 3.5.3
**Reference:** [arangodb/backlog#651](https://github.com/arangodb/backlog/issues/651){:target="_blank"} (internal) | +| **Date Added:** 2020-03-19
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** Operators and functions in `SEARCH` clauses of AQL queries which compare values such as `>`, `>=`, `<`, `<=`, `IN_RANGE()` and `STARTS_WITH()` neither take the server language (`--default-language`) nor the Analyzer locale into account. The alphabetical order of characters as defined by a language is thus not honored and can lead to unexpected results in range queries.
**Affected Versions:** 3.5.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#679](https://github.com/arangodb/backlog/issues/679){:target="_blank"} (internal) | AQL --- diff --git a/3.6/release-notes-known-issues36.md b/3.6/release-notes-known-issues36.md index 868a6b9afe..9811456944 100644 --- a/3.6/release-notes-known-issues36.md +++ b/3.6/release-notes-known-issues36.md @@ -21,6 +21,7 @@ ArangoSearch | **Date Added:** 2018-12-03
**Component:** ArangoSearch
**Deployment Mode:** Cluster
**Description:** Score values evaluated by corresponding score functions (BM25/TFIDF) may differ in single-server and cluster with a collection having more than 1 shard
**Affected Versions:** 3.4.x, 3.5.x, 3.6.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#508](https://github.com/arangodb/backlog/issues/508){:target="_blank"} (internal) | | **Date Added:** 2018-12-03
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** Using a loop variable in expressions within a corresponding SEARCH condition is not supported
**Affected Versions:** 3.4.x, 3.5.x, 3.6.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#318](https://github.com/arangodb/backlog/issues/318){:target="_blank"} (internal) | | **Date Added:** 2019-06-25
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** The `primarySort` attribute in ArangoSearch View definitions can not be set via the web interface. The option is immutable, but the web interface does not allow to set any View properties upfront (it creates a View with default parameters before the user has a chance to configure it).
**Affected Versions:** 3.5.x, 3.6.x
**Fixed in Versions:** -
**Reference:** N/A | +| **Date Added:** 2020-03-19
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** Operators and functions in `SEARCH` clauses of AQL queries which compare values such as `>`, `>=`, `<`, `<=`, `IN_RANGE()` and `STARTS_WITH()` neither take the server language (`--default-language`) nor the Analyzer locale into account. The alphabetical order of characters as defined by a language is thus not honored and can lead to unexpected results in range queries.
**Affected Versions:** 3.5.x, 3.6.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#679](https://github.com/arangodb/backlog/issues/679){:target="_blank"} (internal) | AQL --- diff --git a/3.7/aql/functions-arangosearch.md b/3.7/aql/functions-arangosearch.md index 805d4f2773..8cc1182e46 100644 --- a/3.7/aql/functions-arangosearch.md +++ b/3.7/aql/functions-arangosearch.md @@ -260,6 +260,14 @@ Match documents where the attribute at **path** is greater than (or equal to) *low* and *high* can be numbers or strings (technically also `null`, `true` and `false`), but the data type must be the same for both. +{% hint 'warning' %} +The alphabetical order of characters is not taken into account by ArangoSearch, +i.e. range queries in SEARCH operations against Views will not follow the +language rules as per the defined Analyzer locale nor the server language +(startup option `--default-language`)! +Also see [Known Issues](../release-notes-known-issues35.html#arangosearch). +{% endhint %} + - **path** (attribute path expression): the path of the attribute to test in the document - **low** (number\|string): minimum value of the desired range @@ -438,6 +446,14 @@ is processed by a tokenizing Analyzer (type `"text"` or `"delimiter"`) or if it is an array, then a single token/element starting with the prefix is sufficient to match the document. +{% hint 'warning' %} +The alphabetical order of characters is not taken into account by ArangoSearch, +i.e. range queries in SEARCH operations against Views will not follow the +language rules as per the defined Analyzer locale nor the server language +(startup option `--default-language`)! +Also see [Known Issues](../release-notes-known-issues35.html#arangosearch). +{% endhint %} + - **path** (attribute path expression): the path of the attribute to compare against in the document - **prefix** (string): a string to search at the start of the text diff --git a/3.7/aql/operations-search.md b/3.7/aql/operations-search.md index 5263927c1c..7a7b6dd0f9 100644 --- a/3.7/aql/operations-search.md +++ b/3.7/aql/operations-search.md @@ -65,6 +65,14 @@ are supported: - `IN` (array or range), also `NOT IN` - `LIKE` (introduced in v3.7.0), also `NOT LIKE` +{% hint 'warning' %} +The alphabetical order of characters is not taken into account by ArangoSearch, +i.e. range queries in SEARCH operations against Views will not follow the +language rules as per the defined Analyzer locale nor the server language +(startup option `--default-language`)! +Also see [Known Issues](../release-notes-known-issues37.html#arangosearch). +{% endhint %} + ```js FOR doc IN viewName SEARCH ANALYZER(doc.text == "quick" OR doc.text == "brown", "text_en") diff --git a/3.7/arangosearch-analyzers.md b/3.7/arangosearch-analyzers.md index 21afb12ceb..09492b86ca 100644 --- a/3.7/arangosearch-analyzers.md +++ b/3.7/arangosearch-analyzers.md @@ -135,7 +135,7 @@ attributes: - `locale` (string): a locale in the format `language[_COUNTRY][.encoding][@variant]` (square brackets denote optional parts), e.g. `"de.utf-8"` or `"en_US.utf-8"`. Only UTF-8 encoding is - meaningful in ArangoDB. + meaningful in ArangoDB. Also see [Supported Languages](#supported-languages). ### Norm @@ -148,7 +148,7 @@ attributes: - `locale` (string): a locale in the format `language[_COUNTRY][.encoding][@variant]` (square brackets denote optional parts), e.g. `"de.utf-8"` or `"en_US.utf-8"`. Only UTF-8 encoding is - meaningful in ArangoDB. + meaningful in ArangoDB. Also see [Supported Languages](#supported-languages). - `accent` (boolean, _optional_): - `true` to preserve accented characters (default) - `false` to convert accented characters to their base characters @@ -215,16 +215,13 @@ An Analyzer capable of breaking up strings into individual words while also optionally filtering out stop-words, extracting word stems, applying case conversion and accent removal. -Stemming support is provided by -[Snowball](https://snowballstem.org/){:target="_blank"}. - The *properties* allowed for this Analyzer are an object with the following attributes: - `locale` (string): a locale in the format `language[_COUNTRY][.encoding][@variant]` (square brackets denote optional parts), e.g. `"de.utf-8"` or `"en_US.utf-8"`. Only UTF-8 encoding is - meaningful in ArangoDB. + meaningful in ArangoDB. Also see [Supported Languages](#supported-languages). - `accent` (boolean, _optional_): - `true` to preserve accented characters - `false` to convert accented characters to their base characters (default) @@ -367,3 +364,51 @@ Name | Type | Language `text_ru` | `text` | Russian `text_sv` | `text` | Swedish `text_zh` | `text` | Chinese + +Supported Languages +------------------- + +Analyzers rely on [ICU](http://site.icu-project.org/){:target="_blank"} for +language-dependent tokenization and normalization. The ICU data file +`icudtl.dat` that ArangoDB ships with contains information for a lot of +languages, which are technically all supported. + +{% hint 'warning' %} +The alphabetical order of characters is not taken into account by ArangoSearch, +i.e. range queries in SEARCH operations against Views will not follow the +language rules as per the defined Analyzer locale nor the server language +(startup option `--default-language`)! +Also see [Known Issues](release-notes-known-issues37.html#arangosearch). +{% endhint %} + +Stemming support is provided by [Snowball](https://snowballstem.org/){:target="_blank"}, +which supports the following languages: + +Language | Code +---------- | ---- +Arabic | `ar` +Basque | `eu` +Catalan | `ca` +Danish | `da` +Dutch | `nl` +English | `en` +Finnish | `fi` +French | `fr` +German | `de` +Greek | `el` +Hindi | `hi` +Hungarian | `hu` +Indonesian | `id` +Irish | `ga` +Italian | `it` +Lithuanian | `lt` +Nepali | `ne` +Norwegian | `no` +Portuguese | `pt` +Romanian | `ro` +Russian | `ru` +Serbian | `sr` +Spanish | `es` +Swedish | `sv` +Tamil | `ta` +Turkish | `tr` diff --git a/3.7/release-notes-known-issues35.md b/3.7/release-notes-known-issues35.md index fb5432c009..51ff510aff 100644 --- a/3.7/release-notes-known-issues35.md +++ b/3.7/release-notes-known-issues35.md @@ -23,6 +23,7 @@ ArangoSearch | **Date Added:** 2018-12-03
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** Using a loop variable in expressions within a corresponding SEARCH condition is not supported
**Affected Versions:** 3.4.x, 3.5.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#318](https://github.com/arangodb/backlog/issues/318){:target="_blank"} (internal) | | **Date Added:** 2019-06-25
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** The `primarySort` attribute in ArangoSearch View definitions can not be set via the web interface. The option is immutable, but the web interface does not allow to set any View properties upfront (it creates a View with default parameters before the user has a chance to configure it).
**Affected Versions:** 3.5.x
**Fixed in Versions:** -
**Reference:** N/A | | **Date Added:** 2019-11-06
**Component:** ArangoSearch
**Deployment Mode:** Cluster
**Description:** There is a possibility to get into deadlocks during Coordinator execution if a custom Analyzer was created (and is present in the `_analyzers` system collection). It is recommended not to use custom Analyzers in production environments in affected versions.
**Affected Versions:** 3.5.x
**Fixed in Versions:** 3.5.3
**Reference:** [arangodb/backlog#651](https://github.com/arangodb/backlog/issues/651){:target="_blank"} (internal) | +| **Date Added:** 2020-03-19
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** Operators and functions in `SEARCH` clauses of AQL queries which compare values such as `>`, `>=`, `<`, `<=`, `IN_RANGE()` and `STARTS_WITH()` neither take the server language (`--default-language`) nor the Analyzer locale into account. The alphabetical order of characters as defined by a language is thus not honored and can lead to unexpected results in range queries.
**Affected Versions:** 3.5.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#679](https://github.com/arangodb/backlog/issues/679){:target="_blank"} (internal) | AQL --- diff --git a/3.7/release-notes-known-issues36.md b/3.7/release-notes-known-issues36.md index 868a6b9afe..4c777201b1 100644 --- a/3.7/release-notes-known-issues36.md +++ b/3.7/release-notes-known-issues36.md @@ -21,6 +21,8 @@ ArangoSearch | **Date Added:** 2018-12-03
**Component:** ArangoSearch
**Deployment Mode:** Cluster
**Description:** Score values evaluated by corresponding score functions (BM25/TFIDF) may differ in single-server and cluster with a collection having more than 1 shard
**Affected Versions:** 3.4.x, 3.5.x, 3.6.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#508](https://github.com/arangodb/backlog/issues/508){:target="_blank"} (internal) | | **Date Added:** 2018-12-03
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** Using a loop variable in expressions within a corresponding SEARCH condition is not supported
**Affected Versions:** 3.4.x, 3.5.x, 3.6.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#318](https://github.com/arangodb/backlog/issues/318){:target="_blank"} (internal) | | **Date Added:** 2019-06-25
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** The `primarySort` attribute in ArangoSearch View definitions can not be set via the web interface. The option is immutable, but the web interface does not allow to set any View properties upfront (it creates a View with default parameters before the user has a chance to configure it).
**Affected Versions:** 3.5.x, 3.6.x
**Fixed in Versions:** -
**Reference:** N/A | +| **Date Added:** 2020-03-19
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** Operators and functions in `SEARCH` clauses of AQL queries which compare values such as `>`, `>=`, `<`, `<=`, `IN_RANGE()` and `STARTS_WITH()` neither take the server language (`--default-language`) nor the Analyzer locale into account. The alphabetical order of characters as defined by a language is thus not honored and can lead to unexpected results in range queries.
**Affected Versions:** 3.5.x, 3.6.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#679](https://github.com/arangodb/backlog/issues/679){:target="_blank"} (internal) | + AQL --- diff --git a/3.7/release-notes-known-issues37.md b/3.7/release-notes-known-issues37.md index 34f96ed889..f19ca73865 100644 --- a/3.7/release-notes-known-issues37.md +++ b/3.7/release-notes-known-issues37.md @@ -21,6 +21,7 @@ ArangoSearch | **Date Added:** 2018-12-03
**Component:** ArangoSearch
**Deployment Mode:** Cluster
**Description:** Score values evaluated by corresponding score functions (BM25/TFIDF) may differ in single-server and cluster with a collection having more than 1 shard
**Affected Versions:** 3.4.x, 3.5.x, 3.6.x, 3.7.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#508](https://github.com/arangodb/backlog/issues/508){:target="_blank"} (internal) | | **Date Added:** 2018-12-03
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** Using a loop variable in expressions within a corresponding SEARCH condition is not supported
**Affected Versions:** 3.4.x, 3.5.x, 3.6.x, 3.7.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#318](https://github.com/arangodb/backlog/issues/318){:target="_blank"} (internal) | | **Date Added:** 2019-06-25
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** The `primarySort` attribute in ArangoSearch View definitions can not be set via the web interface. The option is immutable, but the web interface does not allow to set any View properties upfront (it creates a View with default parameters before the user has a chance to configure it).
**Affected Versions:** 3.5.x, 3.6.x, 3.7.x
**Fixed in Versions:** -
**Reference:** N/A | +| **Date Added:** 2020-03-19
**Component:** ArangoSearch
**Deployment Mode:** All
**Description:** Operators and functions in `SEARCH` clauses of AQL queries which compare values such as `>`, `>=`, `<`, `<=`, `IN_RANGE()` and `STARTS_WITH()` neither take the server language (`--default-language`) nor the Analyzer locale into account. The alphabetical order of characters as defined by a language is thus not honored and can lead to unexpected results in range queries.
**Affected Versions:** 3.5.x, 3.6.x
**Fixed in Versions:** -
**Reference:** [arangodb/backlog#679](https://github.com/arangodb/backlog/issues/679){:target="_blank"} (internal) | AQL --- diff --git a/3.7/release-notes-new-features37.md b/3.7/release-notes-new-features37.md index b68213888e..44ddb44049 100644 --- a/3.7/release-notes-new-features37.md +++ b/3.7/release-notes-new-features37.md @@ -30,6 +30,47 @@ FOR doc IN viewName See [ArangoSearch functions](aql/functions-arangosearch.html#like) +### Stemming support for more languages + +The Snowball library was updated to the latest version 2, adding stemming +support for the following languages: + +- Arabic (`ar`) +- Basque (`eu`) +- Catalan (`ca`) +- Danish (`da`) +- Greek (`el`) +- Hindi (`hi`) +- Hungarian (`hu`) +- Indonesian (`id`) +- Irish (`ga`) +- Lithuanian (`lt`) +- Nepali (`ne`) +- Romanian (`ro`) +- Serbian (`sr`) +- Tamil (`ta`) +- Turkish (`tr`) + +Create a custom Analyzer and set the `locale` accordingly in the properties, +e.g. `"el.utf-8"` for Greek. Arangosh example: + +```js +var analyzers = require("@arangodb/analyzers"); + +analyzers.save("text_el", "text", { + locale: "el.utf-8", + stemming: true, + case: "lower", + accent: false, + stopwords: [] +}, ["frequency", "norm", "position"]); + +db._query(`RETURN TOKENS("αυτοκινητουσ πρωταγωνιστούσαν", "text_el")`) +// [ [ "αυτοκινητ", "πρωταγωνιστ" ] ] +``` + +Also see [Analyzers: Supported Languages](arangosearch-analyzers.html#supported-languages) + SatelliteGraphs --------------- diff --git a/3.7/release-notes-upgrading-changes37.md b/3.7/release-notes-upgrading-changes37.md index f392fe79c0..fb0a48b839 100644 --- a/3.7/release-notes-upgrading-changes37.md +++ b/3.7/release-notes-upgrading-changes37.md @@ -25,6 +25,16 @@ will still continue to work in ArangoDB 3.7. However, 3.7 will be the last Arang version supporting the MMFiles storage engines, so users are asked to migrate to the RocksDB storage engine soon. +ArangoSearch +------------ + +The stemming library Snowball was updated, bringing a breaking change to Analyzers: + +There is a 33rd letter in the Russian alphabet, `ё` (`e"`), but it is rarely used +and often replaced by `е` in informal writing. The original algorithm assumed it +had already been mapped to `е` (`e`), but now it actively translates this character +if the locale is set to Russian language. + HTTP REST API -------------