From ac082393f23c866b560a1bcb34e50517f53ee159 Mon Sep 17 00:00:00 2001 From: Simran Spiller Date: Fri, 12 Jan 2024 14:52:49 +0100 Subject: [PATCH 1/8] Initial pass --- .../features/community-edition.md | 2 - .../features/highlights-by-version.md | 4 + .../http-api/indexes/multi-dimensional.md | 83 ++++++++++++++++--- .../multi-dimensional-indexes.md | 11 +-- .../version-3.12/api-changes-in-3-12.md | 21 +++++ .../version-3.12/whats-new-in-3-12.md | 19 +++++ 6 files changed, 120 insertions(+), 20 deletions(-) diff --git a/site/content/3.12/about-arangodb/features/community-edition.md b/site/content/3.12/about-arangodb/features/community-edition.md index 18b084dc38..2a348deb5b 100644 --- a/site/content/3.12/about-arangodb/features/community-edition.md +++ b/site/content/3.12/about-arangodb/features/community-edition.md @@ -201,11 +201,9 @@ see [arangodb.com/community-server/](https://www.arangodb.com/community-server/) the S2 library. Support for composable, distance-based geo-queries ("geo cursors"). -{{% comment %}} Experimental feature - [**Multi-dimensional indexes**](../../index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md): An index type to efficiently intersect multiple range queries, like finding all appointments that intersect a time range. -{{% /comment %}} - [**Background Indexing**](../../index-and-search/indexing/basics.md#creating-indexes-in-background): Indexes can be created in the background to not block queries in the meantime. diff --git a/site/content/3.12/about-arangodb/features/highlights-by-version.md b/site/content/3.12/about-arangodb/features/highlights-by-version.md index be199f5f13..6be3a383d1 100644 --- a/site/content/3.12/about-arangodb/features/highlights-by-version.md +++ b/site/content/3.12/about-arangodb/features/highlights-by-version.md @@ -15,6 +15,10 @@ aliases: - +- [**Multi-dimensional indexes**](../../index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md): + An index type to efficiently intersect multiple range queries, like finding + all appointments that intersect a time range. + **Enterprise Edition** - [**ArangoSearch WAND optimization**](../../index-and-search/arangosearch/performance.md#wand-optimization): diff --git a/site/content/3.12/develop/http-api/indexes/multi-dimensional.md b/site/content/3.12/develop/http-api/indexes/multi-dimensional.md index 6f3536eac5..2bc1289168 100644 --- a/site/content/3.12/develop/http-api/indexes/multi-dimensional.md +++ b/site/content/3.12/develop/http-api/indexes/multi-dimensional.md @@ -10,9 +10,9 @@ archetype: default ```openapi paths: - /_api/index#zkd: + /_api/index#mdi: post: - operationId: createIndexZkd + operationId: createIndexMdi description: | Creates a multi-dimensional index for the collection `collection-name`, if it does not already exist. The call expects an object containing the index @@ -37,7 +37,7 @@ paths: properties: type: description: | - must be equal to `"zkd"`. + must be equal to `"mdi"` or `"mdi-prefixed"`. type: string name: description: | @@ -48,7 +48,42 @@ paths: type: string fields: description: | - an array of attribute names used for each dimension. Array expansions are not allowed. + An array of attribute names used for each dimension. Array expansions are not allowed. + type: array + items: + type: string + fieldValueTypes: + description: | + must be equal to `"double"`. Currently only doubles are supported as values. + type: string + prefixFields: + description: | + Requires `type` to be `"mdi-prefixed"`, and `prefixFields` needs to be set in this case. + + An array of attribute names used as search prefix. Array expansions are not allowed. + type: array + items: + type: string + storedValues: + description: | + The optional `storedValues` attribute can contain an array of paths to additional + attributes to store in the index. These additional attributes cannot be used for + index lookups or for sorting, but they can be used for projections. This allows an + index to fully cover more queries and avoid extra document lookups. + The maximum number of attributes in `storedValues` is 32. + + It is not possible to create multiple indexes with the same `fields` attributes + and uniqueness but different `storedValues` attributes. That means the value of + `storedValues` is not considered by index creation calls when checking if an + index is already present or needs to be created. + + In unique indexes, only the attributes in `fields` are checked for uniqueness, + but the attributes in `storedValues` are not checked for their uniqueness. + Non-existing attributes are stored as `null` values inside `storedValues`. + + Attributes in `storedValues` cannot overlap with attributes + specified in `prefixFields` but you can have the attributes + in both `storedValues` and `fields`. type: array items: type: string @@ -56,16 +91,18 @@ paths: description: | if `true`, then create a unique index. type: boolean + default: false + sparse: + description: | + If `true`, then create a sparse index. + type: boolean + default: false inBackground: description: | You can set this option to `true` to create the index in the background, which will not write-lock the underlying collection for as long as if the index is built in the foreground. The default value is `false`. type: boolean - fieldValueTypes: - description: | - must be equal to `"double"`. Currently only doubles are supported as values. - type: string responses: '200': description: | @@ -91,7 +128,7 @@ paths: --- description: |- Creating a multi-dimensional index -name: RestIndexCreateNewZkd +name: RestIndexCreateNewMdi --- var cn = "intervals"; db._drop(cn); @@ -99,7 +136,7 @@ db._create(cn); var url = "/_api/index?collection=" + cn; var body = { - type: "zkd", + type: "mdi", fields: [ "from", "to" ], fieldValueTypes: "double" }; @@ -111,3 +148,29 @@ db._create(cn); logJsonResponse(response); db._drop(cn); ``` + +```curl +--- +description: |- + Creating a prefixed multi-dimensional index +name: RestIndexCreateNewMdiPrefixed +--- +var cn = "intervals"; +db._drop(cn); +db._create(cn); + + var url = "/_api/index?collection=" + cn; + var body = { + type: "mdi-prefixed", + fields: [ "from", "to" ], + fieldValueTypes: "double", + prefixFields: ["year", "month"] + }; + + var response = logCurlRequest('POST', url, body); + + assert(response.code === 201); + + logJsonResponse(response); +db._drop(cn); +``` diff --git a/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md b/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md index d6596cab75..fef3a5b94e 100644 --- a/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md +++ b/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md @@ -7,13 +7,9 @@ description: >- such as time ranges, for efficient intersection of multiple range queries archetype: default --- -The multi-dimensional index type is called **ZKD**. +The multi-dimensional index type was previously called `"zkd"`. -{{< warning >}} -`zkd` indexes are an **experimental** feature. -{{< /warning >}} - -A multi-dimensional index is setup by setting the index type to `"zkd"`. +A multi-dimensional index is setup by setting the index type to `"mdi"`. The `fields` attribute describes which fields are used as dimensions. The value of each dimension has to be a numeric (double) value. @@ -33,7 +29,7 @@ To do so one creates a multi-dimensional index on the attributes `x`, `y` and ```js db.collection.ensureIndex({ - type: "zkd", + type: "mdi", fields: ["x", "y", "z"], fieldValueTypes: "double" }); @@ -154,7 +150,6 @@ FOR app IN appointments OPTIONS { lookahead: 32 } Currently there are a few limitations: - Using array expansions for attributes is not possible (e.g. `array[*].attr`) -- The `sparse` property is not supported. - You can only index numeric values that are representable as IEEE-754 double. - A high number of dimensions (more than 5) can impact the performance considerably. - The performance can vary depending on the dataset. Densely packed points can diff --git a/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md b/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md index cd5cda57ed..33b637b001 100644 --- a/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md +++ b/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md @@ -126,6 +126,8 @@ for details. #### Index API +##### Inverted indexes + Indexes of type `inverted` accept a new `optimizeTopK` property for the ArangoSearch WAND optimization. It is an array of strings, optional, and defaults to `[]`. @@ -133,6 +135,25 @@ defaults to `[]`. See the [inverted index `optimizeTopK` property](../../develop/http-api/indexes/inverted.md) for details. +##### Multi-dimensional indexes + +The previously experimental `zkd` index is now stable and has been renamed to +`mdi`. Existing indexes keep the `zkd` type. The HTTP API still allows the old +name to create new indexes that behave exactly like `mdi` indexes but this is +discouraged. The `zkd` alias may get removed in a future version. + +An additional `mdi-prefixed` index variant has been added. This is a new index +type in the API with the same settings as the `mdi` index but with one additional +`prefixFields` attribute. It is a required setting and accepts an array of strings +similar to the `fields` attribute. + +Both multi-dimensional index variants now support a `sparse` (boolean) and +`storedValues` (array of strings) setting that were not supported by the `zkd` +index in previous versions. + +See [Working with multi-dimensional indexes](../../index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md) +for details. + #### Optimizer rule descriptions Introduced in: v3.10.9, v3.11.2 diff --git a/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md b/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md index 011506fe6a..9873f09316 100644 --- a/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md +++ b/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md @@ -249,6 +249,25 @@ non-collection data. ## Indexing +### Multi-dimensional indexes + +The previously experimental `zkd` index is now stable and has been renamed to +`mdi`. Existing indexes keep the `zkd` type. + +Multi-dimensional indexes can now be declared as `sparse` to exclude documents +from the index that do not have the defined attributes or are explicitly set to +`null` values. If a non-value is set, it still needs to be numeric. + +Multi-dimensional indexes now support stored values to cover queries for better +performance. + +An additional `mdi-prefixed` index variant has been added that lets you specify +additional attributes for the index to narrow down the search space using +equality checks. + +See [Multi-dimensional indexes](../../index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md) +for details. + ### Stored values can contain the `_id` attribute The usage of the `_id` system attribute was previously disallowed for From 1f259f1c7257d135e9a6626fe8fe805b52759ba8 Mon Sep 17 00:00:00 2001 From: Simran Spiller Date: Fri, 12 Jan 2024 14:58:38 +0100 Subject: [PATCH 2/8] Change mentions of ZKD to MDI --- site/content/3.12/aql/high-level-operations/for.md | 4 ++-- .../3.12/index-and-search/indexing/which-index-to-use-when.md | 2 +- .../index-and-search/indexing/working-with-indexes/_index.md | 3 ++- site/content/3.12/operations/administration/telemetrics.md | 3 ++- 4 files changed, 7 insertions(+), 5 deletions(-) diff --git a/site/content/3.12/aql/high-level-operations/for.md b/site/content/3.12/aql/high-level-operations/for.md index d2c5e22ea7..03c6b16e9b 100644 --- a/site/content/3.12/aql/high-level-operations/for.md +++ b/site/content/3.12/aql/high-level-operations/for.md @@ -240,8 +240,8 @@ Also see [Caching of index values](../../index-and-search/indexing/working-with- ### `lookahead` -The multi-dimensional index type `zkd` supports an optional index hint for -tweaking performance: +The multi-dimensional index types `mdi` and `mdi-prefixed` support an optional +index hint for tweaking performance: ```aql FOR … IN … OPTIONS { lookahead: 32 } diff --git a/site/content/3.12/index-and-search/indexing/which-index-to-use-when.md b/site/content/3.12/index-and-search/indexing/which-index-to-use-when.md index 5e1c13b902..b5c9c9b275 100644 --- a/site/content/3.12/index-and-search/indexing/which-index-to-use-when.md +++ b/site/content/3.12/index-and-search/indexing/which-index-to-use-when.md @@ -66,7 +66,7 @@ different usage scenarios: documents shall expire at the point in time or a given number of seconds after the point in time. -- **multi-dimensional index** (ZKD): a multi dimensional index allows to +- **multi-dimensional index**: a multi dimensional index allows to efficiently intersect multiple range queries. Typical use cases are querying intervals that intersect a given point or interval. For example, if intervals are stored in documents like diff --git a/site/content/3.12/index-and-search/indexing/working-with-indexes/_index.md b/site/content/3.12/index-and-search/indexing/working-with-indexes/_index.md index af965149e4..91ddb0596e 100644 --- a/site/content/3.12/index-and-search/indexing/working-with-indexes/_index.md +++ b/site/content/3.12/index-and-search/indexing/working-with-indexes/_index.md @@ -126,7 +126,8 @@ Other attributes may be necessary, depending on the index type. - `"ttl"`: time-to-live index - `"fulltext"`: full-text index (deprecated from ArangoDB 3.10 onwards) - `"geo"`: geo-spatial index, with _one_ or _two_ attributes - - `"zkd"`: multi-dimensional index (experimental) + - `"mdi"`: multi-dimensional index + - `"mdi-prefixed"`: multi-dimensional index with search prefix - `fields`: an array of attribute paths, containing the document attributes (or sub-attributes) to be indexed. Some indexes allow using only a single path, diff --git a/site/content/3.12/operations/administration/telemetrics.md b/site/content/3.12/operations/administration/telemetrics.md index 4368fc69c4..91bf087dc1 100644 --- a/site/content/3.12/operations/administration/telemetrics.md +++ b/site/content/3.12/operations/administration/telemetrics.md @@ -120,7 +120,8 @@ as well as configuration details in terms of sharding and replication. - The number of fulltext indexes - The number of iresearch indexes - The number of inverted indexes - - The number of zkd indexes + - The number of mdi indexes + - The number of mdi-prefixed indexes - A list of detailed information per index - The index type - Is it a unique index? From ffdb972d8ec7e79e6dc858882130a22cd50485a6 Mon Sep 17 00:00:00 2001 From: Simran Spiller Date: Thu, 8 Feb 2024 09:08:37 +0100 Subject: [PATCH 3/8] Remove line breaks --- site/content/3.10/develop/http-api/indexes/fulltext.md | 1 - site/content/3.10/develop/http-api/indexes/geo-spatial.md | 1 - .../3.10/develop/http-api/indexes/multi-dimensional.md | 1 - site/content/3.10/develop/http-api/indexes/ttl.md | 1 - .../http-api/replication/other-replication-commands.md | 1 - site/content/3.11/develop/http-api/indexes/fulltext.md | 1 - site/content/3.11/develop/http-api/indexes/geo-spatial.md | 1 - site/content/3.11/develop/http-api/indexes/inverted.md | 1 - .../3.11/develop/http-api/indexes/multi-dimensional.md | 1 - site/content/3.11/develop/http-api/indexes/ttl.md | 1 - .../http-api/replication/other-replication-commands.md | 1 - site/content/3.12/develop/http-api/indexes/fulltext.md | 1 - site/content/3.12/develop/http-api/indexes/geo-spatial.md | 1 - site/content/3.12/develop/http-api/indexes/inverted.md | 1 - .../3.12/develop/http-api/indexes/multi-dimensional.md | 5 +++-- site/content/3.12/develop/http-api/indexes/ttl.md | 1 - .../http-api/replication/other-replication-commands.md | 1 - 17 files changed, 3 insertions(+), 18 deletions(-) diff --git a/site/content/3.10/develop/http-api/indexes/fulltext.md b/site/content/3.10/develop/http-api/indexes/fulltext.md index af42da76f5..a770667845 100644 --- a/site/content/3.10/develop/http-api/indexes/fulltext.md +++ b/site/content/3.10/develop/http-api/indexes/fulltext.md @@ -5,7 +5,6 @@ weight: 30 description: '' archetype: default --- - ## Create a full-text index ```openapi diff --git a/site/content/3.10/develop/http-api/indexes/geo-spatial.md b/site/content/3.10/develop/http-api/indexes/geo-spatial.md index 81fd04e17c..50ee3f1ace 100644 --- a/site/content/3.10/develop/http-api/indexes/geo-spatial.md +++ b/site/content/3.10/develop/http-api/indexes/geo-spatial.md @@ -5,7 +5,6 @@ weight: 25 description: '' archetype: default --- - ## Create a geo-spatial index ```openapi diff --git a/site/content/3.10/develop/http-api/indexes/multi-dimensional.md b/site/content/3.10/develop/http-api/indexes/multi-dimensional.md index 6f3536eac5..fd9d2590ee 100644 --- a/site/content/3.10/develop/http-api/indexes/multi-dimensional.md +++ b/site/content/3.10/develop/http-api/indexes/multi-dimensional.md @@ -5,7 +5,6 @@ weight: 20 description: '' archetype: default --- - ## Create a multi-dimensional index ```openapi diff --git a/site/content/3.10/develop/http-api/indexes/ttl.md b/site/content/3.10/develop/http-api/indexes/ttl.md index 847342e1cf..37c29a46c1 100644 --- a/site/content/3.10/develop/http-api/indexes/ttl.md +++ b/site/content/3.10/develop/http-api/indexes/ttl.md @@ -5,7 +5,6 @@ weight: 15 description: '' archetype: default --- - ## Create a TTL index ```openapi diff --git a/site/content/3.10/develop/http-api/replication/other-replication-commands.md b/site/content/3.10/develop/http-api/replication/other-replication-commands.md index 7fe5facc47..6adfffc8ab 100644 --- a/site/content/3.10/develop/http-api/replication/other-replication-commands.md +++ b/site/content/3.10/develop/http-api/replication/other-replication-commands.md @@ -5,7 +5,6 @@ weight: 20 description: '' archetype: default --- - ## Get the replication server ID ```openapi diff --git a/site/content/3.11/develop/http-api/indexes/fulltext.md b/site/content/3.11/develop/http-api/indexes/fulltext.md index af42da76f5..a770667845 100644 --- a/site/content/3.11/develop/http-api/indexes/fulltext.md +++ b/site/content/3.11/develop/http-api/indexes/fulltext.md @@ -5,7 +5,6 @@ weight: 30 description: '' archetype: default --- - ## Create a full-text index ```openapi diff --git a/site/content/3.11/develop/http-api/indexes/geo-spatial.md b/site/content/3.11/develop/http-api/indexes/geo-spatial.md index 81fd04e17c..50ee3f1ace 100644 --- a/site/content/3.11/develop/http-api/indexes/geo-spatial.md +++ b/site/content/3.11/develop/http-api/indexes/geo-spatial.md @@ -5,7 +5,6 @@ weight: 25 description: '' archetype: default --- - ## Create a geo-spatial index ```openapi diff --git a/site/content/3.11/develop/http-api/indexes/inverted.md b/site/content/3.11/develop/http-api/indexes/inverted.md index 4c5a218976..42089ecd18 100644 --- a/site/content/3.11/develop/http-api/indexes/inverted.md +++ b/site/content/3.11/develop/http-api/indexes/inverted.md @@ -5,7 +5,6 @@ weight: 10 description: '' archetype: default --- - ## Create an inverted index ```openapi diff --git a/site/content/3.11/develop/http-api/indexes/multi-dimensional.md b/site/content/3.11/develop/http-api/indexes/multi-dimensional.md index 6f3536eac5..fd9d2590ee 100644 --- a/site/content/3.11/develop/http-api/indexes/multi-dimensional.md +++ b/site/content/3.11/develop/http-api/indexes/multi-dimensional.md @@ -5,7 +5,6 @@ weight: 20 description: '' archetype: default --- - ## Create a multi-dimensional index ```openapi diff --git a/site/content/3.11/develop/http-api/indexes/ttl.md b/site/content/3.11/develop/http-api/indexes/ttl.md index 847342e1cf..37c29a46c1 100644 --- a/site/content/3.11/develop/http-api/indexes/ttl.md +++ b/site/content/3.11/develop/http-api/indexes/ttl.md @@ -5,7 +5,6 @@ weight: 15 description: '' archetype: default --- - ## Create a TTL index ```openapi diff --git a/site/content/3.11/develop/http-api/replication/other-replication-commands.md b/site/content/3.11/develop/http-api/replication/other-replication-commands.md index 7fe5facc47..6adfffc8ab 100644 --- a/site/content/3.11/develop/http-api/replication/other-replication-commands.md +++ b/site/content/3.11/develop/http-api/replication/other-replication-commands.md @@ -5,7 +5,6 @@ weight: 20 description: '' archetype: default --- - ## Get the replication server ID ```openapi diff --git a/site/content/3.12/develop/http-api/indexes/fulltext.md b/site/content/3.12/develop/http-api/indexes/fulltext.md index af42da76f5..a770667845 100644 --- a/site/content/3.12/develop/http-api/indexes/fulltext.md +++ b/site/content/3.12/develop/http-api/indexes/fulltext.md @@ -5,7 +5,6 @@ weight: 30 description: '' archetype: default --- - ## Create a full-text index ```openapi diff --git a/site/content/3.12/develop/http-api/indexes/geo-spatial.md b/site/content/3.12/develop/http-api/indexes/geo-spatial.md index 81fd04e17c..50ee3f1ace 100644 --- a/site/content/3.12/develop/http-api/indexes/geo-spatial.md +++ b/site/content/3.12/develop/http-api/indexes/geo-spatial.md @@ -5,7 +5,6 @@ weight: 25 description: '' archetype: default --- - ## Create a geo-spatial index ```openapi diff --git a/site/content/3.12/develop/http-api/indexes/inverted.md b/site/content/3.12/develop/http-api/indexes/inverted.md index de489c6dfe..6a7d69c55f 100644 --- a/site/content/3.12/develop/http-api/indexes/inverted.md +++ b/site/content/3.12/develop/http-api/indexes/inverted.md @@ -5,7 +5,6 @@ weight: 10 description: '' archetype: default --- - ## Create an inverted index ```openapi diff --git a/site/content/3.12/develop/http-api/indexes/multi-dimensional.md b/site/content/3.12/develop/http-api/indexes/multi-dimensional.md index 2bc1289168..4b93f98f1f 100644 --- a/site/content/3.12/develop/http-api/indexes/multi-dimensional.md +++ b/site/content/3.12/develop/http-api/indexes/multi-dimensional.md @@ -5,7 +5,6 @@ weight: 20 description: '' archetype: default --- - ## Create a multi-dimensional index ```openapi @@ -83,7 +82,9 @@ paths: Attributes in `storedValues` cannot overlap with attributes specified in `prefixFields` but you can have the attributes - in both `storedValues` and `fields`. + in both `storedValues` and `fields` because the attributes + in `fields` cannot be used for projections to cover queries + with the indexed data. type: array items: type: string diff --git a/site/content/3.12/develop/http-api/indexes/ttl.md b/site/content/3.12/develop/http-api/indexes/ttl.md index 847342e1cf..37c29a46c1 100644 --- a/site/content/3.12/develop/http-api/indexes/ttl.md +++ b/site/content/3.12/develop/http-api/indexes/ttl.md @@ -5,7 +5,6 @@ weight: 15 description: '' archetype: default --- - ## Create a TTL index ```openapi diff --git a/site/content/3.12/develop/http-api/replication/other-replication-commands.md b/site/content/3.12/develop/http-api/replication/other-replication-commands.md index 7fe5facc47..6adfffc8ab 100644 --- a/site/content/3.12/develop/http-api/replication/other-replication-commands.md +++ b/site/content/3.12/develop/http-api/replication/other-replication-commands.md @@ -5,7 +5,6 @@ weight: 20 description: '' archetype: default --- - ## Get the replication server ID ```openapi From 8af98c447a5c8b8948d8efa8a56a349c380fe93c Mon Sep 17 00:00:00 2001 From: Simran Spiller Date: Thu, 8 Feb 2024 09:47:57 +0100 Subject: [PATCH 4/8] Improve index progress API changes --- .../version-3.10/api-changes-in-3-10.md | 11 +++++------ .../version-3.10/api-changes-in-3-10.md | 11 +++++------ .../version-3.11/api-changes-in-3-11.md | 13 ++++++------- .../version-3.10/api-changes-in-3-10.md | 11 +++++------ .../version-3.11/api-changes-in-3-11.md | 13 ++++++------- .../version-3.12/api-changes-in-3-12.md | 11 +++++------ 6 files changed, 32 insertions(+), 38 deletions(-) diff --git a/site/content/3.10/release-notes/version-3.10/api-changes-in-3-10.md b/site/content/3.10/release-notes/version-3.10/api-changes-in-3-10.md index 9822efdb6c..f7ef11cee5 100644 --- a/site/content/3.10/release-notes/version-3.10/api-changes-in-3-10.md +++ b/site/content/3.10/release-notes/version-3.10/api-changes-in-3-10.md @@ -766,16 +766,15 @@ figures, and for `arangosearch` Views, `withHidden` needs to be enabled, too: Introduced in: v3.10.13 -The `GET /_api/index` endpoint now returns a `progress` attribute that can -optionally show indexes that are currently being created and indicate progress -on the index generation. +The `GET /_api/index` endpoint may now include a `progress` attribute for the +elements in the `indexes` array. For every index that is currently being created, +it indicates the progress of the index generation (in percent). To return indexes that are not yet fully built but are in the building phase, -add the option `withHidden=true` to `GET /_api/index?collection=`. +add the `withHidden=true` query parameter to the call of the endpoint. ``` -curl --header 'accept: application/json' --dump - -"http://localhost:8529/_api/index?collection=myCollection&withHidden=true" +curl "http://localhost:8529/_api/index?collection=myCollection&withHidden=true" ``` #### Document API diff --git a/site/content/3.11/release-notes/version-3.10/api-changes-in-3-10.md b/site/content/3.11/release-notes/version-3.10/api-changes-in-3-10.md index 78afa1e38d..cd49fb466e 100644 --- a/site/content/3.11/release-notes/version-3.10/api-changes-in-3-10.md +++ b/site/content/3.11/release-notes/version-3.10/api-changes-in-3-10.md @@ -766,16 +766,15 @@ figures, and for `arangosearch` Views, `withHidden` needs to be enabled, too: Introduced in: v3.10.13 -The `GET /_api/index` endpoint now returns a `progress` attribute that can -optionally show indexes that are currently being created and indicate progress -on the index generation. +The `GET /_api/index` endpoint may now include a `progress` attribute for the +elements in the `indexes` array. For every index that is currently being created, +it indicates the progress of the index generation (in percent). To return indexes that are not yet fully built but are in the building phase, -add the option `withHidden=true` to `GET /_api/index?collection=`. +add the `withHidden=true` query parameter to the call of the endpoint. ``` -curl --header 'accept: application/json' --dump - -"http://localhost:8529/_api/index?collection=myCollection&withHidden=true" +curl "http://localhost:8529/_api/index?collection=myCollection&withHidden=true" ``` #### Document API diff --git a/site/content/3.11/release-notes/version-3.11/api-changes-in-3-11.md b/site/content/3.11/release-notes/version-3.11/api-changes-in-3-11.md index cc805efe02..a08179d7e8 100644 --- a/site/content/3.11/release-notes/version-3.11/api-changes-in-3-11.md +++ b/site/content/3.11/release-notes/version-3.11/api-changes-in-3-11.md @@ -484,17 +484,16 @@ and [Monitoring per collection/database/user](../version-3.11/whats-new-in-3-11. Introduced in: v3.10.13, v3.11.7 -The `GET /_api/index` endpoint now returns a `progress` attribute that can -optionally show indexes that are currently being created and indicate progress -on the index generation. +The `GET /_api/index` endpoint may now include a `progress` attribute for the +elements in the `indexes` array. For every index that is currently being created, +it indicates the progress of the index generation (in percent). To return indexes that are not yet fully built but are in the building phase, -add the option `withHidden=true` to `GET /_api/index?collection=`. +add the `withHidden=true` query parameter to the call of the endpoint. ``` -curl --header 'accept: application/json' --dump - -"http://localhost:8529/_api/index?collection=myCollection&withHidden=true" -``` +curl "http://localhost:8529/_api/index?collection=myCollection&withHidden=true" +``` ### Endpoints deprecated diff --git a/site/content/3.12/release-notes/version-3.10/api-changes-in-3-10.md b/site/content/3.12/release-notes/version-3.10/api-changes-in-3-10.md index 78afa1e38d..cd49fb466e 100644 --- a/site/content/3.12/release-notes/version-3.10/api-changes-in-3-10.md +++ b/site/content/3.12/release-notes/version-3.10/api-changes-in-3-10.md @@ -766,16 +766,15 @@ figures, and for `arangosearch` Views, `withHidden` needs to be enabled, too: Introduced in: v3.10.13 -The `GET /_api/index` endpoint now returns a `progress` attribute that can -optionally show indexes that are currently being created and indicate progress -on the index generation. +The `GET /_api/index` endpoint may now include a `progress` attribute for the +elements in the `indexes` array. For every index that is currently being created, +it indicates the progress of the index generation (in percent). To return indexes that are not yet fully built but are in the building phase, -add the option `withHidden=true` to `GET /_api/index?collection=`. +add the `withHidden=true` query parameter to the call of the endpoint. ``` -curl --header 'accept: application/json' --dump - -"http://localhost:8529/_api/index?collection=myCollection&withHidden=true" +curl "http://localhost:8529/_api/index?collection=myCollection&withHidden=true" ``` #### Document API diff --git a/site/content/3.12/release-notes/version-3.11/api-changes-in-3-11.md b/site/content/3.12/release-notes/version-3.11/api-changes-in-3-11.md index cc805efe02..a08179d7e8 100644 --- a/site/content/3.12/release-notes/version-3.11/api-changes-in-3-11.md +++ b/site/content/3.12/release-notes/version-3.11/api-changes-in-3-11.md @@ -484,17 +484,16 @@ and [Monitoring per collection/database/user](../version-3.11/whats-new-in-3-11. Introduced in: v3.10.13, v3.11.7 -The `GET /_api/index` endpoint now returns a `progress` attribute that can -optionally show indexes that are currently being created and indicate progress -on the index generation. +The `GET /_api/index` endpoint may now include a `progress` attribute for the +elements in the `indexes` array. For every index that is currently being created, +it indicates the progress of the index generation (in percent). To return indexes that are not yet fully built but are in the building phase, -add the option `withHidden=true` to `GET /_api/index?collection=`. +add the `withHidden=true` query parameter to the call of the endpoint. ``` -curl --header 'accept: application/json' --dump - -"http://localhost:8529/_api/index?collection=myCollection&withHidden=true" -``` +curl "http://localhost:8529/_api/index?collection=myCollection&withHidden=true" +``` ### Endpoints deprecated diff --git a/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md b/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md index 7897a68488..a7a52490d9 100644 --- a/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md +++ b/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md @@ -298,16 +298,15 @@ for details. Introduced in: v3.10.13, v3.11.7 -The `GET /_api/index` endpoint now returns a `progress` attribute that can -optionally show indexes that are currently being created and indicate progress -on the index generation. +The `GET /_api/index` endpoint may now include a `progress` attribute for the +elements in the `indexes` array. For every index that is currently being created, +it indicates the progress of the index generation (in percent). To return indexes that are not yet fully built but are in the building phase, -add the option `withHidden=true` to `GET /_api/index?collection=`. +add the `withHidden=true` query parameter to the call of the endpoint. ``` -curl --header 'accept: application/json' --dump - -"http://localhost:8529/_api/index?collection=myCollection&withHidden=true" +curl "http://localhost:8529/_api/index?collection=myCollection&withHidden=true" ``` #### Optimizer rule descriptions From 0bdb0fc79669007b9cc1dd6c27943b368b1c540a Mon Sep 17 00:00:00 2001 From: Simran Spiller Date: Fri, 9 Feb 2024 14:11:44 +0100 Subject: [PATCH 5/8] WIP --- .../develop/http-api/indexes/persistent.md | 4 +- .../multi-dimensional-indexes.md | 100 ++++++++++++++++-- .../version-3.12/api-changes-in-3-12.md | 11 +- .../version-3.12/whats-new-in-3-12.md | 4 +- 4 files changed, 101 insertions(+), 18 deletions(-) diff --git a/site/content/3.12/develop/http-api/indexes/persistent.md b/site/content/3.12/develop/http-api/indexes/persistent.md index e0358ea39e..50c950a08b 100644 --- a/site/content/3.12/develop/http-api/indexes/persistent.md +++ b/site/content/3.12/develop/http-api/indexes/persistent.md @@ -125,8 +125,8 @@ paths: competing indexes in AQL queries when there are multiple candidate indexes to choose from. - The `estimates` attribute is optional and defaults to `true` if not set. It will - have no effect on indexes other than `persistent`. + The `estimates` attribute is optional and defaults to `true` if not set. + It has no effect on indexes other than `persistent`, `mdi`, and `mdi-prefixed`. type: boolean cacheEnabled: description: | diff --git a/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md b/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md index fef3a5b94e..2e3b9dbd16 100644 --- a/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md +++ b/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md @@ -7,11 +7,40 @@ description: >- such as time ranges, for efficient intersection of multiple range queries archetype: default --- -The multi-dimensional index type was previously called `"zkd"`. +A multi-dimensional index maps multi-dimensional data in the form of multiple +numeric attributes to one dimension while mostly preserving locality so that +similar values in all of the dimensions remain close to each other in the mapping +to a single dimension. Queries that filter by multiple value ranges at once can +be better accelerated with such an index compared to a persistent index. -A multi-dimensional index is setup by setting the index type to `"mdi"`. -The `fields` attribute describes which fields are used as dimensions. -The value of each dimension has to be a numeric (double) value. +You can choose between two subtypes of multi-dimensional indexes: + +- An `mdi` index with a `fields` property that describes which document + attributes to use as dimensions +- An `mdi-prefixed` index with a `fields` property as well as a `prefixFields` + property to specify one or more mandatory document attributes to narrow down + the search space using equality checks + +Both subtypes require that the attributes described by `fields` have numeric +values. You can optionally omit documents from the index that have any of +the `fields` or `prefixFields` attributes not set or set to `null` by declaring +the index as `sparse`. + +Multi-dimensional indexes can be declared as `unique` to only allow a single +document with a given combination of attribute values, using all of the `prefix` +attributes and (for `mdi-prefixed` indexes) `prefixFields`. Documents omitted +because of `sparse: true` are exempt. + +You can store additional attributes in multi-dimensional indexes with the +`storedValues` property. They can be used for projections (unlike the `fields` +attributes) so that indexes can cover more queries without having to access the +full documents. + +`estimates` + +{{< info >}} +The `mdi` index type was previously called `zkd`. +{{< /info >}} ## Querying documents within a 3D box @@ -35,10 +64,10 @@ db.collection.ensureIndex({ }); ``` -Unlike for other indexes the order of the fields does not matter. +Unlike with other indexes, the order of the `fields` does not matter. -`fieldValueTypes` is required and the only allowed value is `"double"`. -Future extensions of the index will allow other types. +`fieldValueTypes` is required and the only allowed value is `"double"` to use a +double-precision (64-bit) floating-point format internally. Now we can use the index in a query: @@ -124,6 +153,61 @@ FOR app IN appointments RETURN app ``` +## Prefix fields + + + +## Storing additional values in indexes + +Introduced in: v3.10.0 + +Multi-dimensional indexes allow you to store additional attributes in the index +that can be used to satisfy projections of the document. They cannot be used for +index lookups or for sorting, but for projections only. They allow multi-dimensional +indexes to fully cover more queries and avoid extra document lookups. This can +have a great positive effect on index scan performance if the number of scanned +index entries is large. + +You can set the `storedValues` option and specify the additional attributes as +an array of attribute paths when creating a new `mdi` or `mdi-prefixed` index, +similar to the `fields` option: + +```js +db..ensureIndex({ + type: "mdi", + fields: ["value1"], + storedValues: ["value1", "value2"] +}); +``` + +This will index the `value1` attribute in the traditional sense, so that the index +can be used for looking up by `value1` or for sorting by `value1`. The index also +supports projections on `value1` as usual. + +In addition, due to `storedValues` being used here, the index can now also +supply the values for the `value2` attribute for projections without having to +look up the full document. Non-existing attributes are stored as `null` values. + +Attributes in `storedValues` cannot overlap with the attributes specified in +`prefixFields` but you can have the attributes in both `storedValues` and +`fields` as the attributes in `fields` cannot be used for projections to +cover queries because the indexed data is different to the attribute values. + +The maximum number of attributes that you can use in `storedValues` is 32. +You cannot specify the same attribute path in both, the `fields` and the +`storedValues` option. If there is an overlap, the index creation will abort +with an error message. + +In unique indexes, only the index attributes in `fields` and (for `mdi-prefixed` +indexes) `prefixFields` are checked for uniqueness. The index attributes in +`storedValues` are not checked for their uniqueness. + +You cannot create multiple multi-dimensional indexes with the same `fields` +attributes but different `storedValues` settings. That means the value of +`storedValues` is not considered by calls to `ensureIndex()` when checking if an +index is already present or needs to be created. + + ## Lookahead Index Hint Introduced in: v3.10.0 @@ -147,8 +231,6 @@ FOR app IN appointments OPTIONS { lookahead: 32 } ## Limitations -Currently there are a few limitations: - - Using array expansions for attributes is not possible (e.g. `array[*].attr`) - You can only index numeric values that are representable as IEEE-754 double. - A high number of dimensions (more than 5) can impact the performance considerably. diff --git a/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md b/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md index a7a52490d9..436825da64 100644 --- a/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md +++ b/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md @@ -284,12 +284,13 @@ discouraged. The `zkd` alias may get removed in a future version. An additional `mdi-prefixed` index variant has been added. This is a new index type in the API with the same settings as the `mdi` index but with one additional -`prefixFields` attribute. It is a required setting and accepts an array of strings -similar to the `fields` attribute. +`prefixFields` attribute. It is a required setting for the `mdi-prefixed` index +type and accepts an array of strings similar to the `fields` attribute. You can +use it to narrow down the search space using equality checks. -Both multi-dimensional index variants now support a `sparse` (boolean) and -`storedValues` (array of strings) setting that were not supported by the `zkd` -index in previous versions. +Both multi-dimensional index variants now support a `sparse` setting (boolean) +and `storedValues` setting (array of strings) that were not supported by the +`zkd` index type in previous versions. See [Working with multi-dimensional indexes](../../index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md) for details. diff --git a/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md b/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md index b9d9f1bd3c..300dc5afc3 100644 --- a/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md +++ b/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md @@ -362,9 +362,9 @@ The previously experimental `zkd` index is now stable and has been renamed to Multi-dimensional indexes can now be declared as `sparse` to exclude documents from the index that do not have the defined attributes or are explicitly set to -`null` values. If a non-value is set, it still needs to be numeric. +`null` values. If a value other than `null` is set, it still needs to be numeric. -Multi-dimensional indexes now support stored values to cover queries for better +Multi-dimensional indexes now support `storedValues` to cover queries for better performance. An additional `mdi-prefixed` index variant has been added that lets you specify From 37c897cafbd7338f4d9908210345d5cb81bbc595 Mon Sep 17 00:00:00 2001 From: Simran Spiller Date: Sat, 17 Feb 2024 02:29:22 +0100 Subject: [PATCH 6/8] Docs with examples --- .../multi-dimensional-indexes.md | 19 ++- .../multi-dimensional-indexes.md | 19 ++- .../http-api/indexes/multi-dimensional.md | 46 ++++-- .../develop/http-api/indexes/persistent.md | 4 +- .../3.12/index-and-search/indexing/basics.md | 12 ++ .../indexing/working-with-indexes/_index.md | 28 ++-- .../multi-dimensional-indexes.md | 142 ++++++++++++------ .../vertex-centric-indexes.md | 124 ++++++++++----- .../version-3.12/api-changes-in-3-12.md | 4 +- .../version-3.12/whats-new-in-3-12.md | 8 +- 10 files changed, 280 insertions(+), 126 deletions(-) diff --git a/site/content/3.10/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md b/site/content/3.10/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md index d6596cab75..fc05e5cba4 100644 --- a/site/content/3.10/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md +++ b/site/content/3.10/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md @@ -7,15 +7,26 @@ description: >- such as time ranges, for efficient intersection of multiple range queries archetype: default --- -The multi-dimensional index type is called **ZKD**. +A multi-dimensional index maps multi-dimensional data in the form of multiple +numeric attributes to one dimension while mostly preserving locality so that +similar values in all of the dimensions remain close to each other in the mapping +to a single dimension. Queries that filter by multiple value ranges at once can +be better accelerated with such an index compared to a persistent index. + +The multi-dimensional index type is called `zkd`. {{< warning >}} `zkd` indexes are an **experimental** feature. {{< /warning >}} -A multi-dimensional index is setup by setting the index type to `"zkd"`. -The `fields` attribute describes which fields are used as dimensions. -The value of each dimension has to be a numeric (double) value. + +The `fields` property of a `zkd` index describes which document attributes to +use as dimensions. It is required that the attributes are present and have +numeric values. + +Multi-dimensional indexes can be declared as `unique` to only allow a single +document with a given combination of attribute values, using all of the `fields` +attributes. ## Querying documents within a 3D box diff --git a/site/content/3.11/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md b/site/content/3.11/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md index d6596cab75..fc05e5cba4 100644 --- a/site/content/3.11/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md +++ b/site/content/3.11/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md @@ -7,15 +7,26 @@ description: >- such as time ranges, for efficient intersection of multiple range queries archetype: default --- -The multi-dimensional index type is called **ZKD**. +A multi-dimensional index maps multi-dimensional data in the form of multiple +numeric attributes to one dimension while mostly preserving locality so that +similar values in all of the dimensions remain close to each other in the mapping +to a single dimension. Queries that filter by multiple value ranges at once can +be better accelerated with such an index compared to a persistent index. + +The multi-dimensional index type is called `zkd`. {{< warning >}} `zkd` indexes are an **experimental** feature. {{< /warning >}} -A multi-dimensional index is setup by setting the index type to `"zkd"`. -The `fields` attribute describes which fields are used as dimensions. -The value of each dimension has to be a numeric (double) value. + +The `fields` property of a `zkd` index describes which document attributes to +use as dimensions. It is required that the attributes are present and have +numeric values. + +Multi-dimensional indexes can be declared as `unique` to only allow a single +document with a given combination of attribute values, using all of the `fields` +attributes. ## Querying documents within a 3D box diff --git a/site/content/3.12/develop/http-api/indexes/multi-dimensional.md b/site/content/3.12/develop/http-api/indexes/multi-dimensional.md index 4b93f98f1f..0feee4033c 100644 --- a/site/content/3.12/develop/http-api/indexes/multi-dimensional.md +++ b/site/content/3.12/develop/http-api/indexes/multi-dimensional.md @@ -69,22 +69,27 @@ paths: attributes to store in the index. These additional attributes cannot be used for index lookups or for sorting, but they can be used for projections. This allows an index to fully cover more queries and avoid extra document lookups. - The maximum number of attributes in `storedValues` is 32. - It is not possible to create multiple indexes with the same `fields` attributes - and uniqueness but different `storedValues` attributes. That means the value of - `storedValues` is not considered by index creation calls when checking if an - index is already present or needs to be created. + You can have the same attributes in `storedValues` and `fields` as the attributes + in `fields` cannot be used for projections, but you can also store additional + attributes that are not listed in `fields`. + Attributes in `storedValues` cannot overlap with the attributes specified in + `prefixFields`. There is no reason to store them in the index because you need + to specify them in queries in order to use `mdi-prefixed` indexes. + + You cannot create multiple multi-dimensional indexes with the same `sparse`, + `unique`, `fields` and (for `mdi-prefixed` indexes) `prefixFields` attributes + but different `storedValues` settings. That means the value of `storedValues` is + not considered by index creation calls when checking if an index is already + present or needs to be created. + + In unique indexes, only the index attributes in `fields` and (for `mdi-prefixed` + indexes) `prefixFields` are checked for uniqueness. The index attributes in + `storedValues` are not checked for their uniqueness. - In unique indexes, only the attributes in `fields` are checked for uniqueness, - but the attributes in `storedValues` are not checked for their uniqueness. Non-existing attributes are stored as `null` values inside `storedValues`. - Attributes in `storedValues` cannot overlap with attributes - specified in `prefixFields` but you can have the attributes - in both `storedValues` and `fields` because the attributes - in `fields` cannot be used for projections to cover queries - with the indexed data. + The maximum number of attributes in `storedValues` is 32. type: array items: type: string @@ -98,6 +103,23 @@ paths: If `true`, then create a sparse index. type: boolean default: false + estimates: + description: | + This attribute controls whether index selectivity estimates are maintained for the + index. Not maintaining index selectivity estimates can have a slightly positive + impact on write performance. + + The downside of turning off index selectivity estimates is that + the query optimizer is not able to determine the usefulness of different + competing indexes in AQL queries when there are multiple candidate indexes to + choose from. + + The `estimates` attribute is optional and defaults to `true` if not set. + It has no effect on indexes other than `persistent`, `mdi`, and `mdi-prefixed`. + It cannot be disabled for non-unique `mdi` indexes because they have a fixed + selectivity estimate of `1`. + type: boolean + default: true inBackground: description: | You can set this option to `true` to create the index diff --git a/site/content/3.12/develop/http-api/indexes/persistent.md b/site/content/3.12/develop/http-api/indexes/persistent.md index 50c950a08b..d31952f227 100644 --- a/site/content/3.12/develop/http-api/indexes/persistent.md +++ b/site/content/3.12/develop/http-api/indexes/persistent.md @@ -120,8 +120,8 @@ paths: index. Not maintaining index selectivity estimates can have a slightly positive impact on write performance. - The downside of turning off index selectivity estimates will be that - the query optimizer will not be able to determine the usefulness of different + The downside of turning off index selectivity estimates is that + the query optimizer is not able to determine the usefulness of different competing indexes in AQL queries when there are multiple candidate indexes to choose from. diff --git a/site/content/3.12/index-and-search/indexing/basics.md b/site/content/3.12/index-and-search/indexing/basics.md index 5b052ea77d..378818fd8e 100644 --- a/site/content/3.12/index-and-search/indexing/basics.md +++ b/site/content/3.12/index-and-search/indexing/basics.md @@ -152,6 +152,18 @@ based on the costs it estimates, even if a vertex-centric index might in fact be faster. Vertex-centric indexes are more likely to be chosen for highly connected graphs. +You can also use use prefixed multi-dimensional indexes to combine graph +traversals with range queries: + +```aql +FOR v, e, p in 0..3 INBOUND @start GRAPH @graphName + OPTIONS { order: "bfs", uniqueVertices: "path" } + FILTER p.edges[*].type ALL == "friend" + AND p.edges[*].x ALL >= 5 + AND p.edges[*].y ALL <= 7 + RETURN p +``` + ## Persistent Index The persistent index is a sorted index with logarithmic complexity for insert, diff --git a/site/content/3.12/index-and-search/indexing/working-with-indexes/_index.md b/site/content/3.12/index-and-search/indexing/working-with-indexes/_index.md index 91ddb0596e..cc5876315f 100644 --- a/site/content/3.12/index-and-search/indexing/working-with-indexes/_index.md +++ b/site/content/3.12/index-and-search/indexing/working-with-indexes/_index.md @@ -127,7 +127,7 @@ Other attributes may be necessary, depending on the index type. - `"fulltext"`: full-text index (deprecated from ArangoDB 3.10 onwards) - `"geo"`: geo-spatial index, with _one_ or _two_ attributes - `"mdi"`: multi-dimensional index - - `"mdi-prefixed"`: multi-dimensional index with search prefix + - `"mdi-prefixed"`: multi-dimensional index with search prefix, including vertex-centric index - `fields`: an array of attribute paths, containing the document attributes (or sub-attributes) to be indexed. Some indexes allow using only a single path, @@ -143,17 +143,19 @@ Other attributes may be necessary, depending on the index type. that the index attribute value is treated as an array and all array members are indexed separately. This is possible with `persistent` and `inverted` indexes. -- `storedValues`: in indexes of type `persistent` and `inverted`, additional +- `storedValues`: in indexes of type `persistent`, `inverted`, `mdi`, and `mdi-prefixed`, additional attributes can be stored in the index. These additional attributes cannot be used for index lookups or for sorting, but they can be used for projections. This allows an index to fully cover more queries and avoid extra document lookups. Non-existing attributes are stored as `null` values inside `storedValues`. The maximum number of attributes in `storedValues` is 32. It is not possible to create multiple indexes with the same `fields` attributes - and uniqueness but different `storedValues` attributes. That means the value of + (and `sparse`, `unique`, and `prefixFields` attributes if supported) + but different `storedValues` attributes. That means the value of `storedValues` is not considered in index creation calls when checking if an index is already present or needs to be created. - In unique indexes, only the attributes in `fields` are checked for uniqueness, + In unique indexes, only the attributes in `fields` and (for `mdi-prefixed` + indexes) `prefixFields` are checked for uniqueness, but the attributes in `storedValues` are not checked for their uniqueness. - `name`: can be a string. Index names are subject to the same character @@ -166,11 +168,12 @@ Other attributes may be necessary, depending on the index type. If no index hints are used, going with the auto-generated index names is fine. - `sparse`: can be `true` or `false`. - You can control the sparsity for `persistent` indexes. The `inverted`, `fulltext`, - and `geo` index types are [sparse](../which-index-to-use-when.md) by definition. + You can control the sparsity for `persistent`, `mdi`, and `mdi-prefixed` indexes. + The `inverted`, `fulltext`, and `geo` index types are + [sparse](../which-index-to-use-when.md) by definition. -- `unique`: can be `true` or `false` and is supported by `persistent` indexes. - By default, all user-defined indexes are non-unique. +- `unique`: can be `true` or `false` and is supported by `persistent`, `mdi`, + and `mdi-prefixed` indexes. By default, all user-defined indexes are non-unique. Only the attributes in `fields` are checked for uniqueness. Any attributes in from `storedValues` are not checked for their uniqueness. @@ -186,13 +189,12 @@ Other attributes may be necessary, depending on the index type. `persistent`. This attribute controls whether index selectivity estimates are maintained for the index. Not maintaining index selectivity estimates can have a slightly positive impact on write performance. - The downside of turning off index selectivity estimates will be that - the query optimizer will not be able to determine the usefulness of different + The downside of turning off index selectivity estimates is that + the query optimizer is not able to determine the usefulness of different competing indexes in AQL queries when there are multiple candidate indexes to choose from. - The `estimates` attribute is optional and defaults to `true` if not set. It will - have no effect on indexes other than `persistent` (with `hash` and `skiplist` - being mere aliases for the `persistent` index type nowadays). + The `estimates` attribute is optional and defaults to `true` if not set. It + has no effect on indexes other than `persistent`, `mdi`, and `mdi-prefixed`. - `cacheEnabled`: can be `true` or `false` and is supported by indexes of type `persistent`. The attribute controls whether an extra in-memory hash cache is diff --git a/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md b/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md index 2e3b9dbd16..74bcb98243 100644 --- a/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md +++ b/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md @@ -24,19 +24,26 @@ You can choose between two subtypes of multi-dimensional indexes: Both subtypes require that the attributes described by `fields` have numeric values. You can optionally omit documents from the index that have any of the `fields` or `prefixFields` attributes not set or set to `null` by declaring -the index as `sparse`. +the index as sparse with `sparse: true`. -Multi-dimensional indexes can be declared as `unique` to only allow a single -document with a given combination of attribute values, using all of the `prefix` -attributes and (for `mdi-prefixed` indexes) `prefixFields`. Documents omitted -because of `sparse: true` are exempt. +Multi-dimensional indexes can be created with a uniqueness constraint with +`unique: true`. This only allows a single document with a given combination of +attribute values, using all of the `fields` attributes and (for `mdi-prefixed` +indexes) `prefixFields`. Documents omitted because of `sparse: true` are exempt. You can store additional attributes in multi-dimensional indexes with the `storedValues` property. They can be used for projections (unlike the `fields` attributes) so that indexes can cover more queries without having to access the full documents. -`estimates` +Non-unique `mdi` indexes have a fixed selectivity estimate of `1`. For `mdi` +indexes with `unique: true` as well as for `mdi-prefixed` indexes, you can +control whether index selectivity estimates are maintained for the index. +It is enabled by default and you can disable it with `estimates: false`. +Not maintaining index selectivity estimates can have a slightly positive impact +on write performance but the query optimizer is not able to determine the +usefulness of different competing indexes in AQL queries when there are multiple +candidate indexes to choose from. {{< info >}} The `mdi` index type was previously called `zkd`. @@ -57,10 +64,10 @@ To do so one creates a multi-dimensional index on the attributes `x`, `y` and `z`, e.g. in _arangosh_: ```js -db.collection.ensureIndex({ - type: "mdi", - fields: ["x", "y", "z"], - fieldValueTypes: "double" +db.points.ensureIndex({ + type: "mdi", + fields: ["x", "y", "z"], + fieldValueTypes: "double" }); ``` @@ -73,10 +80,10 @@ Now we can use the index in a query: ```aql FOR p IN points - FILTER x0 <= p.x && p.x <= x1 - FILTER y0 <= p.y && p.y <= y1 - FILTER z0 <= p.z && p.z <= z1 - RETURN p + FILTER x0 <= p.x && p.x <= x1 + FILTER y0 <= p.y && p.y <= y1 + FILTER z0 <= p.z && p.z <= z1 + RETURN p ``` ## Possible range queries @@ -92,10 +99,10 @@ is translated to their non-strict counterparts and a post-filter is inserted. ```aql FOR p IN points - FILTER 2 <= p.x && p.x < 9 - FILTER y0 >= 80 - FILTER p.z == 4 - RETURN p + FILTER 2 <= p.x && p.x < 9 + FILTER p.y >= 80 + FILTER p.z == 4 + RETURN p ``` ## Example Use Case @@ -105,9 +112,9 @@ that contains the appointments. The documents would roughly look as follows: ```json { - "from": 345365, - "to": 678934, - "what": "Dentist", + "from": 345365, + "to": 678934, + "what": "Dentist", } ``` @@ -129,9 +136,9 @@ Thus our query would be: ```aql FOR app IN appointments - FILTER f <= app.from - FILTER app.to <= t - RETURN app + FILTER f <= app.from + FILTER app.to <= t + RETURN app ``` ### Finding all appointments that intersect a time range @@ -148,18 +155,51 @@ Thus our query would be: ```aql FOR app IN appointments - FILTER f <= app.to - FILTER app.from <= t - RETURN app + FILTER f <= app.to + FILTER app.from <= t + RETURN app ``` ## Prefix fields +Multi-dimensional indexes can accelerate range queries well but they are +inefficient for queries that check for equality of values. For use cases where +you have a combination of equality and range conditions in queries, you can use +the `mdi-prefixed` subtype instead of `mdi`. It has all the features of the +`mdi` subtype but additionally lets you define one or more document attributes +you want to use for equality checks. This combination allows to efficiently +narrow down the search space to a subset of multi-dimensional index data before +performing the range checking. + +```js +db..ensureIndex({ + type: "mdi-prefixed", + prefixFields: ["v", "w"] + fields: ["x", "y"], + fieldValueTypes: "double" +}); +``` +You need to specify all of the `prefixFields` attributes in your queries to +utilize the index. + +```aql +FOR p IN points + FILTER p.v == "type" + FILTER p.w == "group" + FILTER 2 <= p.x && p.x < 9 + FILTER p.y >= 80 + RETURN p +``` + +You can create `mdi-prefixed` indexes on edge collections with the `_from` or +`_to` edge attribute as the first prefix field. Graph traversals with range filters +can then utilize such indexes. See [Vertex-centric indexes](vertex-centric-indexes.md) +for details. ## Storing additional values in indexes -Introduced in: v3.10.0 +Introduced in: v3.12.0 Multi-dimensional indexes allow you to store additional attributes in the index that can be used to satisfy projections of the document. They cannot be used for @@ -175,38 +215,42 @@ similar to the `fields` option: ```js db..ensureIndex({ type: "mdi", - fields: ["value1"], - storedValues: ["value1", "value2"] + fields: ["x", "y"], + fieldValueTypes: "double", + storedValues: ["y", "z"] }); ``` - -This will index the `value1` attribute in the traditional sense, so that the index -can be used for looking up by `value1` or for sorting by `value1`. The index also -supports projections on `value1` as usual. -In addition, due to `storedValues` being used here, the index can now also -supply the values for the `value2` attribute for projections without having to -look up the full document. Non-existing attributes are stored as `null` values. +This indexes the `x` and `y` attributes so that the index can be used for range +queries by these attributes. Using these document attributes like for returning +them from the query is not covered by the index, however, unless you add the +attributes to `storedValues` in addition to `fields`. The reason is that the +index doesn't store the original values of the attributes. -Attributes in `storedValues` cannot overlap with the attributes specified in -`prefixFields` but you can have the attributes in both `storedValues` and -`fields` as the attributes in `fields` cannot be used for projections to -cover queries because the indexed data is different to the attribute values. +You can have the same attributes in `storedValues` and `fields` as the attributes +in `fields` cannot be used for projections, but you can also store additional +attributes that are not listed in `fields`. +The above example stores the `y` and `z` attribute values in the index using +`storedValues`. The index can thus supply the values for projections without +having to look up the full document. -The maximum number of attributes that you can use in `storedValues` is 32. -You cannot specify the same attribute path in both, the `fields` and the -`storedValues` option. If there is an overlap, the index creation will abort -with an error message. +Attributes in `storedValues` cannot overlap with the attributes specified in +`prefixFields`. There is no reason to store them in the index because you need +to specify them in queries in order to use `mdi-prefixed` indexes. In unique indexes, only the index attributes in `fields` and (for `mdi-prefixed` indexes) `prefixFields` are checked for uniqueness. The index attributes in `storedValues` are not checked for their uniqueness. -You cannot create multiple multi-dimensional indexes with the same `fields` -attributes but different `storedValues` settings. That means the value of -`storedValues` is not considered by calls to `ensureIndex()` when checking if an -index is already present or needs to be created. +You cannot create multiple multi-dimensional indexes with the same `sparse`, +`unique`, `fields` and (for `mdi-prefixed` indexes) `prefixFields` attributes +but different `storedValues` settings. That means the value of `storedValues` is +not considered by index creation calls when checking if an index is already +present or needs to be created. +Non-existing attributes are stored as `null` values. + +The maximum number of attributes that you can use in `storedValues` is 32. ## Lookahead Index Hint diff --git a/site/content/3.12/index-and-search/indexing/working-with-indexes/vertex-centric-indexes.md b/site/content/3.12/index-and-search/indexing/working-with-indexes/vertex-centric-indexes.md index 5c8979c7c7..b3a7708b59 100644 --- a/site/content/3.12/index-and-search/indexing/working-with-indexes/vertex-centric-indexes.md +++ b/site/content/3.12/index-and-search/indexing/working-with-indexes/vertex-centric-indexes.md @@ -3,25 +3,35 @@ title: Vertex-Centric Indexes menuTitle: Vertex-Centric Indexes weight: 35 description: >- - You can create persistent indexes over the `_from` or `_to` attribute and one + You can create indexes over the `_from` or `_to` attribute and one or more additional edge attributes to improve certain graph traversals archetype: default --- -## Introduction to Vertex-Centric Indexes - -In ArangoDB there are special indexes designed to speed up graph operations, -especially if the graph contains supernodes (vertices that have an exceptionally -high amount of connected edges). -These indexes are called vertex-centric indexes and can be used in addition -to the existing edge index. +All edge collections in ArangoDB have a special edge index that enables fast +graph operations. If you have graphs that contain supernodes (vertices that have +an exceptionally high amount of connected edges) and you apply filters in graph +traversal queries, you can create so-called vertex-centric indexes that can +perform better than the default edge indexes. You can use the `persistent` and +`mdi-prefixed` index types for this purpose. ## Motivation -The idea of this index is to index a combination of a vertex, the direction and any arbitrary -set of other attributes on the edges. -To take an example, if we have an attribute called `type` on the edges, we can use an outbound -vertex-centric index on this attribute to find all edges attached to a vertex with a given `type`. -The following query example could benefit from such an index: +The idea of a vertex-centric index is to index a combination of a vertex, the +direction, and an arbitrary set of attributes on the edges. This can be achieved +by indexing the `_from` or `_to` attribute of an edge as the first field, +which contains the document identifier of a vertex and implicitly captures the +direction, followed by any number of other attributes of an edge. + +To support traversals in `OUTBOUND` direction, you need to index the `_from` +attribute as the first attribute. For the `INBOUND` direction, you need to use +the `_to` attribute. To support both (`ANY` or mixed `INBOUND` and `OUTBOUND` +directions), you need to create two indexes, using `_from` in one and `_to` in +the other as the first attribute the index is over. + +For example, if you have an attribute called `type` on the edges and traverse +in `OUTBOUND` direction, you can create a vertex-centric `persistent` index over +`["_from", "type"]` to find all edges attached to a vertex with a given `type`. +The following query can benefit from such an index: ```aql FOR v, e, p IN 3..5 OUTBOUND @start GRAPH @graphName @@ -29,50 +39,90 @@ FOR v, e, p IN 3..5 OUTBOUND @start GRAPH @graphName RETURN v ``` -Using the built-in edge-index ArangoDB can find the list of all edges attached to the vertex fast, -but still it has to walk through this list and check if all of them have the attribute `type == "friend"`. -Using a vertex-centric index would allow ArangoDB to find all edges for the vertex having the attribute `type == "friend"` -in the same time and can save the iteration to verify the condition. +Using the built-in edge-index, ArangoDB can find the list of all edges attached +to the vertex fast but it still it has to walk through this list and check if +all of them have the attribute `type == "friend"`. A vertex-centric index allows +ArangoDB to find all edges with the attribute `type == "friend"` for the vertex +in one go, saving the iteration to verify the condition. + +If you have numeric attributes on edges and want to filter by them using value +ranges, perhaps in addition to filtering by a `type` using an equality check, +you can create a vertex-centric `mdi-prefixed` index. Assuming the numeric +attributes are called `x` and `y`, a possible query could look like this: + +```aql +FOR v, e, p in 0..3 INBOUND @start GRAPH @graphName + OPTIONS { order: "bfs", uniqueVertices: "path" } + FILTER p.edges[*].type ALL == "friend" + AND p.edges[*].x ALL >= 5 + AND p.edges[*].y ALL <= 7 + RETURN p +``` ## Index creation A vertex-centric has to be of the type [Persistent Index](persistent-indexes.md) -and is created using its normal creation operations. However, in the list of -fields used to create the index we have to include either `_from` or `_to`. +or prefixed [Multi-dimensional index](multi-dimensional-indexes.md#prefix-fields) +and is created like any other index of the respective type. However, in the list +of fields used to create the index over, you need to use either `_from` or `_to` +as the first field. -Let us again explain this by an example. -Assume we want to create an hash-based outbound vertex-centric index on the attribute `type`. -This can be created with the following way: +For example, if you want to create a vertex-centric index on the `type` attribute +that supports traversing in the `OUTBOUND` direction, you would create the index +in the following way: ```js --- name: ensureVertexCentricIndex description: '' --- -~db._createEdgeCollection("collection"); -db.collection.ensureIndex({ type: "persistent", fields: [ "_from", "type" ] }) -~db._drop("collection"); +~db._createEdgeCollection("edgeCollection"); +db.edgeCollection.ensureIndex({ type: "persistent", fields: [ "_from", "type" ] }); +~db._drop("edgeCollection"); ``` -All options that are supported by persistent indexes are supported by the -vertex-centric index as well. +If you want to create a vertex-centric index on multi-dimensional data in the +`x` and `y` attributes with a `type` attribute as prefix and support traversing +in the `INBOUND` direction, you would create an index as follows: -## Index usage +```js +--- +name: ensureVertexCentricIndexMultidim +description: '' +--- +~db._createEdgeCollection("edgeCollection"); +db.edgeCollection.ensureIndex({ + type: "mdi-prefixed", + prefixFields: ["_to", "type"], + fields: [ "x", "y" ], + fieldValueTypes: "double" +}); +~db._drop("edgeCollection"); +``` -The AQL optimizer can decide to use a vertex-centric whenever suitable, however it is not guaranteed that this -index is used, the optimizer may estimate that an other index is assumed to be better. -The optimizer will consider this type of indexes on explicit filtering of `_from` respectively `_to`: +All options that are supported by persistent or multi-dimensional indexes are +supported by the vertex-centric index as well. -```aql -FOR edge IN collection - FILTER edge._from == "vertices/123456" AND edge.type == "friend" - RETURN edge -``` +## Index usage + +The AQL optimizer can decide to use a vertex-centric whenever suitable. However, +it is not guaranteed that this index is used. The optimizer may estimate that +another index, in particular the built-in edge index, is a better fit. -and during pattern matching queries: +The optimizer considers vertex-centric indexes in pattern matching queries: ```aql FOR v, e, p IN 3..5 OUTBOUND @start GRAPH @graphName FILTER p.edges[*].type ALL == "friend" RETURN v ``` + +It also considers them when you iterate over an edge collection directly and +explicitly filter on `_from` respectively `_to` and the other indexed attributes: + + +```aql +FOR edge IN edgeCollection + FILTER edge._from == "vertices/123456" AND edge.type == "friend" + RETURN edge +``` diff --git a/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md b/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md index 436825da64..411bf989ef 100644 --- a/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md +++ b/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md @@ -277,8 +277,8 @@ for details. ##### Multi-dimensional indexes -The previously experimental `zkd` index is now stable and has been renamed to -`mdi`. Existing indexes keep the `zkd` type. The HTTP API still allows the old +The previously experimental `zkd` index type is now stable and has been renamed +to `mdi`. Existing indexes keep the `zkd` type. The HTTP API still allows the old name to create new indexes that behave exactly like `mdi` indexes but this is discouraged. The `zkd` alias may get removed in a future version. diff --git a/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md b/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md index 300dc5afc3..c49f321907 100644 --- a/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md +++ b/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md @@ -357,8 +357,8 @@ non-collection data. ### Multi-dimensional indexes -The previously experimental `zkd` index is now stable and has been renamed to -`mdi`. Existing indexes keep the `zkd` type. +The previously experimental `zkd` index type is now stable and has been renamed +to `mdi`. Existing indexes keep the `zkd` type. Multi-dimensional indexes can now be declared as `sparse` to exclude documents from the index that do not have the defined attributes or are explicitly set to @@ -369,7 +369,9 @@ performance. An additional `mdi-prefixed` index variant has been added that lets you specify additional attributes for the index to narrow down the search space using -equality checks. +equality checks. It can be used as a vertex-centric index for graph traversals +if created on an edge collection with the first attribute in `prefixFields` set +to `_from` or `_to`. See [Multi-dimensional indexes](../../index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md) for details. From 431c4142028501bda12c7281475a7de78199dc9b Mon Sep 17 00:00:00 2001 From: CircleCI Job Date: Wed, 21 Feb 2024 12:55:28 +0000 Subject: [PATCH 7/8] [skip ci] Automatic commit of generated files from CircleCI --- site/data/3.12/cache.json | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/site/data/3.12/cache.json b/site/data/3.12/cache.json index ab8f65ff23..760fa3585e 100644 --- a/site/data/3.12/cache.json +++ b/site/data/3.12/cache.json @@ -1271,6 +1271,14 @@ "request": "LS0tCmRlc2NyaXB0aW9uOiB8LQogIENyZWF0aW5nIGFuIGludmVydGVkIGluZGV4OgpuYW1lOiBSZXN0SW5kZXhDcmVhdGVOZXdJbnZlcnRlZAotLS0KdmFyIGNuID0gInByb2R1Y3RzIjsKZGIuX2NyZWF0ZShjbik7Cgp2YXIgdXJsID0gIi9fYXBpL2luZGV4P2NvbGxlY3Rpb249IiArIGNuOwp2YXIgYm9keSA9IHsKICB0eXBlOiAiaW52ZXJ0ZWQiLAogIGZpZWxkczogWyAiYSIsIHsgbmFtZTogImIiLCBhbmFseXplcjogInRleHRfZW4iIH0gXQp9OwoKdmFyIHJlc3BvbnNlID0gbG9nQ3VybFJlcXVlc3QoJ1BPU1QnLCB1cmwsIGJvZHkpOwoKYXNzZXJ0KHJlc3BvbnNlLmNvZGUgPT09IDIwMSk7Cgpsb2dKc29uUmVzcG9uc2UocmVzcG9uc2UpOwpkYi5fZHJvcChjbik7", "response": "eyJpbnB1dCI6ImN1cmwgLVggUE9TVCAtLWhlYWRlciAnYWNjZXB0OiBhcHBsaWNhdGlvbi9qc29uJyAtLWRhdGEtYmluYXJ5IEAtIC0tZHVtcCAtIGh0dHA6Ly9sb2NhbGhvc3Q6ODUyOS9fYXBpL2luZGV4P2NvbGxlY3Rpb249cHJvZHVjdHNcbntcbiAgXCJ0eXBlXCI6IFwiaW52ZXJ0ZWRcIixcbiAgXCJmaWVsZHNcIjogW1xuICAgIFwiYVwiLFxuICAgIHtcbiAgICAgIFwibmFtZVwiOiBcImJcIixcbiAgICAgIFwiYW5hbHl6ZXJcIjogXCJ0ZXh0X2VuXCJcbiAgICB9XG4gIF1cbn0iLCJvdXRwdXQiOiJIVFRQLzEuMSAyMDEgQ3JlYXRlZFxuY29udGVudC10eXBlOiBhcHBsaWNhdGlvbi9qc29uXG5jYWNoZS1jb250cm9sOiBuby1jYWNoZSwgbm8tc3RvcmUsIG11c3QtcmV2YWxpZGF0ZSwgcHJlLWNoZWNrPTAsIHBvc3QtY2hlY2s9MCwgbWF4LWFnZT0wLCBzLW1heGFnZT0wXG5jb25uZWN0aW9uOiBLZWVwLUFsaXZlXG5jb250ZW50LWxlbmd0aDogNzcyXG5jb250ZW50LXNlY3VyaXR5LXBvbGljeTogZnJhbWUtYW5jZXN0b3JzICdzZWxmJzsgZm9ybS1hY3Rpb24gJ3NlbGYnO1xuZXhwaXJlczogMFxucHJhZ21hOiBuby1jYWNoZVxuc2VydmVyOiBBcmFuZ29EQlxuc3RyaWN0LXRyYW5zcG9ydC1zZWN1cml0eTogbWF4LWFnZT0zMTUzNjAwMCA7IGluY2x1ZGVTdWJEb21haW5zXG54LWFyYW5nby1xdWV1ZS10aW1lLXNlY29uZHM6IDAuMDAwMDAwXG54LWNvbnRlbnQtdHlwZS1vcHRpb25zOiBub3NuaWZmXG5cbnsgXG4gIFwiYW5hbHl6ZXJcIiA6IFwiaWRlbnRpdHlcIiwgXG4gIFwiY2xlYW51cEludGVydmFsU3RlcFwiIDogMiwgXG4gIFwiY29tbWl0SW50ZXJ2YWxNc2VjXCIgOiAxMDAwLCBcbiAgXCJjb25zb2xpZGF0aW9uSW50ZXJ2YWxNc2VjXCIgOiAxMDAwLCBcbiAgXCJjb25zb2xpZGF0aW9uUG9saWN5XCIgOiB7IFxuICAgIFwidHlwZVwiIDogXCJ0aWVyXCIsIFxuICAgIFwic2VnbWVudHNCeXRlc0Zsb29yXCIgOiAyMDk3MTUyLCBcbiAgICBcInNlZ21lbnRzQnl0ZXNNYXhcIiA6IDUzNjg3MDkxMjAsIFxuICAgIFwic2VnbWVudHNNYXhcIiA6IDEwLCBcbiAgICBcInNlZ21lbnRzTWluXCIgOiAxLCBcbiAgICBcIm1pblNjb3JlXCIgOiAwIFxuICB9LCBcbiAgXCJmZWF0dXJlc1wiIDogWyBcbiAgICBcImZyZXF1ZW5jeVwiLCBcbiAgICBcIm5vcm1cIiBcbiAgXSwgXG4gIFwiZmllbGRzXCIgOiBbIFxuICAgIHsgXG4gICAgICBcIm5hbWVcIiA6IFwiYVwiIFxuICAgIH0sIFxuICAgIHsgXG4gICAgICBcImZlYXR1cmVzXCIgOiBbIFxuICAgICAgICBcImZyZXF1ZW5jeVwiLCBcbiAgICAgICAgXCJwb3NpdGlvblwiLCBcbiAgICAgICAgXCJub3JtXCIgXG4gICAgICBdLCBcbiAgICAgIFwiYW5hbHl6ZXJcIiA6IFwidGV4dF9lblwiLCBcbiAgICAgIFwibmFtZVwiIDogXCJiXCIgXG4gICAgfSBcbiAgXSwgXG4gIFwiaWRcIiA6IFwicHJvZHVjdHMvNjk4ODZcIiwgXG4gIFwiaW5jbHVkZUFsbEZpZWxkc1wiIDogZmFsc2UsIFxuICBcImlzTmV3bHlDcmVhdGVkXCIgOiB0cnVlLCBcbiAgXCJuYW1lXCIgOiBcImlkeF8xNzc3NDgxMDM4Mzg0MDcwNjU2XCIsIFxuICBcIm9wdGltaXplVG9wS1wiIDogWyBdLCBcbiAgXCJwcmltYXJ5U29ydFwiIDogeyBcbiAgICBcImZpZWxkc1wiIDogWyBdLCBcbiAgICBcImNvbXByZXNzaW9uXCIgOiBcImx6NFwiIFxuICB9LCBcbiAgXCJzZWFyY2hGaWVsZFwiIDogZmFsc2UsIFxuICBcInNwYXJzZVwiIDogdHJ1ZSwgXG4gIFwic3RvcmVkVmFsdWVzXCIgOiBbIF0sIFxuICBcInRyYWNrTGlzdFBvc2l0aW9uc1wiIDogZmFsc2UsIFxuICBcInR5cGVcIiA6IFwiaW52ZXJ0ZWRcIiwgXG4gIFwidW5pcXVlXCIgOiBmYWxzZSwgXG4gIFwidmVyc2lvblwiIDogMSwgXG4gIFwid3JpdGVidWZmZXJBY3RpdmVcIiA6IDAsIFxuICBcIndyaXRlYnVmZmVySWRsZVwiIDogNjQsIFxuICBcIndyaXRlYnVmZmVyU2l6ZU1heFwiIDogMzM1NTQ0MzIsIFxuICBcImNvZGVcIiA6IDIwMSwgXG4gIFwiZXJyb3JcIiA6IGZhbHNlIFxufSIsImVycm9yIjoiIiwib3B0aW9ucyI6eyJkZXNjcmlwdGlvbiI6IkNyZWF0aW5nIGFuIGludmVydGVkIGluZGV4OiIsIm5hbWUiOiJSZXN0SW5kZXhDcmVhdGVOZXdJbnZlcnRlZCIsInR5cGUiOiJzaW5nbGUiLCJyZW5kZXIiOiJpbnB1dC9vdXRwdXQifX0K" }, + "RestIndexCreateNewMdiPrefixed_single": { + "request": "LS0tCmRlc2NyaXB0aW9uOiB8LQogIENyZWF0aW5nIGEgcHJlZml4ZWQgbXVsdGktZGltZW5zaW9uYWwgaW5kZXgKbmFtZTogUmVzdEluZGV4Q3JlYXRlTmV3TWRpUHJlZml4ZWQKLS0tCnZhciBjbiA9ICJpbnRlcnZhbHMiOwpkYi5fZHJvcChjbik7CmRiLl9jcmVhdGUoY24pOwoKICAgIHZhciB1cmwgPSAiL19hcGkvaW5kZXg/Y29sbGVjdGlvbj0iICsgY247CiAgICB2YXIgYm9keSA9IHsKICAgICAgdHlwZTogIm1kaS1wcmVmaXhlZCIsCiAgICAgIGZpZWxkczogWyAiZnJvbSIsICJ0byIgXSwKICAgICAgZmllbGRWYWx1ZVR5cGVzOiAiZG91YmxlIiwKICAgICAgcHJlZml4RmllbGRzOiBbInllYXIiLCAibW9udGgiXQogICAgfTsKCiAgICB2YXIgcmVzcG9uc2UgPSBsb2dDdXJsUmVxdWVzdCgnUE9TVCcsIHVybCwgYm9keSk7CgogICAgYXNzZXJ0KHJlc3BvbnNlLmNvZGUgPT09IDIwMSk7CgogICAgbG9nSnNvblJlc3BvbnNlKHJlc3BvbnNlKTsKZGIuX2Ryb3AoY24pOw==", + "response": "eyJpbnB1dCI6ImN1cmwgLVggUE9TVCAtLWhlYWRlciAnYWNjZXB0OiBhcHBsaWNhdGlvbi9qc29uJyAtLWRhdGEtYmluYXJ5IEAtIC0tZHVtcCAtIGh0dHA6Ly9sb2NhbGhvc3Q6ODUyOS9fYXBpL2luZGV4P2NvbGxlY3Rpb249aW50ZXJ2YWxzXG57XG4gIFwidHlwZVwiOiBcIm1kaS1wcmVmaXhlZFwiLFxuICBcImZpZWxkc1wiOiBbXG4gICAgXCJmcm9tXCIsXG4gICAgXCJ0b1wiXG4gIF0sXG4gIFwiZmllbGRWYWx1ZVR5cGVzXCI6IFwiZG91YmxlXCIsXG4gIFwicHJlZml4RmllbGRzXCI6IFtcbiAgICBcInllYXJcIixcbiAgICBcIm1vbnRoXCJcbiAgXVxufSIsIm91dHB1dCI6IkhUVFAvMS4xIDIwMSBDcmVhdGVkXG5jb250ZW50LXR5cGU6IGFwcGxpY2F0aW9uL2pzb25cbmNhY2hlLWNvbnRyb2w6IG5vLWNhY2hlLCBuby1zdG9yZSwgbXVzdC1yZXZhbGlkYXRlLCBwcmUtY2hlY2s9MCwgcG9zdC1jaGVjaz0wLCBtYXgtYWdlPTAsIHMtbWF4YWdlPTBcbmNvbm5lY3Rpb246IEtlZXAtQWxpdmVcbmNvbnRlbnQtbGVuZ3RoOiAyNzlcbmNvbnRlbnQtc2VjdXJpdHktcG9saWN5OiBmcmFtZS1hbmNlc3RvcnMgJ3NlbGYnOyBmb3JtLWFjdGlvbiAnc2VsZic7XG5leHBpcmVzOiAwXG5wcmFnbWE6IG5vLWNhY2hlXG5zZXJ2ZXI6IEFyYW5nb0RCXG5zdHJpY3QtdHJhbnNwb3J0LXNlY3VyaXR5OiBtYXgtYWdlPTMxNTM2MDAwIDsgaW5jbHVkZVN1YkRvbWFpbnNcbngtYXJhbmdvLXF1ZXVlLXRpbWUtc2Vjb25kczogMC4wMDAwMDBcbngtY29udGVudC10eXBlLW9wdGlvbnM6IG5vc25pZmZcblxueyBcbiAgXCJlc3RpbWF0ZXNcIiA6IHRydWUsIFxuICBcImZpZWxkVmFsdWVUeXBlc1wiIDogXCJkb3VibGVcIiwgXG4gIFwiZmllbGRzXCIgOiBbIFxuICAgIFwiZnJvbVwiLCBcbiAgICBcInRvXCIgXG4gIF0sIFxuICBcImlkXCIgOiBcImludGVydmFscy82OTgwN1wiLCBcbiAgXCJpc05ld2x5Q3JlYXRlZFwiIDogdHJ1ZSwgXG4gIFwibmFtZVwiIDogXCJpZHhfMTc5MTUxMzE1MTU4NzQ4MzY0OFwiLCBcbiAgXCJwcmVmaXhGaWVsZHNcIiA6IFsgXG4gICAgXCJ5ZWFyXCIsIFxuICAgIFwibW9udGhcIiBcbiAgXSwgXG4gIFwic2VsZWN0aXZpdHlFc3RpbWF0ZVwiIDogMSwgXG4gIFwic3BhcnNlXCIgOiBmYWxzZSwgXG4gIFwidHlwZVwiIDogXCJtZGktcHJlZml4ZWRcIiwgXG4gIFwidW5pcXVlXCIgOiBmYWxzZSwgXG4gIFwiY29kZVwiIDogMjAxLCBcbiAgXCJlcnJvclwiIDogZmFsc2UgXG59IiwiZXJyb3IiOiIiLCJvcHRpb25zIjp7ImRlc2NyaXB0aW9uIjoiQ3JlYXRpbmcgYSBwcmVmaXhlZCBtdWx0aS1kaW1lbnNpb25hbCBpbmRleCIsIm5hbWUiOiJSZXN0SW5kZXhDcmVhdGVOZXdNZGlQcmVmaXhlZCIsInR5cGUiOiJzaW5nbGUiLCJyZW5kZXIiOiJpbnB1dC9vdXRwdXQifX0K" + }, + "RestIndexCreateNewMdi_single": { + "request": "LS0tCmRlc2NyaXB0aW9uOiB8LQogIENyZWF0aW5nIGEgbXVsdGktZGltZW5zaW9uYWwgaW5kZXgKbmFtZTogUmVzdEluZGV4Q3JlYXRlTmV3TWRpCi0tLQp2YXIgY24gPSAiaW50ZXJ2YWxzIjsKZGIuX2Ryb3AoY24pOwpkYi5fY3JlYXRlKGNuKTsKCiAgICB2YXIgdXJsID0gIi9fYXBpL2luZGV4P2NvbGxlY3Rpb249IiArIGNuOwogICAgdmFyIGJvZHkgPSB7CiAgICAgIHR5cGU6ICJtZGkiLAogICAgICBmaWVsZHM6IFsgImZyb20iLCAidG8iIF0sCiAgICAgIGZpZWxkVmFsdWVUeXBlczogImRvdWJsZSIKICAgIH07CgogICAgdmFyIHJlc3BvbnNlID0gbG9nQ3VybFJlcXVlc3QoJ1BPU1QnLCB1cmwsIGJvZHkpOwoKICAgIGFzc2VydChyZXNwb25zZS5jb2RlID09PSAyMDEpOwoKICAgIGxvZ0pzb25SZXNwb25zZShyZXNwb25zZSk7CmRiLl9kcm9wKGNuKTs=", + "response": "eyJpbnB1dCI6ImN1cmwgLVggUE9TVCAtLWhlYWRlciAnYWNjZXB0OiBhcHBsaWNhdGlvbi9qc29uJyAtLWRhdGEtYmluYXJ5IEAtIC0tZHVtcCAtIGh0dHA6Ly9sb2NhbGhvc3Q6ODUyOS9fYXBpL2luZGV4P2NvbGxlY3Rpb249aW50ZXJ2YWxzXG57XG4gIFwidHlwZVwiOiBcIm1kaVwiLFxuICBcImZpZWxkc1wiOiBbXG4gICAgXCJmcm9tXCIsXG4gICAgXCJ0b1wiXG4gIF0sXG4gIFwiZmllbGRWYWx1ZVR5cGVzXCI6IFwiZG91YmxlXCJcbn0iLCJvdXRwdXQiOiJIVFRQLzEuMSAyMDEgQ3JlYXRlZFxuY29udGVudC10eXBlOiBhcHBsaWNhdGlvbi9qc29uXG5jYWNoZS1jb250cm9sOiBuby1jYWNoZSwgbm8tc3RvcmUsIG11c3QtcmV2YWxpZGF0ZSwgcHJlLWNoZWNrPTAsIHBvc3QtY2hlY2s9MCwgbWF4LWFnZT0wLCBzLW1heGFnZT0wXG5jb25uZWN0aW9uOiBLZWVwLUFsaXZlXG5jb250ZW50LWxlbmd0aDogMjE1XG5jb250ZW50LXNlY3VyaXR5LXBvbGljeTogZnJhbWUtYW5jZXN0b3JzICdzZWxmJzsgZm9ybS1hY3Rpb24gJ3NlbGYnO1xuZXhwaXJlczogMFxucHJhZ21hOiBuby1jYWNoZVxuc2VydmVyOiBBcmFuZ29EQlxuc3RyaWN0LXRyYW5zcG9ydC1zZWN1cml0eTogbWF4LWFnZT0zMTUzNjAwMCA7IGluY2x1ZGVTdWJEb21haW5zXG54LWFyYW5nby1xdWV1ZS10aW1lLXNlY29uZHM6IDAuMDAwMDAwXG54LWNvbnRlbnQtdHlwZS1vcHRpb25zOiBub3NuaWZmXG5cbnsgXG4gIFwiZXN0aW1hdGVzXCIgOiBmYWxzZSwgXG4gIFwiZmllbGRWYWx1ZVR5cGVzXCIgOiBcImRvdWJsZVwiLCBcbiAgXCJmaWVsZHNcIiA6IFsgXG4gICAgXCJmcm9tXCIsIFxuICAgIFwidG9cIiBcbiAgXSwgXG4gIFwiaWRcIiA6IFwiaW50ZXJ2YWxzLzY5NjgxXCIsIFxuICBcImlzTmV3bHlDcmVhdGVkXCIgOiB0cnVlLCBcbiAgXCJuYW1lXCIgOiBcImlkeF8xNzkxNTEzMTUxNTMyOTU3Njk3XCIsIFxuICBcInNwYXJzZVwiIDogZmFsc2UsIFxuICBcInR5cGVcIiA6IFwibWRpXCIsIFxuICBcInVuaXF1ZVwiIDogZmFsc2UsIFxuICBcImNvZGVcIiA6IDIwMSwgXG4gIFwiZXJyb3JcIiA6IGZhbHNlIFxufSIsImVycm9yIjoiIiwib3B0aW9ucyI6eyJkZXNjcmlwdGlvbiI6IkNyZWF0aW5nIGEgbXVsdGktZGltZW5zaW9uYWwgaW5kZXgiLCJuYW1lIjoiUmVzdEluZGV4Q3JlYXRlTmV3TWRpIiwidHlwZSI6InNpbmdsZSIsInJlbmRlciI6ImlucHV0L291dHB1dCJ9fQo=" + }, "RestIndexCreateNewPersistent_single": { "request": "LS0tCmRlc2NyaXB0aW9uOiB8LQogIENyZWF0aW5nIGEgcGVyc2lzdGVudCBpbmRleApuYW1lOiBSZXN0SW5kZXhDcmVhdGVOZXdQZXJzaXN0ZW50Ci0tLQp2YXIgY24gPSAicHJvZHVjdHMiOwpkYi5fZHJvcChjbik7CmRiLl9jcmVhdGUoY24pOwoKdmFyIHVybCA9ICIvX2FwaS9pbmRleD9jb2xsZWN0aW9uPSIgKyBjbjsKdmFyIGJvZHkgPSB7CiAgdHlwZTogInBlcnNpc3RlbnQiLAogIHVuaXF1ZTogZmFsc2UsCiAgZmllbGRzOiBbICJhIiwgImIiIF0KfTsKCnZhciByZXNwb25zZSA9IGxvZ0N1cmxSZXF1ZXN0KCdQT1NUJywgdXJsLCBib2R5KTsKCmFzc2VydChyZXNwb25zZS5jb2RlID09PSAyMDEpOwoKbG9nSnNvblJlc3BvbnNlKHJlc3BvbnNlKTsKZGIuX2Ryb3AoY24pOw==", "response": "eyJpbnB1dCI6ImN1cmwgLVggUE9TVCAtLWhlYWRlciAnYWNjZXB0OiBhcHBsaWNhdGlvbi9qc29uJyAtLWRhdGEtYmluYXJ5IEAtIC0tZHVtcCAtIGh0dHA6Ly9sb2NhbGhvc3Q6ODUyOS9fYXBpL2luZGV4P2NvbGxlY3Rpb249cHJvZHVjdHNcbntcbiAgXCJ0eXBlXCI6IFwicGVyc2lzdGVudFwiLFxuICBcInVuaXF1ZVwiOiBmYWxzZSxcbiAgXCJmaWVsZHNcIjogW1xuICAgIFwiYVwiLFxuICAgIFwiYlwiXG4gIF1cbn0iLCJvdXRwdXQiOiJIVFRQLzEuMSAyMDEgQ3JlYXRlZFxuY29udGVudC10eXBlOiBhcHBsaWNhdGlvbi9qc29uXG5jYWNoZS1jb250cm9sOiBuby1jYWNoZSwgbm8tc3RvcmUsIG11c3QtcmV2YWxpZGF0ZSwgcHJlLWNoZWNrPTAsIHBvc3QtY2hlY2s9MCwgbWF4LWFnZT0wLCBzLW1heGFnZT0wXG5jb25uZWN0aW9uOiBLZWVwLUFsaXZlXG5jb250ZW50LWxlbmd0aDogMjUzXG5jb250ZW50LXNlY3VyaXR5LXBvbGljeTogZnJhbWUtYW5jZXN0b3JzICdzZWxmJzsgZm9ybS1hY3Rpb24gJ3NlbGYnO1xuZXhwaXJlczogMFxucHJhZ21hOiBuby1jYWNoZVxuc2VydmVyOiBBcmFuZ29EQlxuc3RyaWN0LXRyYW5zcG9ydC1zZWN1cml0eTogbWF4LWFnZT0zMTUzNjAwMCA7IGluY2x1ZGVTdWJEb21haW5zXG54LWFyYW5nby1xdWV1ZS10aW1lLXNlY29uZHM6IDAuMDAwMDAwXG54LWNvbnRlbnQtdHlwZS1vcHRpb25zOiBub3NuaWZmXG5cbnsgXG4gIFwiY2FjaGVFbmFibGVkXCIgOiBmYWxzZSwgXG4gIFwiZGVkdXBsaWNhdGVcIiA6IHRydWUsIFxuICBcImVzdGltYXRlc1wiIDogdHJ1ZSwgXG4gIFwiZmllbGRzXCIgOiBbIFxuICAgIFwiYVwiLCBcbiAgICBcImJcIiBcbiAgXSwgXG4gIFwiaWRcIiA6IFwicHJvZHVjdHMvNjk5OTdcIiwgXG4gIFwiaXNOZXdseUNyZWF0ZWRcIiA6IHRydWUsIFxuICBcIm5hbWVcIiA6IFwiaWR4XzE3Nzc0ODEwMzg1OTE2ODg3MDRcIiwgXG4gIFwic2VsZWN0aXZpdHlFc3RpbWF0ZVwiIDogMSwgXG4gIFwic3BhcnNlXCIgOiBmYWxzZSwgXG4gIFwidHlwZVwiIDogXCJwZXJzaXN0ZW50XCIsIFxuICBcInVuaXF1ZVwiIDogZmFsc2UsIFxuICBcImNvZGVcIiA6IDIwMSwgXG4gIFwiZXJyb3JcIiA6IGZhbHNlIFxufSIsImVycm9yIjoiIiwib3B0aW9ucyI6eyJkZXNjcmlwdGlvbiI6IkNyZWF0aW5nIGEgcGVyc2lzdGVudCBpbmRleCIsIm5hbWUiOiJSZXN0SW5kZXhDcmVhdGVOZXdQZXJzaXN0ZW50IiwidHlwZSI6InNpbmdsZSIsInJlbmRlciI6ImlucHV0L291dHB1dCJ9fQo=" @@ -3383,9 +3391,13 @@ "request": "LS0tCm5hbWU6IGVuc3VyZVVuaXF1ZVBlcnNpc3RlbnRTaW5nbGUKZGVzY3JpcHRpb246ICcnCi0tLQp+ZGIuX2NyZWF0ZSgiaWRzIik7CmRiLmlkcy5lbnN1cmVJbmRleCh7IHR5cGU6ICJwZXJzaXN0ZW50IiwgZmllbGRzOiBbICJteUlkIiBdLCB1bmlxdWU6IHRydWUgfSk7CmRiLmlkcy5zYXZlKHsgIm15SWQiOiAxMjMgfSk7CmRiLmlkcy5zYXZlKHsgIm15SWQiOiA0NTYgfSk7CmRiLmlkcy5zYXZlKHsgIm15SWQiOiA3ODkgfSk7CmRiLmlkcy5zYXZlKHsgIm15SWQiOiAxMjMgfSk7ICAvLyB4cEVycm9yKEVSUk9SX0FSQU5HT19VTklRVUVfQ09OU1RSQUlOVF9WSU9MQVRFRCkKfmRiLl9kcm9wKCJpZHMiKTs=", "response": "eyJpbnB1dCI6ImRiLmlkcy5lbnN1cmVJbmRleCh7IHR5cGU6IFwicGVyc2lzdGVudFwiLCBmaWVsZHM6IFsgXCJteUlkXCIgXSwgdW5pcXVlOiB0cnVlIH0pO1xuZGIuaWRzLnNhdmUoeyBcIm15SWRcIjogMTIzIH0pO1xuZGIuaWRzLnNhdmUoeyBcIm15SWRcIjogNDU2IH0pO1xuZGIuaWRzLnNhdmUoeyBcIm15SWRcIjogNzg5IH0pO1xuZGIuaWRzLnNhdmUoeyBcIm15SWRcIjogMTIzIH0pOyAgIiwib3V0cHV0IjoieyBcbiAgXCJjYWNoZUVuYWJsZWRcIiA6IGZhbHNlLCBcbiAgXCJkZWR1cGxpY2F0ZVwiIDogdHJ1ZSwgXG4gIFwiZXN0aW1hdGVzXCIgOiB0cnVlLCBcbiAgXCJmaWVsZHNcIiA6IFsgXG4gICAgXCJteUlkXCIgXG4gIF0sIFxuICBcImlkXCIgOiBcImlkcy83NzEyMlwiLCBcbiAgXCJpc05ld2x5Q3JlYXRlZFwiIDogdHJ1ZSwgXG4gIFwibmFtZVwiIDogXCJpZHhfMTc3NzQ4MTA2MDg0NTYxNzE1M1wiLCBcbiAgXCJzZWxlY3Rpdml0eUVzdGltYXRlXCIgOiAxLCBcbiAgXCJzcGFyc2VcIiA6IGZhbHNlLCBcbiAgXCJ0eXBlXCIgOiBcInBlcnNpc3RlbnRcIiwgXG4gIFwidW5pcXVlXCIgOiB0cnVlLCBcbiAgXCJjb2RlXCIgOiAyMDEgXG59XG5cbnsgXG4gIFwiX2lkXCIgOiBcImlkcy83NzEyNlwiLCBcbiAgXCJfa2V5XCIgOiBcIjc3MTI2XCIsIFxuICBcIl9yZXZcIiA6IFwiX2dvMlhyNHUtLS1cIiBcbn1cblxueyBcbiAgXCJfaWRcIiA6IFwiaWRzLzc3MTI4XCIsIFxuICBcIl9rZXlcIiA6IFwiNzcxMjhcIiwgXG4gIFwiX3JldlwiIDogXCJfZ28yWHI0dS0tX1wiIFxufVxuXG57IFxuICBcIl9pZFwiIDogXCJpZHMvNzcxMzBcIiwgXG4gIFwiX2tleVwiIDogXCI3NzEzMFwiLCBcbiAgXCJfcmV2XCIgOiBcIl9nbzJYcjR1LS1BXCIgXG59XG5cbltBcmFuZ29FcnJvciAxMjEwOiB1bmlxdWUgY29uc3RyYWludCB2aW9sYXRlZCAtIGluIGluZGV4IGlkeF8xNzc3NDgxMDYwODQ1NjE3MTUzIG9mIHR5cGUgcGVyc2lzdGVudCBvdmVyICdteUlkJzsgY29uZmxpY3Rpbmcga2V5OiA3NzEyNl0iLCJlcnJvciI6IiIsIm9wdGlvbnMiOnsiZGVzY3JpcHRpb24iOiIiLCJuYW1lIjoiZW5zdXJlVW5pcXVlUGVyc2lzdGVudFNpbmdsZSIsInR5cGUiOiJzaW5nbGUiLCJyZW5kZXIiOiJpbnB1dC9vdXRwdXQifX0K" }, + "ensureVertexCentricIndexMultidim_single": { + "request": "LS0tCm5hbWU6IGVuc3VyZVZlcnRleENlbnRyaWNJbmRleE11bHRpZGltCmRlc2NyaXB0aW9uOiAnJwotLS0KfmRiLl9jcmVhdGVFZGdlQ29sbGVjdGlvbigiZWRnZUNvbGxlY3Rpb24iKTsKZGIuZWRnZUNvbGxlY3Rpb24uZW5zdXJlSW5kZXgoewogIHR5cGU6ICJtZGktcHJlZml4ZWQiLAogIHByZWZpeEZpZWxkczogWyJfdG8iLCAidHlwZSJdLAogIGZpZWxkczogWyAieCIsICJ5IiBdLAogIGZpZWxkVmFsdWVUeXBlczogImRvdWJsZSIKfSk7Cn5kYi5fZHJvcCgiZWRnZUNvbGxlY3Rpb24iKTs=", + "response": "eyJpbnB1dCI6ImRiLmVkZ2VDb2xsZWN0aW9uLmVuc3VyZUluZGV4KHtcbiAgdHlwZTogXCJtZGktcHJlZml4ZWRcIixcbiAgcHJlZml4RmllbGRzOiBbXCJfdG9cIiwgXCJ0eXBlXCJdLFxuICBmaWVsZHM6IFsgXCJ4XCIsIFwieVwiIF0sXG4gIGZpZWxkVmFsdWVUeXBlczogXCJkb3VibGVcIlxufSk7Iiwib3V0cHV0IjoieyBcbiAgXCJlc3RpbWF0ZXNcIiA6IHRydWUsIFxuICBcImZpZWxkVmFsdWVUeXBlc1wiIDogXCJkb3VibGVcIiwgXG4gIFwiZmllbGRzXCIgOiBbIFxuICAgIFwieFwiLCBcbiAgICBcInlcIiBcbiAgXSwgXG4gIFwiaWRcIiA6IFwiZWRnZUNvbGxlY3Rpb24vODE5NTBcIiwgXG4gIFwiaXNOZXdseUNyZWF0ZWRcIiA6IHRydWUsIFxuICBcIm5hbWVcIiA6IFwiaWR4XzE3OTE1MTMxNzE4MDkyNzE4MDlcIiwgXG4gIFwicHJlZml4RmllbGRzXCIgOiBbIFxuICAgIFwiX3RvXCIsIFxuICAgIFwidHlwZVwiIFxuICBdLCBcbiAgXCJzZWxlY3Rpdml0eUVzdGltYXRlXCIgOiAxLCBcbiAgXCJzcGFyc2VcIiA6IGZhbHNlLCBcbiAgXCJ0eXBlXCIgOiBcIm1kaS1wcmVmaXhlZFwiLCBcbiAgXCJ1bmlxdWVcIiA6IGZhbHNlLCBcbiAgXCJjb2RlXCIgOiAyMDEgXG59IiwiZXJyb3IiOiIiLCJvcHRpb25zIjp7ImRlc2NyaXB0aW9uIjoiIiwibmFtZSI6ImVuc3VyZVZlcnRleENlbnRyaWNJbmRleE11bHRpZGltIiwidHlwZSI6InNpbmdsZSIsInJlbmRlciI6ImlucHV0L291dHB1dCJ9fQo=" + }, "ensureVertexCentricIndex_single": { - "request": "LS0tCm5hbWU6IGVuc3VyZVZlcnRleENlbnRyaWNJbmRleApkZXNjcmlwdGlvbjogJycKLS0tCn5kYi5fY3JlYXRlRWRnZUNvbGxlY3Rpb24oImNvbGxlY3Rpb24iKTsKZGIuY29sbGVjdGlvbi5lbnN1cmVJbmRleCh7IHR5cGU6ICJwZXJzaXN0ZW50IiwgZmllbGRzOiBbICJfZnJvbSIsICJ0eXBlIiBdIH0pCn5kYi5fZHJvcCgiY29sbGVjdGlvbiIpOw==", - "response": "eyJpbnB1dCI6ImRiLmNvbGxlY3Rpb24uZW5zdXJlSW5kZXgoeyB0eXBlOiBcInBlcnNpc3RlbnRcIiwgZmllbGRzOiBbIFwiX2Zyb21cIiwgXCJ0eXBlXCIgXSB9KSIsIm91dHB1dCI6InsgXG4gIFwiY2FjaGVFbmFibGVkXCIgOiBmYWxzZSwgXG4gIFwiZGVkdXBsaWNhdGVcIiA6IHRydWUsIFxuICBcImVzdGltYXRlc1wiIDogdHJ1ZSwgXG4gIFwiZmllbGRzXCIgOiBbIFxuICAgIFwiX2Zyb21cIiwgXG4gICAgXCJ0eXBlXCIgXG4gIF0sIFxuICBcImlkXCIgOiBcImNvbGxlY3Rpb24vODA3NzFcIiwgXG4gIFwiaXNOZXdseUNyZWF0ZWRcIiA6IHRydWUsIFxuICBcIm5hbWVcIiA6IFwiaWR4XzE3Nzc0ODEwNjIxNzMxMTQzNjhcIiwgXG4gIFwic2VsZWN0aXZpdHlFc3RpbWF0ZVwiIDogMSwgXG4gIFwic3BhcnNlXCIgOiBmYWxzZSwgXG4gIFwidHlwZVwiIDogXCJwZXJzaXN0ZW50XCIsIFxuICBcInVuaXF1ZVwiIDogZmFsc2UsIFxuICBcImNvZGVcIiA6IDIwMSBcbn0iLCJlcnJvciI6IiIsIm9wdGlvbnMiOnsiZGVzY3JpcHRpb24iOiIiLCJuYW1lIjoiZW5zdXJlVmVydGV4Q2VudHJpY0luZGV4IiwidHlwZSI6InNpbmdsZSIsInJlbmRlciI6ImlucHV0L291dHB1dCJ9fQo=" + "request": "LS0tCm5hbWU6IGVuc3VyZVZlcnRleENlbnRyaWNJbmRleApkZXNjcmlwdGlvbjogJycKLS0tCn5kYi5fY3JlYXRlRWRnZUNvbGxlY3Rpb24oImVkZ2VDb2xsZWN0aW9uIik7CmRiLmVkZ2VDb2xsZWN0aW9uLmVuc3VyZUluZGV4KHsgdHlwZTogInBlcnNpc3RlbnQiLCBmaWVsZHM6IFsgIl9mcm9tIiwgInR5cGUiIF0gfSk7Cn5kYi5fZHJvcCgiZWRnZUNvbGxlY3Rpb24iKTs=", + "response": "eyJpbnB1dCI6ImRiLmVkZ2VDb2xsZWN0aW9uLmVuc3VyZUluZGV4KHsgdHlwZTogXCJwZXJzaXN0ZW50XCIsIGZpZWxkczogWyBcIl9mcm9tXCIsIFwidHlwZVwiIF0gfSk7Iiwib3V0cHV0IjoieyBcbiAgXCJjYWNoZUVuYWJsZWRcIiA6IGZhbHNlLCBcbiAgXCJkZWR1cGxpY2F0ZVwiIDogdHJ1ZSwgXG4gIFwiZXN0aW1hdGVzXCIgOiB0cnVlLCBcbiAgXCJmaWVsZHNcIiA6IFsgXG4gICAgXCJfZnJvbVwiLCBcbiAgICBcInR5cGVcIiBcbiAgXSwgXG4gIFwiaWRcIiA6IFwiZWRnZUNvbGxlY3Rpb24vODE2NTZcIiwgXG4gIFwiaXNOZXdseUNyZWF0ZWRcIiA6IHRydWUsIFxuICBcIm5hbWVcIiA6IFwiaWR4XzE3OTE1MTMxNzE1MDQxMzYxOTNcIiwgXG4gIFwic2VsZWN0aXZpdHlFc3RpbWF0ZVwiIDogMSwgXG4gIFwic3BhcnNlXCIgOiBmYWxzZSwgXG4gIFwidHlwZVwiIDogXCJwZXJzaXN0ZW50XCIsIFxuICBcInVuaXF1ZVwiIDogZmFsc2UsIFxuICBcImNvZGVcIiA6IDIwMSBcbn0iLCJlcnJvciI6IiIsIm9wdGlvbnMiOnsiZGVzY3JpcHRpb24iOiIiLCJuYW1lIjoiZW5zdXJlVmVydGV4Q2VudHJpY0luZGV4IiwidHlwZSI6InNpbmdsZSIsInJlbmRlciI6ImlucHV0L291dHB1dCJ9fQo=" }, "enterpriseGraphCreate1_cluster": { "request": "LS0tCm5hbWU6IGVudGVycHJpc2VHcmFwaENyZWF0ZTEKZGVzY3JpcHRpb246ICcnCnR5cGU6IGNsdXN0ZXIKLS0tCnZhciBncmFwaF9tb2R1bGUgPSByZXF1aXJlKCJAYXJhbmdvZGIvZW50ZXJwcmlzZS1ncmFwaCIpOwp2YXIgZ3JhcGggPSBncmFwaF9tb2R1bGUuX2NyZWF0ZSgibXlHcmFwaCIsIFtdLCBbXSwge2lzU21hcnQ6IHRydWUsIG51bWJlck9mU2hhcmRzOiA5fSk7CmdyYXBoOwp+Z3JhcGhfbW9kdWxlLl9kcm9wKCJteUdyYXBoIiwgdHJ1ZSk7", From 7efdbad73446301adee6de809226969ad85c2c1b Mon Sep 17 00:00:00 2001 From: Simran Spiller Date: Thu, 22 Feb 2024 13:37:24 +0100 Subject: [PATCH 8/8] Review feedback --- .../multi-dimensional-indexes.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md b/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md index 74bcb98243..e15be61a1f 100644 --- a/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md +++ b/site/content/3.12/index-and-search/indexing/working-with-indexes/multi-dimensional-indexes.md @@ -19,7 +19,7 @@ You can choose between two subtypes of multi-dimensional indexes: attributes to use as dimensions - An `mdi-prefixed` index with a `fields` property as well as a `prefixFields` property to specify one or more mandatory document attributes to narrow down - the search space using equality checks + the search space using equality checks, see [Prefix fields](#prefix-fields) Both subtypes require that the attributes described by `fields` have numeric values. You can optionally omit documents from the index that have any of @@ -31,10 +31,10 @@ Multi-dimensional indexes can be created with a uniqueness constraint with attribute values, using all of the `fields` attributes and (for `mdi-prefixed` indexes) `prefixFields`. Documents omitted because of `sparse: true` are exempt. -You can store additional attributes in multi-dimensional indexes with the -`storedValues` property. They can be used for projections (unlike the `fields` -attributes) so that indexes can cover more queries without having to access the -full documents. +You can [store additional attributes](#storing-additional-values-in-indexes) in +multi-dimensional indexes with the `storedValues` property. They can be used for +projections (unlike the `fields` attributes) so that indexes can cover more +queries without having to access the full documents. Non-unique `mdi` indexes have a fixed selectivity estimate of `1`. For `mdi` indexes with `unique: true` as well as for `mdi-prefixed` indexes, you can @@ -171,6 +171,10 @@ you want to use for equality checks. This combination allows to efficiently narrow down the search space to a subset of multi-dimensional index data before performing the range checking. +The attributes for equality checking are specified via the `prefixFields` +property of `mdi-prefix` indexes. These attributes can have non-numeric values, +unlike the attributes you use as `fields`. + ```js db..ensureIndex({ type: "mdi-prefixed",