Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions modules/guides/pages/create-index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,7 @@ The following query creates a secondary index on an expression using the `name`

[source,sqlpp]
----
include::learn:example$services-and-indexes/indexes/create-idx-expr.n1ql[]
include::indexes:example$create-idx-expr.n1ql[]
----

For further details and examples, refer to xref:n1ql:n1ql-language-reference/createindex.adoc[CREATE INDEX].
Expand Down Expand Up @@ -384,7 +384,7 @@ include::partial$query-context.adoc[tag=step]
.Query
[source,sqlpp]
----
include::learn:example$services-and-indexes/indexes/create-idx-composite.n1ql[]
include::indexes:example$create-idx-composite.n1ql[]
----

For further details and examples, refer to xref:n1ql:n1ql-language-reference/createindex.adoc[CREATE INDEX].
Expand Down Expand Up @@ -507,7 +507,7 @@ include::partial$query-context.adoc[tag=step]
.Query
[source,sqlpp]
----
include::learn:example$services-and-indexes/indexes/create-idx-array.n1ql[]
include::indexes:example$create-idx-array.n1ql[]
----
====

Expand All @@ -528,11 +528,11 @@ include::partial$query-context.adoc[tag=step]
.Query
[source,sqlpp]
----
include::learn:example$services-and-indexes/indexes/create-idx-partial.n1ql[]
include::indexes:example$create-idx-partial.n1ql[]
----
====

For further details and examples, refer to xref:n1ql:n1ql-language-reference/indexing-and-query-perf.adoc#partial-index[Partial Index].
For further details and examples, refer to xref:indexes:indexing-and-query-perf.adoc#partial-index[Partial Index].

=== Creating a Covering Index

Expand All @@ -542,13 +542,13 @@ If a query uses a covering index, the query can get all the data it needs from t

To create a covering index, make sure the index includes all the fields and expressions required by the query.

For further details and examples, refer to xref:n1ql:n1ql-language-reference/covering-indexes.adoc[Covering Indexes].
For further details and examples, refer to xref:indexes:covering-indexes.adoc[Covering Indexes].

== Related Links

Reference and explanation:

* xref:learn:services-and-indexes/indexes/global-secondary-indexes.adoc[Using Indexes]
* xref:indexes:indexing-overview.adoc[]

Administrator guides:

Expand Down
2 changes: 1 addition & 1 deletion modules/guides/pages/defer-index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -302,7 +302,7 @@ For further details, refer to https://docs.couchbase.com/sdk-api/couchbase-pytho

Reference and explanation:

* xref:learn:services-and-indexes/indexes/global-secondary-indexes.adoc[Using Indexes]
* xref:indexes:indexing-overview.adoc[]

Administrator guides:

Expand Down
2 changes: 1 addition & 1 deletion modules/guides/pages/drop-index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -335,7 +335,7 @@ For further details, refer to https://docs.couchbase.com/sdk-api/couchbase-pytho

Reference and explanation:

* xref:learn:services-and-indexes/indexes/global-secondary-indexes.adoc[Using Indexes]
* xref:indexes:indexing-overview.adoc[]

Administrator guides:

Expand Down
2 changes: 1 addition & 1 deletion modules/guides/pages/index-advisor.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@ For more details, refer to xref:n1ql:n1ql-language-reference/advisor.adoc#adviso

Reference and explanation:

* xref:learn:services-and-indexes/indexes/global-secondary-indexes.adoc[Using Indexes]
* xref:indexes:indexing-overview.adoc[]

Administrator guides:

Expand Down
8 changes: 4 additions & 4 deletions modules/guides/pages/place-index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ include::partial$query-context.adoc[tag=step]
.Query
[source,sqlpp]
----
include::learn:example$services-and-indexes/indexes/replication-num.n1ql[]
include::indexes:example$replication-num.n1ql[]
----
====

Expand All @@ -126,13 +126,13 @@ include::partial$query-context.adoc[tag=step]
.Query
[source,sqlpp]
----
include::learn:example$services-and-indexes/indexes/replication-nodes.n1ql[]
include::indexes:example$replication-nodes.n1ql[]
----
====

NOTE: If you specify both `num_replica` _and_ `nodes`, the number of nodes must be _one greater_ than the number of replicas.

For further details and examples, refer to xref:learn:services-and-indexes/indexes/index-replication.adoc#index-replication[Index Replication].
For further details and examples, refer to xref:indexes:index-replication.adoc#index-replication[Index Replication].

== Altering Index Placement

Expand Down Expand Up @@ -166,7 +166,7 @@ For further details and examples, refer to xref:n1ql:n1ql-language-reference/alt

Reference and explanation:

* xref:learn:services-and-indexes/indexes/global-secondary-indexes.adoc[Using Indexes]
* xref:indexes:indexing-overview.adoc[]

Administrator guides:

Expand Down
2 changes: 1 addition & 1 deletion modules/guides/pages/select-index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ For further details and examples, refer to xref:n1ql:n1ql-language-reference/key

Reference and explanation:

* xref:learn:services-and-indexes/indexes/global-secondary-indexes.adoc[Using Indexes]
* xref:indexes:indexing-overview.adoc[]
* xref:n1ql:n1ql-language-reference/selectintro.adoc[SELECT]
* xref:n1ql:n1ql-language-reference/optimizer-hints.adoc[Hints]

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
= Covering Indexes
:description: When an index includes the actual values of all the fields specified in the query, the index covers the query and does not require an additional step to fetch the actual values from the data service.
:page-topic-type: concept
:page-aliases: indexes:covering-indexes.adoc
:imagesdir: ../../assets/images
:page-aliases: n1ql:n1ql-language-reference/covering-indexes
:imagesdir: ../images

[abstract]
{description}
Expand All @@ -24,15 +24,15 @@ The following diagram illustrates the query execution work flow without covering
.Query execution workflow including fetch request from Data service
[plantuml,n1ql-query-workflow,svg]
....
include::learn:partial$diagrams/query-execution.puml[tags=!example]
include::indexes:partial$diagrams/query-execution.puml[tags=!example]
....

The following diagram illustrates the query execution work flow with covering indexes:

.Query execution workflow with no fetch request from Data service
[plantuml,n1ql-query-workflow-cover-idx,svg]
....
include::learn:partial$diagrams/query-execution.puml[tags=!example;!fetch]
include::indexes:partial$diagrams/query-execution.puml[tags=!example;!fetch]
....

As you can see in the second diagram, a well designed query that uses a covering index avoids the additional steps to fetch the data from the data service.
Expand Down Expand Up @@ -467,7 +467,7 @@ The query plan for the above query shows that the index covers the query.

== Related Links

* xref:learn:services-and-indexes/indexes/index-scans.adoc#query-execution-details[Query Execution: Details]
* xref:learn:services-and-indexes/indexes/index_pushdowns.adoc[Index Pushdown Optimizations]
* xref:n1ql:n1ql-language-reference/groupby-aggregate-performance.adoc[Grouping and Aggregate Pushdown]
* xref:learn:services-and-indexes/indexes/early-filters-and-pagination.adoc[Early Filters, Ordering and Pagination]
* xref:index-scans.adoc#query-execution-details[Query Execution: Details]
* xref:index_pushdowns.adoc[]
* xref:groupby-aggregate-performance.adoc[]
* xref:early-filters-and-pagination.adoc[]
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= Early Filters, Order, and Pagination
:page-topic-type: concept
:page-aliases: learn:services-and-indexes/indexes/early-filters-and-pagination
:description: When covering indexes and index pushdowns are not available, the Query Service may use early filtering, early ordering, and early pagination to improve the query response time.

[abstract]
Expand All @@ -15,7 +16,7 @@ include::ROOT:partial$query-context.adoc[tag=statement]

== Overview

If a query has a xref:n1ql:n1ql-language-reference/covering-indexes.adoc[covering index], and the query is on a single keyspace, then the index contains all the data needed to perform the query.
If a query has a xref:covering-indexes.adoc[covering index], and the query is on a single keyspace, then the index contains all the data needed to perform the query.
The Query service does not need to fetch data from the Data service.
Any filters, order, or pagination (offset and limit) are performed by the Index service.

Expand Down Expand Up @@ -103,7 +104,7 @@ Any feature that can reduce the amount of data to be fetched will improve the qu
It would obviously be inefficient to fetch an entire dataset, and then discard a large portion of it by applying filters and pagination.
Couchbase Server therefore attempts to apply filters, order, and pagination _before_ fetching data, wherever possible.

Index xref:learn:services-and-indexes/indexes/index_pushdowns.adoc[pushdowns] provide one way to reduce the amount of data to be fetched.
Index xref:index_pushdowns.adoc[pushdowns] provide one way to reduce the amount of data to be fetched.
Some filters, ordering, and pagination may be pushed down to the Index service, along with grouping and aggregates.
The Index service applies these operations to the result of the index scan.
The Query service then only needs to fetch the remaining data.
Expand Down Expand Up @@ -213,7 +214,7 @@ For an illustration of early filtering, consider the following index.
.Index I
[source,sqlpp]
----
include::example$services-and-indexes/indexes/early-filter-idx.n1ql[]
include::example$early-filter-idx.n1ql[]
----
====

Expand All @@ -225,15 +226,15 @@ This index contains all the data needed to perform <<early-filter-1>>; it theref
.Query
[source,sqlpp]
----
include::example$services-and-indexes/indexes/early-filter-1.n1ql[]
include::example$early-filter-1.n1ql[]
----

The query plan shows that the query is covered.
In addition, the predicate `city = "Paris"` is applied as a span by the Index service.

[source,json]
----
include::example$services-and-indexes/indexes/early-filter-1.jsonc[tags=extract;ellipsis]
include::example$early-filter-1.jsonc[tags=extract;ellipsis]
----

<.> The `covers` section shows that the query is covered.
Expand All @@ -249,15 +250,15 @@ Now the query is no longer covered, and the Query service needs to fetch documen
.Query
[source,sqlpp]
----
include::example$services-and-indexes/indexes/early-filter-2.n1ql[]
include::example$early-filter-2.n1ql[]
----

This query does not run too slowly, since the predicate `city = "Paris"` is pushed down to the Index service.
The index only returns document keys for documents matching the predicate.

[source,json]
----
include::example$services-and-indexes/indexes/early-filter-2.jsonc[tags=extract;ellipsis]
include::example$early-filter-2.jsonc[tags=extract;ellipsis]
----

<.> The `range` section shows that this predicate is pushed down to the Index service.
Expand All @@ -272,7 +273,7 @@ Now let's consider the case when the filter is not on the index key exactly, but
.Query
[source,sqlpp]
----
include::example$services-and-indexes/indexes/early-filter-3.n1ql[]
include::example$early-filter-3.n1ql[]
----

In this case, the filter cannot be pushed down to the Index service.
Expand All @@ -281,7 +282,7 @@ The Query service can apply the specified filter to the results of the index sca

[source,json]
----
include::example$services-and-indexes/indexes/early-filter-3.jsonc[tags=extract;ellipsis]
include::example$early-filter-3.jsonc[tags=extract;ellipsis]
----

<.> The early index filters, including all the filters that can be evaluated on the index keys.
Expand All @@ -299,15 +300,15 @@ If necessary, the Query service may use a mixture of index pushdowns and early f
.Query
[source,sqlpp]
----
include::example$services-and-indexes/indexes/early-filter-4.n1ql[]
include::example$early-filter-4.n1ql[]
----

The Index service evaluates the filters used to generate the index spans first, before returning to the Query service.
As a result, the early filters only contain the predicates that were not pushed down to the Index service.

[source,json]
----
include::example$services-and-indexes/indexes/early-filter-4.jsonc[tags=extract;ellipsis]
include::example$early-filter-4.jsonc[tags=extract;ellipsis]
----

<.> The filter `name LIKE "%Paris%"` does not generate an exact index span, and needs to be evaluated as an early index filter.
Expand All @@ -324,7 +325,7 @@ As an illustration, consider the following index.
.Index II
[source,sqlpp]
----
include::example$services-and-indexes/indexes/early-paginate-idx.n1ql[]
include::example$early-paginate-idx.n1ql[]
----
====

Expand All @@ -336,15 +337,15 @@ This index acts as a covering index to <<early-paginate-1>>.
.Query
[source,sqlpp]
----
include::example$services-and-indexes/indexes/early-paginate-1.n1ql[]
include::example$early-paginate-1.n1ql[]
----

The `covers` section of the query plan shows that the query is covered.
In addition, the ordering, offset, and limit are performed by the Index service.

[source,json]
----
include::example$services-and-indexes/indexes/early-paginate-1.jsonc[tags=extract;ellipsis]
include::example$early-paginate-1.jsonc[tags=extract;ellipsis]
----

<.> Index order performed by index operator
Expand All @@ -361,14 +362,14 @@ Now the query is no longer covered.
.Query
[source,sqlpp]
----
include::example$services-and-indexes/indexes/early-paginate-2.n1ql[]
include::example$early-paginate-2.n1ql[]
----

However the query runs pretty quickly, since the ordering, offset, and limit can still be pushed down to the Index service.

[source,json]
----
include::example$services-and-indexes/indexes/early-paginate-2.jsonc[tags=extract;ellipsis]
include::example$early-paginate-2.jsonc[tags=extract;ellipsis]
----

<.> Index order pushdown
Expand All @@ -384,14 +385,14 @@ However, if we change the order of the ordering terms so that they no longer mat
.Query
[source,sqlpp]
----
include::example$services-and-indexes/indexes/early-paginate-3.n1ql[]
include::example$early-paginate-3.n1ql[]
----

So in this case, early ordering, limit, and offset are done by the Query service before fetch.

[source,json]
----
include::example$services-and-indexes/indexes/early-paginate-3.jsonc[tags=extract;ellipsis]
include::example$early-paginate-3.jsonc[tags=extract;ellipsis]
----

<.> Early ordering
Expand All @@ -407,14 +408,14 @@ Similarly, if we change the direction of the sort from descending to ascending,
.Query
[source,sqlpp]
----
include::example$services-and-indexes/indexes/early-paginate-4.n1ql[]
include::example$early-paginate-4.n1ql[]
----

In this case also, early ordering, limit, and offset are done by the Query service before fetch.

[source,json]
----
include::example$services-and-indexes/indexes/early-paginate-4.jsonc[tags=extract;ellipsis]
include::example$early-paginate-4.jsonc[tags=extract;ellipsis]
----

<.> The `sort_terms` section uses `_index_key` expressions to sort the results using the index keys.
Expand Down Expand Up @@ -460,10 +461,10 @@ The following table summarizes the different methods of performing filtering, or
^<<note>>^

| Fetch required?
| icon:times-circle[red] No
| icon:check-circle[green] Yes
| icon:check-circle[green] Yes
| icon:check-circle[green] Yes
| icon:times[fw] No
| icon:check[fw] Yes
| icon:check[fw] Yes
| icon:check[fw] Yes

| Filters, order, and pagination +
performed:
Expand All @@ -486,10 +487,10 @@ NOTE: An uncovered query may use a mixture of these methods.

== Related Links

* xref:learn:services-and-indexes/indexes/index-scans.adoc#query-execution-details[Query Execution: Details]
* xref:n1ql:n1ql-language-reference/covering-indexes.adoc[Covering Indexes]
* xref:learn:services-and-indexes/indexes/index_pushdowns.adoc[Index Pushdown Optimizations]
* xref:n1ql:n1ql-language-reference/groupby-aggregate-performance.adoc[Grouping and Aggregate Pushdown]
* xref:n1ql:n1ql-language-reference/where.adoc[WHERE Clause] -- to apply a filter
* xref:n1ql:n1ql-language-reference/orderby.adoc[ORDER BY Clause] -- to apply a sort
* xref:n1ql:n1ql-language-reference/offset.adoc[OFFSET Clause] and xref:n1ql:n1ql-language-reference/limit.adoc[LIMIT Clause] -- to apply pagination
* xref:index-scans.adoc#query-execution-details[Query Execution: Details]
* xref:covering-indexes.adoc[]
* xref:index_pushdowns.adoc[]
* xref:groupby-aggregate-performance.adoc[]
* xref:n1ql:n1ql-language-reference/where.adoc[] -- to apply a filter
* xref:n1ql:n1ql-language-reference/orderby.adoc[] -- to apply a sort
* xref:n1ql:n1ql-language-reference/offset.adoc[] and xref:n1ql:n1ql-language-reference/limit.adoc[] -- to apply pagination
Loading