From 023fe2a973cb23239ed1dd4ed4683ef0b0d63888 Mon Sep 17 00:00:00 2001 From: Simon Dew <39966290+simon-dew@users.noreply.github.com> Date: Fri, 10 May 2024 13:01:20 +0100 Subject: [PATCH 1/4] DOC-11231: Migrate Monitoring Queries page to Devex (#218) * Add monitoring-n1ql-query from docs-server * Enable escape hatch * Update page aliases * Update links to monitoring-n1ql-query * Rearrange management landing page to match nav --- modules/guides/pages/cbo.adoc | 2 +- modules/guides/pages/index-advisor.adoc | 2 +- modules/guides/pages/prep-statements.adoc | 2 +- .../indexes/index_pushdowns.adoc | 2 +- .../indexes/query-without-index.adoc | 2 +- modules/n1ql/pages/n1ql-intro/sysinfo.adoc | 2 +- .../n1ql-language-reference/advisor.adoc | 2 +- .../createfunction.adoc | 2 +- .../n1ql-language-reference/dropfunction.adoc | 2 +- .../n1ql-language-reference/execfunction.adoc | 2 +- .../explainfunction.adoc | 2 +- .../n1ql-language-reference/transactions.adoc | 2 +- .../updatestatistics.adoc | 4 +- .../n1ql-language-reference/userfun.adoc | 2 +- modules/n1ql/pages/n1ql-manage/index.adoc | 12 +- .../n1ql-manage/monitoring-n1ql-query.adoc | 2743 +++++++++++++++++ modules/n1ql/partials/nav.adoc | 1 + 17 files changed, 2765 insertions(+), 21 deletions(-) create mode 100644 modules/n1ql/pages/n1ql-manage/monitoring-n1ql-query.adoc diff --git a/modules/guides/pages/cbo.adoc b/modules/guides/pages/cbo.adoc index 9c30402f5..9720fc75e 100644 --- a/modules/guides/pages/cbo.adoc +++ b/modules/guides/pages/cbo.adoc @@ -225,7 +225,7 @@ Reference: Administrator guides: * xref:clusters:query-service/query-workbench.adoc#query-settings[Modify Query Settings] -* xref:server:manage:monitor/monitoring-n1ql-query.adoc#sys-dictionary[Monitoring Optimizer Statistics] +* xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-dictionary[Monitor Statistics] Querying with SDKs: diff --git a/modules/guides/pages/index-advisor.adoc b/modules/guides/pages/index-advisor.adoc index 20097bfc0..12cc5fcd3 100644 --- a/modules/guides/pages/index-advisor.adoc +++ b/modules/guides/pages/index-advisor.adoc @@ -118,7 +118,7 @@ The `ADVISOR()` function also enables you to get index recommendations for multi To get advice for multiple queries, use the `ADVISOR()` function with an array argument containing strings which represent each query. -TIP: You can query the xref:server:manage:monitor/monitoring-n1ql-query.adoc#sys-completed-req[system:completed_requests] catalog to get a list of recently-completed queries. +TIP: You can query the xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-completed-req[system:completed_requests] catalog to get a list of recently-completed queries. ==== The following example gets index advice for recently-completed queries. diff --git a/modules/guides/pages/prep-statements.adoc b/modules/guides/pages/prep-statements.adoc index 69095aeb2..ab0de88cc 100644 --- a/modules/guides/pages/prep-statements.adoc +++ b/modules/guides/pages/prep-statements.adoc @@ -554,7 +554,7 @@ Overview: Reference: -* xref:server:manage:monitor/monitoring-n1ql-query.adoc#sys-prepared[Monitoring Prepared Statements] +* xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-prepared[Manage and Monitor Prepared Statements] ifdef::flag-devex-rest-api[] * xref:n1ql:n1ql-rest-api/admin.adoc#_prepared_statements_resource[{sqlpp} Admin REST API for Prepared Statements] endif::flag-devex-rest-api[] diff --git a/modules/learn/pages/services-and-indexes/indexes/index_pushdowns.adoc b/modules/learn/pages/services-and-indexes/indexes/index_pushdowns.adoc index a89924842..1e13ac226 100644 --- a/modules/learn/pages/services-and-indexes/indexes/index_pushdowns.adoc +++ b/modules/learn/pages/services-and-indexes/indexes/index_pushdowns.adoc @@ -182,7 +182,7 @@ include::example$services-and-indexes/indexes/pushdown-comp-explain.jsonc[tag=ex <.> The second element of `range[]` corresponds to the index-key `sourceairport` with (`low`, `high`) values (`"LAX"`, `"LAX"`) representing equals condition. The Indexer processes the lookup request and exactly returns only the documents matching the predicate conditions. -For example, when you xref:server:manage:monitor/monitoring-n1ql-query.adoc[enable monitoring] with the configuration parameter `profile = "timings"` for this query, you can see that the indexer returns 165 documents, which is the same as the final result set of the query. +For example, when you xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc[enable monitoring] with the configuration parameter `profile = "timings"` for this query, you can see that the indexer returns 165 documents, which is the same as the final result set of the query. [source,json] ---- diff --git a/modules/learn/pages/services-and-indexes/indexes/query-without-index.adoc b/modules/learn/pages/services-and-indexes/indexes/query-without-index.adoc index 5ef2e2a45..eb67baae5 100644 --- a/modules/learn/pages/services-and-indexes/indexes/query-without-index.adoc +++ b/modules/learn/pages/services-and-indexes/indexes/query-without-index.adoc @@ -203,7 +203,7 @@ In most cases, this indicates that you should create an index to support the req Statistics on sequential scan usage are also available in request profiling information. -For more details, see xref:server:manage:monitor/monitoring-n1ql-query.adoc[]. +For more details, see xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc[]. === Examples diff --git a/modules/n1ql/pages/n1ql-intro/sysinfo.adoc b/modules/n1ql/pages/n1ql-intro/sysinfo.adoc index 982e5fabd..c9d0a3b10 100644 --- a/modules/n1ql/pages/n1ql-intro/sysinfo.adoc +++ b/modules/n1ql/pages/n1ql-intro/sysinfo.adoc @@ -236,4 +236,4 @@ The query returns the result of the expression, 7 in this case. == Related Links -* Refer to xref:server:manage:monitor/monitoring-n1ql-query.adoc[Monitor Queries] for more information on the system namespace. \ No newline at end of file +* Refer to xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc[] for more information on the system namespace. \ No newline at end of file diff --git a/modules/n1ql/pages/n1ql-language-reference/advisor.adoc b/modules/n1ql/pages/n1ql-language-reference/advisor.adoc index 7b23d281a..220110876 100644 --- a/modules/n1ql/pages/n1ql-language-reference/advisor.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/advisor.adoc @@ -12,7 +12,7 @@ :advise: {n1ql}/advise.adoc :rules: {n1ql}/advise.adoc#recommendation-rules -:monitor: xref:server:manage:monitor/monitoring-n1ql-query.adoc +:monitor: xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc :sys-completed-req: {monitor}#sys-completed-req :sys-tasks-cache: {monitor}#sys-tasks-cache diff --git a/modules/n1ql/pages/n1ql-language-reference/createfunction.adoc b/modules/n1ql/pages/n1ql-language-reference/createfunction.adoc index d69ea5c54..cb66d5e5c 100644 --- a/modules/n1ql/pages/n1ql-language-reference/createfunction.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/createfunction.adoc @@ -673,5 +673,5 @@ endif::flag-devex-javascript-udfs[] * To execute a user-defined function, see xref:n1ql-language-reference/execfunction.adoc[]. * To see the execution plan for a user-defined function, see xref:n1ql-language-reference/explainfunction.adoc[]. * To include a user-defined function in an expression, see xref:n1ql-language-reference/userfun.adoc[]. -* To monitor user-defined functions, see xref:server:manage:monitor/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. +* To monitor user-defined functions, see xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. * To drop a user-defined function, see xref:n1ql-language-reference/dropfunction.adoc[]. diff --git a/modules/n1ql/pages/n1ql-language-reference/dropfunction.adoc b/modules/n1ql/pages/n1ql-language-reference/dropfunction.adoc index c166f645e..0d694570f 100644 --- a/modules/n1ql/pages/n1ql-language-reference/dropfunction.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/dropfunction.adoc @@ -160,4 +160,4 @@ endif::flag-devex-javascript-udfs[] * To execute a user-defined function, see xref:n1ql-language-reference/execfunction.adoc[]. * To see the execution plan for a user-defined function, see xref:n1ql-language-reference/explainfunction.adoc[]. * To include a user-defined function in an expression, see xref:n1ql-language-reference/userfun.adoc[]. -* To monitor user-defined functions, see xref:server:manage:monitor/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. +* To monitor user-defined functions, see xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. diff --git a/modules/n1ql/pages/n1ql-language-reference/execfunction.adoc b/modules/n1ql/pages/n1ql-language-reference/execfunction.adoc index 5ade20528..5d570a638 100644 --- a/modules/n1ql/pages/n1ql-language-reference/execfunction.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/execfunction.adoc @@ -95,5 +95,5 @@ For examples, refer to xref:n1ql-language-reference/createfunction.adoc#examples * To manage external libraries and external functions, see xref:server:n1ql:n1ql-rest-api/functions.adoc[]. * To see the execution plan for a user-defined function, see xref:n1ql-language-reference/explainfunction.adoc[]. * To include a user-defined function in an expression, see xref:n1ql-language-reference/userfun.adoc[]. -* To monitor user-defined functions, see xref:server:manage:monitor/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. +* To monitor user-defined functions, see xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. * To drop a user-defined function, see xref:n1ql-language-reference/dropfunction.adoc[]. diff --git a/modules/n1ql/pages/n1ql-language-reference/explainfunction.adoc b/modules/n1ql/pages/n1ql-language-reference/explainfunction.adoc index 3921b4e0a..9264a44dc 100644 --- a/modules/n1ql/pages/n1ql-language-reference/explainfunction.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/explainfunction.adoc @@ -140,5 +140,5 @@ include::n1ql:example$utility/explainfunctionjs.jsonc[] * To manage external libraries and external functions, see xref:server:n1ql:n1ql-rest-api/functions.adoc[]. * To execute a user-defined function, see xref:n1ql-language-reference/execfunction.adoc[]. * To include a user-defined function in an expression, see xref:n1ql-language-reference/userfun.adoc[]. -* To monitor user-defined functions, see xref:server:manage:monitor/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. +* To monitor user-defined functions, see xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. * To drop a user-defined function, see xref:n1ql-language-reference/dropfunction.adoc[]. diff --git a/modules/n1ql/pages/n1ql-language-reference/transactions.adoc b/modules/n1ql/pages/n1ql-language-reference/transactions.adoc index 321b894fc..8951a0805 100644 --- a/modules/n1ql/pages/n1ql-language-reference/transactions.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/transactions.adoc @@ -22,7 +22,7 @@ // Manage cross-references :install-sample-buckets: xref:clusters:data-service/import-data-documents.adoc#import-overview -:sys-transactions: xref:server:manage:monitor/monitoring-n1ql-query.adoc#sys-transactions +:sys-transactions: xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-transactions // Tools cross-references :query-workbench: xref:clusters:query-service/query-workbench.adoc diff --git a/modules/n1ql/pages/n1ql-language-reference/updatestatistics.adoc b/modules/n1ql/pages/n1ql-language-reference/updatestatistics.adoc index c8fcd4782..4a8ad9616 100644 --- a/modules/n1ql/pages/n1ql-language-reference/updatestatistics.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/updatestatistics.adoc @@ -4,7 +4,7 @@ :imagesdir: ../../assets/images // Cross-references -:monitor: xref:server:manage:monitor/monitoring-n1ql-query.adoc +:monitor: xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc :n1ql: xref:n1ql-language-reference :cbo: {n1ql}/cost-based-optimizer.adoc :select: {n1ql}/selectintro.adoc @@ -120,7 +120,7 @@ You don't need to run `UPDATE STATISTICS` with the same index expression for mul === Monitoring Optimizer Statistics To monitor the optimizer statistics, query the `system:dictionary` and `system:dictionary_cache` keyspaces. -For further details, refer to {monitor}[Monitor Queries]. +For further details, refer to {monitor}[]. == Related Links diff --git a/modules/n1ql/pages/n1ql-language-reference/userfun.adoc b/modules/n1ql/pages/n1ql-language-reference/userfun.adoc index 99cd035b0..854718a2c 100644 --- a/modules/n1ql/pages/n1ql-language-reference/userfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/userfun.adoc @@ -189,5 +189,5 @@ At the latitude of the selected hotel, each geohash represents an area of approx * To manage external libraries and external functions, see xref:server:n1ql:n1ql-rest-api/functions.adoc[]. * To execute a user-defined function, see xref:n1ql-language-reference/execfunction.adoc[]. * To see the execution plan for a user-defined function, see xref:n1ql-language-reference/explainfunction.adoc[]. -* To monitor user-defined functions, see xref:server:manage:monitor/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. +* To monitor user-defined functions, see xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. * To drop a user-defined function, see xref:n1ql-language-reference/dropfunction.adoc[]. diff --git a/modules/n1ql/pages/n1ql-manage/index.adoc b/modules/n1ql/pages/n1ql-manage/index.adoc index f3989665d..fa329c828 100644 --- a/modules/n1ql/pages/n1ql-manage/index.adoc +++ b/modules/n1ql/pages/n1ql-manage/index.adoc @@ -24,17 +24,17 @@ You can query the system namespace to get information about the data containers. * xref:n1ql:n1ql-intro/sysinfo.adoc[] -== Manage Primary and Secondary Indexes +== Manage Queries -You can monitor and manage primary and secondary indexes using the Couchbase Capella UI. +You can monitor and manage queries using {sqlpp}. -* xref:clusters:index-service/manage-indexes.adoc[] +* xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc[] -== Manage Queries +== Manage Primary and Secondary Indexes -The Monitor Queries page in the Couchbase Server documentation gives more information about monitoring queries. +You can monitor and manage primary and secondary indexes using the Couchbase Capella UI. -* xref:server:manage:monitor/monitoring-n1ql-query.adoc[] +* xref:clusters:index-service/manage-indexes.adoc[] == Settings and Parameters diff --git a/modules/n1ql/pages/n1ql-manage/monitoring-n1ql-query.adoc b/modules/n1ql/pages/n1ql-manage/monitoring-n1ql-query.adoc new file mode 100644 index 000000000..1fcf85c63 --- /dev/null +++ b/modules/n1ql/pages/n1ql-manage/monitoring-n1ql-query.adoc @@ -0,0 +1,2743 @@ += Manage and Monitor Queries +:description: Monitoring and profiling {sqlpp} queries, query service engines, and corresponding system resources is very important for smoother operational performance and efficiency of the system. +:page-aliases: monitoring:monitoring-n1ql-query, manage:monitor/monitoring-n1ql-query + +[abstract] +{description} +In fact, often it is vital for diagnosing and troubleshooting issues such as query performance, resource bottlenecks, and overloading of various services. + +include::ROOT:partial$component-signpost.adoc[] + +System keyspaces provide various monitoring details and statistics about individual queries and query service. +When running on a cluster with multiple query nodes, stats about all queries on all query nodes are collected in these system keyspaces. + +For example, this can help identify: + +* The top 10 slow or fast queries running on a particular query engine or the cluster. +* Resource usage statistics of the query service, or resources used for a particular query. +* Details about the active, completed, and prepared queries. +* Find long running queries that are running for more than 2 minutes. + +These system keyspaces are like virtual keyspaces that are transient in nature, and are not persisted to disk or permanent storage. +Hence, the information in the keyspaces pertains to the current instantiation of the query service. +You can access the keyspaces using any of the following: + +* {sqlpp} language (from the cbq shell or Query Workbench) +* REST API +* Monitoring SDK + +NOTE: All the power of the {sqlpp} query language can be applied on the keyspaces to obtain various insights. + +The following diagnostics are provided: + +[cols="1,3"] +|=== +| System Catalogs +a| +* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-datastores[system:datastores] +* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-namespaces[system:namespaces] +* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-buckets[system:buckets] +* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-scopes[system:scopes] +* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-keyspaces[system:keyspaces] +* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-indexes[system:indexes] +* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-dual[system:dual] + +| Monitoring Catalogs +a| +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> + +| Security Catalogs +a| +* <> +* <> +* <> +* <> + +| Other +a| +* <> + +NOTE: These are only available using REST APIs. + +|=== + +== Authentication and Client Privileges + +Client applications must be authenticated with sufficient privileges to access system keyspaces. + +* Users must have the *Query System Catalog* role to access restricted system keyspaces. +For more details about user roles, see xref:learn:security/authorization-overview.adoc[Authorization]. + +* In addition, users must have permission to access a bucket, scope, or collection to be able to view that item in the system catalog. +Similarly, users must have SELECT permission on the target of an index to be able to view that index in the system catalog. + +* The following system keyspaces are considered open, that is, all users can access them without any special privileges: + ** `system:dual` + ** `system:datastores` + ** `system:namespaces` + +== Query Monitoring Settings + +The monitoring settings can be set for each query engine (using the REST API) or for each query statement (using the cbq command line tool). +Both settings are actually set via REST endpoints: using the xref:n1ql:n1ql-rest-api/admin.adoc[Admin REST API] (`/admin/settings` endpoint) and the xref:n1ql:n1ql-rest-api/index.adoc[Query REST API] (`/query/service` endpoint). + +The cbq shell and Query Workbench use the Query REST API to set monitoring at the request level. +The Query Workbench automatically enables the profiling and timings. +It can be disabled using the [.ui]*Preferences* option. +For more information refer to the xref:tools:query-workbench.adoc[Query Workbench] section. + +Use the following query parameters to enable, disable, and control the monitoring capabilities, and the level of monitoring and profiling details for each query or globally at a query engine level: + +* profile +* controls + +For more details and examples, see xref:n1ql:n1ql-manage/query-settings.adoc[]. + +=== Enable Settings for a Query Engine + +You can enable profile settings for each query engine. +These examples use local host IP address and default port numbers. +You need to provide correct credentials, IP address, and port details of your setup. + +. Get the current query settings: ++ +[source,sh] +---- +include::n1ql:example$settings/save-node-level-settings.sh[tag=curl] +---- ++ +[source,sh] +---- +cat ./query_settings.json +---- ++ +[source,json] +---- +include::n1ql:example$settings/node-level-settings.jsonc[] +---- + +. Set current query settings profile: + .. To set the query settings saved in a file [.path]_./query_settings.json_, enter the following query: ++ +[source,sh] +---- +curl http://localhost:8093/admin/settings -u user:pword \ + -X POST \ + -d@./query_settings.json +---- + + .. To explicitly specify the settings, enter the following query: ++ +[source,sh] +---- +curl http://localhost:8093/admin/settings -u user:pword \ + -H 'Content-Type: application/json' \ + -d '{"profile": "phases"}' +---- + +. Verify the settings are changed as specified: ++ +[source,sh] +---- +include::n1ql:example$settings/node-level-settings.sh[tag=curl] +---- ++ +[source,json] +---- +{ + // ... + "profile":"phases", + "request-size-cap": 67108864, + "scan-cap": 512, + "servicers": 4, + "timeout": 0, + "txtimeout": "0s", + "use-cbo": true +} +---- + +=== Enable Settings per Session or per Query + +You can enable monitoring and profiling settings for each query statement. +To set query settings using the cbq shell, use the `\SET` command: + +[source,sqlpp] +---- +\set -profile "timings"; +\set; +---- + +[source,none] +---- + Query Parameters : + Parameter name : profile + Value : ["timings"] + ... +---- + +To set query settings using the REST API, specify the parameters in the request body: + +[source,sh] +---- +curl http://localhost:8093/query/service -u user:pword \ + -d 'profile=timings&statement=SELECT * FROM "world" AS hello' +---- + +[#monitor-profile-details] +== Monitoring and Profiling Details + +Couchbase Server provides detailed query monitoring and profiling information. +The profiling and finer query execution timing details can be obtained for any query. + +When a query executes a user-defined function, profiling information is available for the {sqlpp} queries within the user-defined function as well. + +[[profile]] +=== Attribute Profile in Query Response + +When profiling is enabled, a query response includes the profile attribute. +The attribute details are as follows: + +.Attribute Details +[cols="14,8"] +|=== +| Attribute | Example + +a| +`phaseTimes` -- Cumulative execution times for various phases involved in the query execution, such as authorize, indexscan, fetch, parse, plan, run, etc. + +[NOTE] +==== +This value will be dynamic, depending on the documents processed by various phases up to this moment in time. + +A new query on `system:active_requests` will return different values. +==== +a| +[source,json] +---- +"phaseTimes": { + "authorize": "823.631µs", + "fetch": "656.873µs", + "indexScan": "29.146543ms", + "instantiate": "236.221µs", + "parse": "826.382µs", + "plan": "11.831101ms", + "run": "16.892181ms" +} +---- + +a| +`phaseCounts` -- Count of documents processed at selective phases involved in the query execution, such as authorize, indexscan, fetch, parse, plan, run, etc. + +[NOTE] +==== +This value will be dynamic, depending on the documents processed by various phases up to this moment in time. + +A new query on `system:active_requests` will return different values. +==== +a| +[source,json] +---- +"phaseCounts": { + "fetch": 16, + "indexScan": 187 +} +---- + +a| +`phaseOperators` -- Indicates the number of each kind of query operators involved in different phases of the query processing. +For instance, this example, one non covering index path was taken, which involves 1 indexScan and 1 fetch operator. + +A join would have probably involved 2 fetches (1 per keyspace) + +A union select would have twice as many operator counts (1 per each branch of the union). + +This is in essence the count of all the operators in the `executionTimings` field. +a| +[source,json] +---- +"phaseOperators": { + "authorize": 1, + "fetch": 1, + "indexScan": 2 +} +---- + +a| +`executionTimings` -- The execution details such as kernel and service execution times, number of documents processed at each query operator in each phase, and number of phase switches, for various phases involved in the query execution. + +The following statistics are collected for each operator: + +`#operator`:: +Name of the operator. + +`#stats`:: +These values will be dynamic, depending on the documents processed by various phases up to this moment in time. ++ +A new query on `system:active_requests` will return different values. + +`#itemsIn`;; +Number of input documents to the operator. + +`#itemsOut`;; +Number of output documents after the operator processing. + +`#phaseSwitches`;; +Number of switches between executing, waiting for services, or waiting for the `goroutine` scheduler. + +`execTime`;; +Time spent executing the operator code inside {sqlpp} query engine. + +`kernTime`;; +Time spent waiting to be scheduled for CPU time. + +`servTime`;; +Time spent waiting for another service, such as index or data. ++ +* For index scan, it is time spent waiting for GSI/indexer. +* For fetch, it is time spent waiting on the KV store. + +a| +[source,json] +---- +"executionTimings": { + // … + [{ + "#operator": "Fetch", + "#stats": { + "#itemsIn": 187, + "#itemsOut": 16, + "#phaseSwitches": 413, + "execTime": "128.434µs", + "kernTime": "15.027879ms", + "servTime": "1.590934ms", + "state": "services" + }, + "keyspace": "travel-sample", + "namespace": "default" + }, + { + "#operator": "IntersectScan", + "#stats": { + "#itemsIn": 187, + "#itemsOut": 187, + "#phaseSwitches": 749, + "execTime": "449.944µs", + "kernTime": "14.625524ms" + } + // … + ]} +---- + +|=== + +These statistics (`kernTime`, `servTime`, and `execTime`) can be very helpful in troubleshooting query performance issues, such as: + +* A high `servTime` for a low number of items processed is an indication that the indexer or KV store is stressed. +* A high `kernTime` means there is a downstream issue in the query plan or the query server having many requests to process (so the scheduled waiting time will be more for CPU time). + +.Phases Profile +==== +The cbq engine must be started with authorization, for example: + +[source,sh] +---- +./cbq -engine=http://localhost:8091/ -u Administrator -p pword +---- + +Using the cbq shell, show the statistics collected when the `profile` is set to `phases`: + +[source,sqlpp] +---- +\set -profile "phases"; +SELECT * FROM `travel-sample`.inventory.airline LIMIT 1; +---- + +[source,json] +---- +{ + "requestID": "06d6c1c2-1a8a-4989-a856-7314f9eddee5", + "signature": { + "*": "*" + }, + "results": [ + { + "airline": { + "callsign": "MILE-AIR", + "country": "United States", + "iata": "Q5", + "icao": "MLA", + "id": 10, + "name": "40-Mile Air", + "type": "airline" + } + } + ], + "status": "success", + "metrics": { + "elapsedTime": "12.77927ms", + "executionTime": "12.570648ms", + "resultCount": 1, + "resultSize": 254, + "serviceLoad": 12 + }, + "profile": { + "phaseTimes": { + "authorize": "19.629µs", + "fetch": "401.997µs", + "instantiate": "147.686µs", + "parse": "4.545234ms", + "plan": "409.364µs", + "primaryScan": "6.103775ms", + "run": "6.699056ms" + }, + "phaseCounts": { + "fetch": 1, + "primaryScan": 1 + }, + "phaseOperators": { + "authorize": 1, + "fetch": 1, + "primaryScan": 1 + }, + "requestTime": "2021-04-30T18:37:56.394Z", + "servicingHost": "127.0.0.1:8091" + } +} +---- +==== + +.Timings Profile +==== +Using the cbq shell, show the statistics collected when `profile` is set to `timings`: + +[source,sqlpp] +---- +\set -profile "timings"; +SELECT * FROM `travel-sample`.inventory.airline LIMIT 1; +---- + +[source,json] +---- +{ + "requestID": "268a1240-6864-43a2-af13-ccb8d1e50abf", + "signature": { + "*": "*" + }, + "results": [ + { + "airline": { + "callsign": "MILE-AIR", + "country": "United States", + "iata": "Q5", + "icao": "MLA", + "id": 10, + "name": "40-Mile Air", + "type": "airline" + } + } + ], + "status": "success", + "metrics": { + "elapsedTime": "2.915245ms", + "executionTime": "2.755355ms", + "resultCount": 1, + "resultSize": 254, + "serviceLoad": 12 + }, + "profile": { + "phaseTimes": { + "authorize": "18.096µs", + "fetch": "388.122µs", + "instantiate": "31.702µs", + "parse": "646.157µs", + "plan": "120.427µs", + "primaryScan": "1.402918ms", + "run": "1.936852ms" + }, + "phaseCounts": { + "fetch": 1, + "primaryScan": 1 + }, + "phaseOperators": { + "authorize": 1, + "fetch": 1, + "primaryScan": 1 + }, + "requestTime": "2021-04-30T18:40:13.239Z", + "servicingHost": "127.0.0.1:8091", + "executionTimings": { + "#operator": "Authorize", + "#stats": { + "#phaseSwitches": 4, + "execTime": "1.084µs", + "servTime": "17.012µs" + }, + "privileges": { + "List": [ + { + "Target": "default:travel-sample.inventory.airline", + "Priv": 7, + "Props": 0 + } + ] + }, + "~child": { + "#operator": "Sequence", + "#stats": { + "#phaseSwitches": 1, + "execTime": "2.474µs" + }, + "~children": [ + { + "#operator": "PrimaryScan3", + "#stats": { + "#itemsOut": 1, + "#phaseSwitches": 7, + "execTime": "18.584µs", + "kernTime": "8.869µs", + "servTime": "1.384334ms" + }, + "bucket": "travel-sample", + "index": "def_inventory_airline_primary", + "index_projection": { + "primary_key": true + }, + "keyspace": "airline", + "limit": "1", + "namespace": "default", + "scope": "inventory", + "using": "gsi" + }, + { + "#operator": "Fetch", + "#stats": { + "#itemsIn": 1, + "#itemsOut": 1, + "#phaseSwitches": 10, + "execTime": "25.64µs", + "kernTime": "1.427752ms", + "servTime": "362.482µs" + }, + "bucket": "travel-sample", + "keyspace": "airline", + "namespace": "default", + "scope": "inventory" + }, + { + "#operator": "InitialProject", + "#stats": { + "#itemsIn": 1, + "#itemsOut": 1, + "#phaseSwitches": 9, + "execTime": "6.006µs", + "kernTime": "1.825917ms" + }, + "result_terms": [ + { + "expr": "self", + "star": true + } + ] + }, + { + "#operator": "Limit", + "#stats": { + "#itemsIn": 1, + "#itemsOut": 1, + "#phaseSwitches": 4, + "execTime": "2.409µs", + "kernTime": "2.094µs" + }, + "expr": "1" + }, + { + "#operator": "Stream", + "#stats": { + "#itemsIn": 1, + "#itemsOut": 1, + "#phaseSwitches": 6, + "execTime": "46.964µs", + "kernTime": "1.844828ms" + } + } + ] + }, + "~versions": [ + "7.0.0-N1QL", + "7.0.0-4960-enterprise" + ] + } + } +} +---- +==== + +[[plan]] +=== Attribute Meta in System Keyspaces + +The `meta().plan` virtual attribute captures the whole query plan, and monitoring stats of various phases and involved query operators. +The `meta().plan` must be explicitly called in the SELECT query projection list. + +The `meta().plan` attribute is enabled only for individual requests that are running (`active_requests`) or completed (`completed_requests`) when the profile is set to timings (`profile ="timings"`) for each individual request. +If at the engine level, the profile is set to off and individual requests have been run with `profile ="timings"`, then the system keyspaces will return the plan only for those requests. + +Since there may be a combination of profile settings for all of the requests reported by the system keyspaces, not all requests returned will have a `meta().plan` attachment. + +NOTE: For the `system:prepareds` requests, the `meta().plan` is available at all times since the `PREPARE` statement is not dependant on the profile setting. + +This attribute is enabled for the following system keyspaces: + +* <> +* <> +* <> + +For a detailed example, see <>. + +== Monitor Query Clusters + +Couchbase Server allows you to monitor many aspects of an active cluster: cluster-aware operations, diagnostics, and more system keyspaces features that cover multiple nodes. +Functionalities include: + +* Ability to access active / completed / prepared requests across all Query nodes from {sqlpp}. +* Ability to list nodes by type and with status from {sqlpp}. +* Ability to list system keyspaces from `system:keyspaces`. +* Extra fields in `system:active_requests` and `system:completed_requests`. +* Counters to keep track of specific requests, such as cancelled requests. +* Killing request for CREATE INDEX. + +=== System Keyspaces + +* The `system:keyspaces` keyspace can be augmented to list system keyspaces with a static map. +The small disadvantage of this is that it has to be maintained as new system keyspaces are added. +* The `system:active_requests` and `system:completed_requests` keyspaces can report scan consistency. +* The `system:prepareds` keyspace can list min and max execution and service times, as well as average times. + +=== cbq-engine-cbauth + +`cbq-engine-cbauth` is an internal user that the query service uses to allow Query Workbench clients to query across multiple query nodes. + +Since Query Workbench can connect to the same node only when cbq-engine is running, Query Workbench cannot do any query-clustered operations. + +To get around this block, once the Query Workbench clients connect to a query node, this internal user (cbq-engine-cbauth) will be used to do any further inter-node user verification. + +[#vitals] +== System Vitals + +The [.cmd]`Vitals` API provides data about the running state and health of the query engine, such as number of logical cores, active threads, queued threads, CPU utilization, memory usage, network utilization, garbage collection percentage, and so on. +This information can be very useful to assess the current workload and performance characteristics of a query engine, and hence load-balance the requests being sent to various query engines. + +For field names and meanings, refer to xref:n1ql:n1ql-rest-api/admin.adoc#_vitals[Vitals]. + +=== Get System Vitals + +[source,sh] +---- +curl -u Administrator:pword http://localhost:8093/admin/vitals +---- + +[source,json] +---- +{ + "uptime": "7h39m32.668577197s", + "local.time": "2021-04-30 18:42:39.517208807 +0000 UTC m=+27573.945319668", + "version": "7.0.0-N1QL", + "total.threads": 191, + "cores": 2, + "gc.num": 669810600, + "gc.pause.time": "57.586373ms", + "gc.pause.percent": 0, + "memory.usage": 247985184, + "memory.total": 11132383704, + "memory.system": 495554808, + "cpu.user.percent": 0, + "cpu.sys.percent": 0, + "request.completed.count": 140, + "request.active.count": 0, + "request.per.sec.1min": 0.0018, + "request.per.sec.5min": 0.0055, + "request.per.sec.15min": 0.0033, + "request_time.mean": "536.348163ms", + "request_time.median": "54.065567ms", + "request_time.80percentile": "981.869933ms", + "request_time.95percentile": "2.543128455s", + "request_time.99percentile": "4.627922799s", + "request.prepared.percent": 0 +} +---- + +[#sys-active-req] +== Monitor and Manage Active Requests + +The `system:active_requests` catalog lists all currently executing active requests or queries. + +For field names and meanings, refer to xref:n1ql:n1ql-rest-api/admin.adoc#_requests[Requests]. +The profile related attributes are described in the section <> + +[[sys-active-get]] +=== Get Active Requests + +To view active requests with Admin REST API: + +[source,sh] +---- +curl -u Administrator:pword http://localhost:8093/admin/active_requests +---- + +To view active requests with {sqlpp}, including the query plan: + +[source,sqlpp] +---- +SELECT *, meta().plan FROM system:active_requests; +---- + +[[sys-active-delete]] +=== Terminate an Active Request + +The DELETE command can be used to terminate an active request, for instance, a non-responding or a long-running query. + +To terminate an active request [.var]`uuid` with the Admin REST API: + +[source,sh] +---- +curl -u Administrator:pword -X DELETE http://localhost:8093/admin/active_requests/uuid +---- + +To terminate an active request [.var]`uuid` with {sqlpp}: + +[source,sqlpp] +---- +DELETE FROM system:active_requests WHERE requestId = "uuid"; +---- + +[[sys-active-examples]] +=== Examples + +.Get Active +==== +[source,sqlpp] +---- +SELECT *, meta().plan FROM system:active_requests; +---- + +[source,json] +---- +[ + { + "active_requests": { + "clientContextID": "81984707-a9cd-4b78-9110-d130eb580d7f", + "elapsedTime": "65.543946ms", + "executionTime": "65.507111ms", + "node": "127.0.0.1:8091", + "phaseCounts": { + "primaryScan": 1 + }, + "phaseOperators": { + "authorize": 1, + "fetch": 1, + "primaryScan": 1 + }, + "phaseTimes": { + "authorize": "2.361862ms", + "fetch": "7.222µs", + "instantiate": "13.233µs", + "parse": "660.048µs", + "plan": "52.877µs", + "primaryScan": "41.125271ms" + }, + "remoteAddr": "127.0.0.1:57065", + "requestId": "27e73286-c6cc-4c26-8977-ef8d68e91c8f", + "requestTime": "2019-05-06 09:08:42.431161361 -0700 PDT m=+6976.301141271", + "scanConsistency": "unbounded", + "state": "running", + "statement": "SELECT *, meta().plan FROM system:active_requests;", + "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:66.0) Gecko/20100101 Firefox/66.0 (Couchbase Query Workbench (6.5.0-2859-enterprise))", + "users": "Administrator" + }, + "plan": { // <1> + "#operator": "Sequence", + "#stats": { + "#phaseSwitches": 1, + "execTime": "1.661µs" + }, + "~children": [ + { + "#operator": "Authorize", + "#stats": { + "#phaseSwitches": 3, + "execTime": "4.844µs", + "servTime": "2.357018ms" + }, + // ... + } + ] + }, + "plan": { + "#operator": "Sequence", + "#stats": { + "#phaseSwitches": 1, + "execTime": "1.661µs" + }, + "~children": [ + { + "#operator": "Authorize", + "#stats": { + "#phaseSwitches": 3, + "execTime": "4.844µs", + "servTime": "2.357018ms" + }, + "privileges": { + "List": [ + { + "Priv": 4, + "Target": "#system:active_requests" + } + ] + }, + "~child": { + "#operator": "Sequence", + "#stats": { + "#phaseSwitches": 1, + "execTime": "2.71µs" + }, + "~children": [ + { + "#operator": "PrimaryScan", + "#stats": { + "#itemsOut": 1, + "#phaseSwitches": 7, + "execTime": "22.314µs", + "kernTime": "2.13µs", + "servTime": "41.102957ms" + }, + "index": "#primary", + "keyspace": "active_requests", + "namespace": "#system", + "using": "system" + }, + { + "#operator": "Fetch", + "#stats": { + "#itemsIn": 1, + "#phaseSwitches": 5, + "execTime": "7.222µs", + "kernTime": "41.130913ms", + "servTime": "22.888285ms", + "state": "services" + }, + "keyspace": "active_requests", + "namespace": "#system" + }, + { + "#operator": "Sequence", + "#stats": { + "#phaseSwitches": 1, + "execTime": "1.544µs" + }, + "~children": [ + { + "#operator": "InitialProject", + "#stats": { + "#phaseSwitches": 1, + "execTime": "980ns", + "kernTime": "64.066618ms", + "state": "kernel" + }, + "result_terms": [ + { + "expr": "self", + "star": true + }, + { + "expr": "(meta(`active_requests`).`plan`)" + } + ] + }, + { + "#operator": "FinalProject", + "#stats": { + "#phaseSwitches": 1, + "execTime": "827ns" + } + } + ] + } + ] + } + }, + { + "#operator": "Stream", + "#stats": { + "#phaseSwitches": 1, + "execTime": "1.29µs", + "kernTime": "66.511806ms", + "state": "kernel" + } + } + ], + "~versions": [ + "2.0.0-N1QL", + "6.5.0-2859-enterprise" + ] + } + } +] +---- + +<1> The *plan* section contains a tree of operators that combine to execute the {sqlpp} query. +The root operator is a Sequence, which itself has a collection of child operators like Authorize, PrimaryScan, Fetch, and possibly even more Sequences. +==== + +[#sys-prepared] +== Monitor and Manage Prepared Statements + +The `system:prepareds` catalog provides data about the known prepared statements and their state in a query engine's prepared statement cache. +For each prepared statement, this catalog provides information such as name, statement, query plan, last use time, number of uses, and so on. + +For field names and meanings, refer to xref:n1ql:n1ql-rest-api/admin.adoc#_statements[Statements]. +The `system:prepareds` catalog returns all the properties that the {sqlpp} Admin REST API would return for a specific prepared statement. +In addition, the `system:prepareds` catalog also returns the following properties. + +[options="header", cols=".^3a,.^11a,.^4a"] +|=== +|Name|Description|Schema +|**node** + +__required__ +|The node on which the prepared statement is stored. +|string + +|**namespace** + +__required__ +|The namespace in which the prepared statement is stored. +Currently, only the `default` namespace is available. +|string +|=== + +A prepared statement is created and stored relative to the current xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context[query context]. +You can create multiple prepared statements with the same name, each stored relative to a different query context. +This enables you to run multiple instances of the same application against different datasets. + +When there are multiple prepared statements with the same name in different query contexts, the name of the prepared statement in the `system:prepareds` catalog includes the associated query context in brackets. + +[[sys-prepared-get]] +=== Get Prepared Statements + +To get a list of all known prepared statements, you can use the Admin REST API or a {sqlpp} query: + +[source,sh] +---- +curl -u Administrator:pword http://localhost:8093/admin/prepareds +---- + +[source,sqlpp] +---- +SELECT * FROM system:prepareds; +---- + +To get information about a specific prepared statement [.var]`example1`, you can use the Admin REST API or a {sqlpp} query: + +[source,sh] +---- +curl -u Administrator:pword http://localhost:8093/admin/prepareds/example1 +---- + +[source,sqlpp] +---- +SELECT * FROM system:prepareds WHERE name = "example1"; +---- + +[[sys-prepared-delete]] +=== Delete Prepared Statements + +To delete a specific prepared statement [.var]`p1`, you can use the Admin REST API or a {sqlpp} query: + +[source,sh] +---- +curl -u Administrator:pword -X DELETE http://localhost:8093/admin/prepareds/p1 +---- + +[source,sqlpp] +---- +DELETE FROM system:prepareds WHERE name = "p1"; +---- + +To delete all the known prepared statements, you must use a {sqlpp} query: + +[source,sqlpp] +---- +DELETE FROM system:prepareds; +---- + +[[sys-prepared-examples]] +=== Examples + +.Get Prepared +==== +.Prepared statement with default query context -- using cbq +[source,sqlpp] +---- +\UNSET -query_context; +PREPARE p1 AS SELECT * FROM `travel-sample`.inventory.airline WHERE iata = "U2"; +---- + +[source,json] +---- +{ + "requestID": "64069886-eb17-4fa6-8cc8-e60ebe93d97c", + "signature": "json", + "results": [ + { + "encoded_plan": "H4sIAAAAAAAA/wEAAP//AAAAAAAAAAA=", + "featureControls": 76, + "indexApiVersion": 4, + "indexScanKeyspaces": { + "default:travel-sample.inventory.airline": false + }, + "name": "[127.0.0.1:8091]p1", + "namespace": "default", + "operator": { + "#operator": "Authorize", + "privileges": { + "List": [ + { + "Priv": 7, + "Props": 0, + "Target": "default:travel-sample.inventory.airline" + } + ] + }, + "~child": { + "#operator": "Sequence", + "~children": [ + { + "#operator": "Sequence", + "~children": [ + { + "#operator": "PrimaryScan3", + "bucket": "travel-sample", + "index": "def_inventory_airline_primary", + "index_projection": { + "primary_key": true + }, + "keyspace": "airline", + "namespace": "default", + "scope": "inventory", + "using": "gsi" + }, + { + "#operator": "Fetch", + "bucket": "travel-sample", + "keyspace": "airline", + "namespace": "default", + "scope": "inventory" + }, + { + "#operator": "Parallel", + "~child": { + "#operator": "Sequence", + "~children": [ + { + "#operator": "Filter", + "condition": "((`airline`.`iata`) = \"U2\")" + }, + { + "#operator": "InitialProject", + "result_terms": [ + { + "expr": "self", + "star": true + } + ] + } + ] + } + } + ] + }, + { + "#operator": "Stream" + } + ] + } + }, + "queryContext": "", + "reqType": "SELECT", + "signature": { + "*": "*" + }, + "text": "PREPARE p1 AS SELECT * FROM `travel-sample`.inventory.airline WHERE iata = \"U2\";", + "useCBO": true + } + ], + "status": "success", + "metrics": { + "elapsedTime": "105.814848ms", + "executionTime": "105.648798ms", + "resultCount": 1, + "resultSize": 3301, + "serviceLoad": 12 + } +} +---- + +.Prepared statement with specified query context -- using cbq +[source,sqlpp] +---- +\SET -query_context travel-sample.inventory; +PREPARE p1 AS SELECT * FROM airline WHERE iata = "U2"; +---- + +[source,json] +---- +{ + "requestID": "1c90603e-5e42-42b4-9362-4fc96bf895ac", + "signature": "json", + "results": [ + { + "encoded_plan": "H4sIAAAAAAAA/wEAAP//AAAAAAAAAAA=", + "featureControls": 76, + "indexApiVersion": 4, + "indexScanKeyspaces": { + "default:travel-sample.inventory.airline": false + }, + "name": "[127.0.0.1:8091]p1", + "namespace": "default", + "operator": { + "#operator": "Authorize", + "privileges": { + "List": [ + { + "Priv": 7, + "Props": 0, + "Target": "default:travel-sample.inventory.airline" + } + ] + }, + "~child": { + "#operator": "Sequence", + "~children": [ + { + "#operator": "Sequence", + "~children": [ + { + "#operator": "PrimaryScan3", + "bucket": "travel-sample", + "index": "def_inventory_airline_primary", + "index_projection": { + "primary_key": true + }, + "keyspace": "airline", + "namespace": "default", + "scope": "inventory", + "using": "gsi" + }, + { + "#operator": "Fetch", + "bucket": "travel-sample", + "keyspace": "airline", + "namespace": "default", + "scope": "inventory" + }, + { + "#operator": "Parallel", + "~child": { + "#operator": "Sequence", + "~children": [ + { + "#operator": "Filter", + "condition": "((`airline`.`iata`) = \"U2\")" + }, + { + "#operator": "InitialProject", + "result_terms": [ + { + "expr": "self", + "star": true + } + ] + } + ] + } + } + ] + }, + { + "#operator": "Stream" + } + ] + } + }, + "queryContext": "travel-sample.inventory", + "reqType": "SELECT", + "signature": { + "*": "*" + }, + "text": "PREPARE p1 AS SELECT * FROM airline WHERE iata = \"U2\";", + "useCBO": true + } + ], + "status": "success", + "metrics": { + "elapsedTime": "17.476424ms", + "executionTime": "17.187836ms", + "resultCount": 1, + "resultSize": 3298, + "serviceLoad": 12 + } +} +---- + +.List prepared statements +[source,sqlpp] +---- +SELECT *, meta().plan FROM system:prepareds; +---- + +[source,json] +---- +{ + "requestID": "d976e59a-d74e-4350-b0df-fa137099d594", + "signature": { + "*": "*", + "plan": "json" + }, + "results": [ + { + "plan": { + // ... + }, + "prepareds": { + "encoded_plan": "H4sIAAAAAAAA/6RTUW/TPBT9K9H5XrbJ30QBMcmIhzJ1AjG0qh3wwKbEJLedmWt71061UIXfjpxkndYhENpbYt97z7nn+GxAtnQVVbk3ykICAgtSsWY6djayMwHy6JWAthXdjr3+TBy0s5Avh7N5qewHaoJXJQXIDSpaqNpEGVmtyfwf1MobOtR2TTY6bg6VZqMtQS6UCdQKWLUiSPgR+u9uFOTdIAg4T6yi4zT+v/sfjOt45Vj/IAh41mttaNmTONUhQn7d4FzxkuL9tL/SEpiyXkMepQ/nA+Sz9rIV+FleaVPtMpjTTU22TG19AZPtcP+5aMp6pbhJcr6AwLe6vO54P+CLQfR+n3zLPh/Y576fcleXe3bfqYydYxsMt/k1NZCR66T+9eAdJO4l+L0NoXQ+nWxhIVAHbZeQWAaNVjxc6YRiefWnXZ6EvYs2VayMIYMneXWiTSSGQOlspXvhsLdXDPyKw0KrqIr97E12gU/PL7D/iMh7q6NWZtpLDwGmUJuYR+JV6ADp1qfCQGaRVouKBzsu28s2vbYd4pFJrZDuBFJOtyE8Go0EbmriJqWVbmOfYKab86aTaz45nRyfJxC9tF2skyoHkDhAKzC0TGeT6Xg2yfwoG8+zvic7yE5mZx+z4oFpxePEZF/eTWaTLMmyFeV19zLo+O3ZsNivAAAA//+q+jhuaAQAAA==", + "featuresControl": 76, + "indexApiVersion": 4, + "indexScanKeyspaces": { + "default:travel-sample.inventory.airline": false + }, + "name": "p1", // <1> + "namespace": "default", + "node": "127.0.0.1:8091", + "statement": "PREPARE p1 AS SELECT * FROM `travel-sample`.inventory.airline WHERE iata = \"U2\";", + "uses": 0 + } + }, + { + "plan": { + // ... + }, + "prepareds": { + "encoded_plan": "H4sIAAAAAAAA/6STT28TMRDFv8rqcWkrExFAVDLiEKpUIIoaJQUOtNqY3Ulq6tju2Bt1iZbPjry7TWmKQKg3/xnPvPk9zwZkC1dSmXujLCQgsCAVK6YjZyM7EyAPXwloW9LNyOvPxEE7C/myP5sVyn6gOnhVUIDcoKSFqkyUkdWazNOgVt7QQNs12ei4HijNRluCXCgTqBGwakWQ8EN06zYV5G0iCDhPrKLjlP7J3QajKl461j8IAp71WhtadiJOdIiQXzc4U7ykeJftn7IEJqzXkIdp4XyAfNZcNAI/i0ttyl0FM7quyBbpWRfAZNu6/x00Yb1SXCecLyDwrSquWt339KKH3vWTb9Xnvfrcd1lu43LP7jsVsXVsg/42v6IaMnKV6F/13kHiDsGfbQiF8+lkWxYCVdB2CYll0GjE/ZaOKRaXf+vlUbV3q00UK2PI4FFeHWsTiSFQOFvqDhz29ua9vvlgrlVU8/3sTXaOT8/Psf9AyHuro1Zm0qGHAFOoTMwj8Sq0BenGp8BAZpFai4p7Oy6aiyb9th3hkUmtkO4E0pxuh/BwOBS4rojrNK108wDy4HevmK7P6pbibHwyPjpLtfXSttOeYB1A4gCNQJ9pMh1PRtNx5ofZaJZ1b7KD7Hh6+jHreWRf3o2n4ywx2RJ53X4LOnp72nf1KwAA////9+bsZQQAAA==", + "featuresControl": 76, + "indexApiVersion": 4, + "indexScanKeyspaces": { + "default:travel-sample.inventory.airline": false + }, + "name": "p1(travel-sample.inventory)", // <2> + "namespace": "default", + "node": "127.0.0.1:8091", + "statement": "PREPARE p1 AS SELECT * FROM airline WHERE iata = \"U2\";", + "uses": 0 + } + } + ], + "status": "success", + "metrics": { + "elapsedTime": "25.323496ms", + "executionTime": "25.173646ms", + "resultCount": 2, + "resultSize": 7891, + "serviceLoad": 12 + } +} +---- + +Note that the names of the prepared statements are identical, but they are associated with different query contexts. + +<.> The name of the prepared statement for the default query context +<.> The name of the prepared statement showing the associated query context +==== + +[#sys-completed-req] +== Monitor and Manage Completed Requests + +By default, the `system:completed_requests` catalog maintains a list of the most recent completed requests that have run longer than a predefined threshold of time. +(You can also log completed requests that meet other conditions that you define.) + +For each completed request, this catalog maintains information such as requestId, statement text, prepared name (if prepared statement), request time, service time, and so on. +This information provides a general insight into the health and performance of the query engine and the cluster. + +For field names and meanings, refer to xref:n1ql:n1ql-rest-api/admin.adoc#_requests[Requests]. +Most field names and meanings match exactly those of `system:active_requests`. + +Note that the `completed` state means that the request was started and completed by the Query service, but it does not mean that it was necessarily successful. +The request could have been successful, or completed with errors. + +To find requests that completed successfully, search for completed requests whose `state` is `completed` and whose `errorCount` field has the value `0`. + +[NOTE] +==== +Request profiling affects the `system:completed_requests` keyspace in the following ways: + +* When the feature is turned on, completed requests are stored with their execution plan. +* Profiling information is likely to use 100KB+ per entry. +* Due to the added overhead of running both profiling and logging, we recommend turning on both of them only when needed. +Running only one of them continuously has no noticeable affect on performance. +* Profiling does not carry any extra cost beyond memory for completed requests, so it's fine to run it continuously. +==== + +[[sys-completed-get]] +=== Get Completed Requests + +To get a list of all logged completed requests using the Admin REST API: + +[source,sh] +---- +curl -u Administrator:pword http://localhost:8093/admin/completed_requests +---- + +To get a list of all logged completed requests using {sqlpp}, including the query plan: + +[source,sqlpp] +---- +SELECT *, meta().plan FROM system:completed_requests; +---- + +To get a list of all logged completed requests using {sqlpp}, including only successful requests: + +[source,sqlpp] +---- +SELECT * FROM system:completed_requests +WHERE state = "completed" AND errorCount = 0; +---- + +[[sys-completed-delete]] +=== Purge the Completed Requests + +To purge a completed request [.var]`uuid` with the Admin REST API: + +[source,sh] +---- +curl -u Administrator:pword -X DELETE http://localhost:8093/admin/completed_requests/uuid +---- + +To purge a completed request [.var]`uuid` with {sqlpp}: + +[source,sqlpp] +---- +DELETE FROM system:completed_requests WHERE requestId = "uuid"; +---- + +To purge the completed requests for a given time period, use: + +[source,sqlpp] +---- +DELETE FROM system:completed_requests WHERE requestTime LIKE "2015-09-09%"; +---- + +[[sys-completed-config]] +=== Configure the Completed Requests + +You can configure the `system:completed_requests` keyspace by specifying parameters through the Admin API settings endpoint. + +In Couchbase Server 6.5 and later, you can specify the conditions for completed request logging using the `completed` field. + +This field takes a JSON object containing the names and values of logging qualifiers. +Completed requests that meet the defined qualifiers are logged. + +[source,sh] +---- +curl http://localhost:8093/admin/settings -u Administrator:password \ + -H 'Content-Type: application/json' \ + -d '{"completed": {"user": "marco", "error": 12003}}' +---- + +==== Logging Qualifiers + +You can specify the following logging qualifiers. +A completed request is logged if _any_ of the qualifiers are met (logical OR). + +[horizontal] +`threshold`:: The execution time threshold in milliseconds. +`aborted`:: Whether to log requests that generate a panic. +`error`:: Log requests returning this error number. +`client`:: Log requests from this IP address. +`user`:: Log requests with this user name. +`context`:: Log requests with this client context ID. + +For full details, refer to xref:n1ql:n1ql-rest-api/admin.adoc#_logging_parameters[Logging parameters]. + +The basic syntax adds a qualifier to the logging parameters, i.e. any existing qualifiers are not removed. +You can change the value of a logging qualifier by specifying the same qualifier again with a new value. + +To add a new instance of an existing qualifier, use a plus sign (`+`) before the qualifier name, e.g. `+user`. +To remove a qualifier, use a minus sign (`-`) before the qualifier name, e.g. `-user`. + +For example, the following request will add user `simon` to those tracked, and remove error `12003`. + +[source,sh] +---- +curl http://localhost:8093/admin/settings -u Administrator:password \ + -H 'Content-Type: application/json' \ + -d '{"completed": {"+user": "simon", "-error": 12003}}' +---- + +Similarly, you could remove all logging by execution time with the following request, as long as the value matches the existing threshold. + +[source,sh] +---- +curl http://localhost:8093/admin/settings -u Administrator:password \ + -H 'Content-Type: application/json' \ + -d '{"completed": {"-threshold": 1000}}' +---- + +==== Tagged Sets + +You can also specify qualifiers that have to be met as a group for the completed request to be logged (logical AND). + +To do this, specify the `tag` field along with a set of qualifiers, like so: + +[source,sh] +---- +curl http://localhost:8093/admin/settings -u Administrator:password \ + -H 'Content-Type: application/json' \ + -d '{"completed": {"user": "marco", "error": 12003, "tag": "both_user_and_error"}}' +---- + +In this case, the request will be logged when both user and error match. + +The tag name can be any string that is meaningful and unique. +Requests that match a tagged set of conditions are logged with a field `~tag`, which is set to the name of the tag. + +To add a qualifier to a tagged set, specify the tag name again along with the new qualifier: + +[source,sh] +---- +curl http://localhost:8093/admin/settings -u Administrator:password \ + -H 'Content-Type: application/json' \ + -d '{"completed": {"client": "172.1.2.3", "tag": "both_user_and_error"}}' +---- + +You cannot add a new instance of an existing qualifier to a tagged set using a plus sign (`+`) before the qualifier name. +For example, you cannot add a `user` qualifier to a tagged set that already contains a `user` qualifier. +If you need to track two users with the same error, create two tagged sets, one per user. + +You can remove a qualifier from a tagged set using a minus sign (`-`) before the qualifier name, e.g. `-user`. +When you remove the last qualifier from a tagged set, the tagged set is removed. + +[NOTE] +-- +You can specify multiple tagged sets. +In this case, completed requests are logged if they match all of the qualifiers in any of the tagged sets. + +You can also specify a mixture of tagged sets and individual qualifiers. +In this case, completed requests are logged if they match any of the individual qualifiers, or all of the qualifiers in any of the tagged sets. +-- + +==== Completed Threshold + +The [.param]`completed-threshold` field provides another way of specifying the `threshold` qualifier within the `completed` field. + +This field sets the minimum request duration after which requests are added to the `system:completed_requests` catalog. +The default value is 1000ms. +Specify [.in]`0` to log all requests and [.in]`-1` to not log any requests to the keyspace. + +To specify a different value, use: + +[source,sh] +---- +curl http://localhost:port/admin/settings -u user:pword \ + -H 'Content-Type: application/json' \ + -d '{"completed-threshold":0}' +---- + +==== Completed Limit + +The [.param]`completed-limit` field sets the number of most recent requests to be tracked in the `system:completed_requests` catalog. +The default value is 4000. +Specify [.in]`0` to not track any requests and [.in]`-1` to set no limit. + +To specify a different value, use: + +[source,sh] +---- +curl http://localhost:port/admin/settings -u user:pword \ + -H 'Content-Type: application/json' \ + -d '{"completed-limit":1000}' +---- + +[[sys-completed-examples]] +=== Examples + +[[example-2]] +.Completed Request +==== +First, using the cbq shell, we set `profile = "timings"` and run a long query which takes at least 1000ms (the default value of the `completed-threshold` query setting) to get registered in the `system:completed_requests` keyspace: + +.Query 1 +[source,sqlpp] +---- +\set -profile "timings"; +SELECT * FROM `travel-sample`.inventory.route ORDER BY sourceairport; +---- + +Now, using the cbq shell, we change the profile setting to "phases" and rerun another long query: + +.Query 2 +[source,sqlpp] +---- +\set -profile "phases"; +SELECT * FROM `travel-sample`.inventory.route ORDER BY destinationairport; +---- + +Run a query `system:completed_requests` keyspace with `meta().plan`. + +.Query 3 +[source,sqlpp] +---- +SELECT meta().plan, * from system:completed_requests; +---- + +.Result +[source,json] +---- +{ + "requestID": "4a36e1dc-cea0-4ba2-a428-258511d50582", + "signature": { + "*": "*", + "plan": "json" + }, + "results": [ + // ... + { // <1> + "completed_requests": { + "elapsedTime": "15.641879295s", + "errorCount": 0, + "node": "127.0.0.1:8091", + "phaseCounts": { + "fetch": 24024, + "primaryScan": 24024, + "sort": 24024 + }, + "phaseOperators": { + "authorize": 1, + "fetch": 1, + "primaryScan": 1, + "sort": 1 + }, + "phaseTimes": { + "authorize": "51.305µs", + "fetch": "3.27276723s", + "instantiate": "60.662µs", + "parse": "66.701943ms", + "plan": "15.12951ms", + "primaryScan": "171.439769ms", + "run": "15.548781894s", + "sort": "153.767638ms" + }, + "remoteAddr": "172.17.0.1:56962", + "requestId": "08445bae-66ef-4ccd-8b2d-ea899b453a1b", + "requestTime": "2021-04-30T21:14:57.576Z", + "resultCount": 24024, + "resultSize": 81821919, + "scanConsistency": "unbounded", + "serviceTime": "15.630714144s", + "state": "completed", + "statement": "SELECT * FROM `travel-sample`.inventory.route ORDER BY destinationairport;", + "useCBO": true, + "userAgent": "Go-http-client/1.1 (CBQ/2.0)", + "users": "Administrator" + } + }, + // ... + { // <2> + "completed_requests": { + "elapsedTime": "15.321128463s", + "errorCount": 0, + "node": "127.0.0.1:8091", + "phaseCounts": { + "fetch": 24024, + "primaryScan": 24024, + "sort": 24024 + }, + "phaseOperators": { + "authorize": 1, + "fetch": 1, + "primaryScan": 1, + "sort": 1 + }, + "phaseTimes": { + "authorize": "23.037µs", + "fetch": "3.092306635s", + "instantiate": "7.313569ms", + "parse": "579.368µs", + "plan": "2.449143ms", + "primaryScan": "153.686873ms", + "run": "15.29433203s", + "sort": "147.889352ms" + }, + "remoteAddr": "172.17.0.1:56900", + "requestId": "5eefc9e5-bdaa-4824-bcd7-47977eb1f08a", + "requestTime": "2021-04-30T21:13:58.707Z", + "resultCount": 24024, + "resultSize": 81821919, + "scanConsistency": "unbounded", + "serviceTime": "15.30510306s", + "state": "completed", + "statement": "SELECT * FROM `travel-sample`.inventory.route ORDER BY sourceairport;", + "useCBO": true, + "userAgent": "Go-http-client/1.1 (CBQ/2.0)", + "users": "Administrator" + }, + "plan": { + "#operator": "Authorize", + "#stats": { + "#phaseSwitches": 4, + "execTime": "1.725µs", + "servTime": "21.312µs" + }, + "privileges": { + "List": [ + { + "Priv": 7, + "Props": 0, + "Target": "default:travel-sample.inventory.route" + } + ] + }, + "~child": { + "#operator": "Sequence", + "#stats": { + "#phaseSwitches": 2, + "execTime": "1.499µs" + }, + "~children": [ + { + "#operator": "PrimaryScan3", + "#stats": { + "#heartbeatYields": 6, + "#itemsOut": 24024, + "#phaseSwitches": 96099, + "execTime": "84.366121ms", + "kernTime": "3.021901421s", + "servTime": "69.320752ms" + }, + "bucket": "travel-sample", + "index": "def_inventory_route_primary", + "index_projection": { + "primary_key": true + }, + "keyspace": "route", + "namespace": "default", + "scope": "inventory", + "using": "gsi" + }, + { + "#operator": "Fetch", + "#stats": { + "#heartbeatYields": 7258, + "#itemsIn": 24024, + "#itemsOut": 24024, + "#phaseSwitches": 99104, + "execTime": "70.34694ms", + "kernTime": "142.630196ms", + "servTime": "3.021959695s" + }, + "bucket": "travel-sample", + "keyspace": "route", + "namespace": "default", + "scope": "inventory" + }, + { + "#operator": "InitialProject", + "#stats": { + "#itemsIn": 24024, + "#itemsOut": 24024, + "#phaseSwitches": 96100, + "execTime": "15.331951ms", + "kernTime": "3.219612458s" + }, + "result_terms": [ + { + "expr": "self", + "star": true + } + ] + }, + { + "#operator": "Order", + "#stats": { + "#itemsIn": 24024, + "#itemsOut": 24024, + "#phaseSwitches": 72078, + "execTime": "147.889352ms", + "kernTime": "3.229055752s" + }, + "sort_terms": [ + { + "expr": "(`route`.`sourceairport`)" + } + ] + }, + { + "#operator": "Stream", + "#stats": { + "#itemsIn": 24024, + "#itemsOut": 24024, + "#phaseSwitches": 24025, + "execTime": "11.851634134s" + } + } + ] + }, + "~versions": [ + "7.0.0-N1QL", + "7.0.0-4960-enterprise" + ] + } + } + ], + "status": "success", + "metrics": { + "elapsedTime": "172.831251ms", + "executionTime": "172.586836ms", + "resultCount": 40, + "resultSize": 65181, + "serviceLoad": 12 + }, + "profile": { // <3> + "phaseTimes": { + "authorize": "19.912µs", + "fetch": "123.022426ms", + "instantiate": "29.424µs", + "parse": "6.414711ms", + "plan": "3.19076ms", + "primaryScan": "55.521683ms", + "run": "158.514001ms" + }, + "phaseCounts": { + "fetch": 40, + "primaryScan": 40 + }, + "phaseOperators": { + "authorize": 1, + "fetch": 1, + "primaryScan": 1 + }, + "requestTime": "2021-04-30T21:15:18.104Z", + "servicingHost": "127.0.0.1:8091" + } +} +---- + +This example shows: + +<1> The profile attribute with all phases-related statistics for Query 2. +<2> The `meta().plan` with all detailed statistics collected for Query 1. +<3> The profile attribute with all phases-related statistics for this query itself, which is querying the `system:completed_requests` keyspace. +==== + +[#sys_my-user-info] +== Monitor Your User Info + +The `system:my_user_info` catalog maintains a list of all information of your profile. + +To see your current information, use: + +[source,sqlpp] +---- +SELECT * FROM system:my_user_info; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "my_user_info": { + "domain": "local", + "external_groups": [], + "groups": [], + "id": "jane", + "name": "Jane Doe", + "password_change_date": "2019-05-07T02:31:53.000Z", + "roles": [ + { + "origins": [ + { + "type": "user" + } + ], + "role": "admin" + } + ] + } + } +] +---- + +[#sys-user-info] +== Monitor All User Info + +The `system:user_info` catalog maintains a list of all current users in your bucket and their information. + +To see the list of all current users, use: + +[source,sqlpp] +---- +SELECT * FROM system:user_info; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "user_info": { + "domain": "local", + "external_groups": [], + "groups": [], + "id": "jane", + "name": "Jane Doe", + "password_change_date": "2019-05-07T02:31:53.000Z", + "roles": [ + { + "origins": [ + { + "type": "user" + } + ], + "role": "admin" + } + ] + } + }, + { + "user_info": { + "domain": "ns_server", + "id": "Administrator", + "name": "Administrator", + "roles": [ + { + "role": "admin" + } + ] + } + } +] +---- + +[#sys-nodes] +== Monitor Nodes + +The `system:nodes` catalog shows the datastore topology information. +This is separate from the Query clustering view, in that Query clustering shows a map of the Query cluster, as provided by the cluster manager, while `system:nodes` shows a view of the nodes and services that make up the actual datastore, which may or may not include Query nodes. + +* The dichotomy is important in that Query nodes could be clustered by one entity (e.g. Zookeeper) and be connected to a clustered datastore (e.g. Couchbase) such that each does not have visibility of the other. +* Should {sqlpp} be extended to be able to concurrently connect to multiple datastores, each datastore will report its own topology, so that `system:nodes` offers a complete view of all the storage nodes, whatever those may be. +* The `system:nodes` keyspace provides a way to report services advertised by each node as well as services that are actually running. +This is datastore dependent. +* Query clustering is still reported by the `/admin` endpoints. + +To see the list of all current node information, use: + +[source,sqlpp] +---- +SELECT * FROM system:nodes; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "nodes": { + "name": "127.0.0.1:8091", + "ports": { + "cbas": 8095, + "cbasAdmin": 9110, + "cbasCc": 9111, + "cbasSSL": 18095, + "eventingAdminPort": 8096, + "eventingSSL": 18096, + "fts": 8094, + "ftsSSL": 18094, + "indexAdmin": 9100, + "indexHttp": 9102, + "indexHttps": 19102, + "indexScan": 9101, + "indexStreamCatchup": 9104, + "indexStreamInit": 9103, + "indexStreamMaint": 9105, + "kv": 11210, + "kvSSL": 11207, + "n1ql": 8093, + "n1qlSSL": 18093 + }, + "services": [ + "cbas", + "eventing", + "fts", + "index", + "kv", + "n1ql" + ] + } + } +] +---- + +[#sys-app-roles] +== Monitor Applicable Roles + +The `system:applicable_roles` catalog maintains a list of all applicable roles and grantee of each bucket. + +To see the list of all current applicable role information, use: + +[source,sqlpp] +---- +SELECT * FROM system:applicable_roles; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "applicable_roles": { + "grantee": "anil", + "role": "replication_admin" + } + }, + { + "applicable_roles": { + "bucket_name": "travel-sample", + "grantee": "anil", + "role": "select" + } + }, + { + "applicable_roles": { + "bucket_name": "*", + "grantee": "anil", + "role": "select" + } + } +] +---- + +For more examples, take a look at the blog: https://blog.couchbase.com/optimize-n1ql-performance-using-request-profiling/[Optimize {sqlpp} performance using request profiling^]. + +[#sys-dictionary] +== Monitor Statistics + +The `system:dictionary` catalog maintains a list of the on-disk optimizer statistics stored in the `_query` collection within the `_system` scope. + +If you have multiple query nodes, the data retrieved from this catalog will be the same, regardless of the node on which you run the query. + +To see the list of on-disk optimizer statistics, use: + +[source,sqlpp] +---- +SELECT * FROM system:dictionary; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "dictionary": { + "avgDocKeySize": 12, + "avgDocSize": 278, + "bucket": "travel-sample", + "distributionKeys": [ + "airportname", + "faa", + "city" + ], + "docCount": 1968, + "indexes": [ + { + "indexId": "bc3048e87bf84828", + "indexName": "def_inventory_airport_primary", + "indexStats": [ + { + "avgItemSize": 24, + "avgPageSize": 11760, + "numItems": 1968, + "numPages": 4, + "resRatio": 1 + } + ] + }, + // ... + ], + "keyspace": "airport", + "namespace": "default", + "scope": "inventory" + } + }, + // ... +] +---- + +This catalog contains an array of dictionaries, one for each keyspace for which optimizer statistics are available. +Each dictionary gives the following information: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**avgDocKeySize** + +__required__ +|Average doc key size. +|Integer + +|**avgDocSize** + +__required__ +|Average doc size. +|Integer + +|**bucket** + +__required__ +|The bucket for which statistics are available. +|String + +|**keyspace** + +__required__ +|The keyspace for which statistics are available. +|String + +|**namespace** + +__required__ +|The namespace for which statistics are available. +|String + +|**scope** + +__required__ +|The scope for which statistics are available. +|String + +|**distributionKeys** + +__required__ +|Distribution keys for which histograms are available. +|String array + +|**docCount** + +__required__ +|Document count. +|Integer + +|**indexes** + +__required__ +|An array of indexes in this keyspace for which statistics are available. +|<> array + +|**node** + +__required__ +|The query node where this dictionary cache is resident. +|String +|=== + +[[indexes]] +**Indexes** + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**indexId** + +__required__ +|The index ID. +|String + +|**indexName** + +__required__ +|The index name. +|String + +|**indexStats** + +__required__ +|An array of statistics for each index, with one element for each index partition. +|<> array +|=== + +[[indexStats]] +**Index Statistics** + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**avgItemSize** + +__required__ +|Average item size. +|Integer + +|**avgPageSize** + +__required__ +|Average page size. +|Integer + +|**numItems** + +__required__ +|Number of items. +|Integer + +|**numPages** + +__required__ +|Number of pages. +|Integer + +|**resRatio** + +__required__ +|Resident ratio. +|Integer +|=== + +For further details, refer to xref:n1ql:n1ql-language-reference/updatestatistics.adoc[UPDATE STATISTICS]. + +[#sys-dictionary-cache] +== Monitor Cached Statistics + +The `system:dictionary_cache` catalog maintains a list of the in-memory cached subset of the optimizer statistics. + +If you have multiple query nodes, the data retrieved from this node shows cached optimizer statistics from all nodes. +Individual nodes may have a different subset of cached information. + +To see the list of in-memory optimizer statistics, use: + +[source,sqlpp] +---- +SELECT * FROM system:dictionary_cache; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "dictionary_cache": { + "avgDocKeySize": 12, + "avgDocSize": 278, + "bucket": "travel-sample", + "distributionKeys": [ + "airportname", + "faa", + "city" + ], + "docCount": 1968, + "indexes": [ + { + "indexId": "bc3048e87bf84828", + "indexName": "def_inventory_airport_primary", + "indexStats": [ + { + "avgItemSize": 24, + "avgPageSize": 11760, + "numItems": 1968, + "numPages": 4, + "resRatio": 1 + } + ] + }, + // ... + ], + "keyspace": "airport", + "namespace": "default", + "node": "172.23.0.3:8091", + "scope": "inventory" + } + }, + // ... +] +---- + +This catalog contains an array of dictionary caches, one for each keyspace for which optimizer statistics are available. +Each dictionary cache gives the same information as the <> catalog. + +For further details, refer to xref:n1ql:n1ql-language-reference/updatestatistics.adoc[UPDATE STATISTICS]. + +[#sys-functions] +== Monitor Functions + +The `system:functions` catalog maintains a list of all user-defined functions across all nodes. +To see the list of all user-defined functions, use: + +[source,sqlpp] +---- +SELECT * FROM system:functions; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "functions": { + "definition": { + "#language": "inline", + "expression": "(((`fahrenheit` - 32) * 5) / 9)", + "parameters": [ + "fahrenheit" + ], + "text": "((fahrenheit - 32) * 5/9)" + }, + "identity": { + "bucket": "travel-sample", + "name": "celsius", + "namespace": "default", + "scope": "inventory", + "type": "scope" + } + } + }, + { + "functions": { + "definition": { + "#language": "javascript", + "library": "geohash-js", + "name": "geohash-js", + "object": "calculateAdjacent", + "parameters": [ + "src", + "dir" + ] + }, + "identity": { + "name": "adjacent", + "namespace": "default", + "type": "global" + } + } + }, + // ... +] +---- + +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**definition** + +__required__ +|The definition of the function. +|<> object + +|**identity** + +__required__ +|The identity of the function. +|<> object +|=== + +[[definition]] +**Definition** + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**#language** + +__required__ +|The language of the function. + +*Example*: `inline` +|String + +|**parameters** + +__required__ +|The parameters required by the function. +|String array + +|**expression** + +__optional__ +|For inline functions only: the expression defining the function. +|String + +|**text** + +__optional__ +|For inline functions: the verbatim text of the function. + +''' + +For {sqlpp} managed user-defined functions: the external code defining the function. +|String + +|**library** + +__optional__ +|For external functions only: the library containing the function. +|String + +|**name** + +__optional__ +|For external functions only: the relative name of the library. +|String + +|**object** + +__optional__ +|For external functions only: the object defining the function. +|String +|=== + +[[identity]] +**Identity** + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**name** + +__required__ +|The name of the function. +|String + +|**namespace** + +__required__ +|The namespace of the function. + +*Example*: `default` +|String + +|**type** + +__required__ +|The type of the function. + +*Example*: `global` +|String + +|**bucket** + +__optional__ +|For scoped functions only: the bucket containing the function. +|String + +|**scope** + +__optional__ +|For scoped functions only: the scope containing the function. +|String +|=== + +[#sys-functions-cache] +== Monitor Cached Functions + +The `system:functions_cache` catalog maintains a list of recently-used user-defined functions across all nodes. +The catalog also lists user-defined functions that have been called recently, but do not exist. +To see the list of recently-used user-defined functions, use: + +[source,sqlpp] +---- +SELECT * FROM system:functions_cache; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "functions_cache": { + "#language": "inline", + "avgServiceTime": "3.066847ms", + "expression": "(((`fahrenheit` - 32) * 5) / 9)", + "lastUse": "2022-03-09 00:17:59.60659793 +0000 UTC m=+35951.429537902", + "maxServiceTime": "3.066847ms", + "minServiceTime": "0s", + "name": "celsius", + "namespace": "default", + "node": "127.0.0.1:8091", + "parameters": [ + "fahrenheit" + ], + "scope": "inventory", + "text": "((fahrenheit - 32) * 5/9)", + "type": "scope", + "uses": 1 + } + }, + { + "functions_cache": { + "#language": "javascript", + "avgServiceTime": "56.892636ms", + "lastUse": "2022-03-09 00:15:46.289934029 +0000 UTC m=+35818.007560703", + "library": "geohash-js", + "maxServiceTime": "146.025426ms", + "minServiceTime": "0s", + "name": "geohash-js", + "namespace": "default", + "node": "127.0.0.1:8091", + "object": "calculateAdjacent", + "parameters": [ + "src", + "dir" + ], + "type": "global", + "uses": 4 + } + }, + { + "functions_cache": { + "avgServiceTime": "3.057421ms", + "lastUse": "2022-03-09 00:17:25.396840275 +0000 UTC m=+35917.199008929", + "maxServiceTime": "3.057421ms", + "minServiceTime": "0s", + "name": "notFound", + "namespace": "default", + "node": "127.0.0.1:8091", + "type": "global", + "undefined_function": true, + "uses": 1 + } + } +] +---- + +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**#language** + +__required__ +|The language of the function. + +*Example*: `inline` +|String + +|**name** + +__required__ +|The name of the function. +|String + +|**namespace** + +__required__ +|The namespace of the function. + +*Example*: `default` +|String + +|**parameters** + +__required__ +|The parameters required by the function. +|String array + +|**type** + +__required__ +|The type of the function. + +*Example*: `global` +|String + +|**scope** + +__optional__ +|For scoped functions only: the scope containing the function. +|String + +|**expression** + +__optional__ +|For inline functions only: the expression defining the function. +|String + +|**text** + +__optional__ +|For inline functions: the verbatim text of the function. + +''' + +For {sqlpp} managed user-defined functions: the external code defining the function. +|String + +|**library** + +__optional__ +|For external functions only: the library containing the function. +|String + +|**object** + +__optional__ +|For external functions only: the object defining the function. +|String + +|**avgServiceTime** + +__required__ +|The mean service time for the function. +|String + +|**lastUse** + +__required__ +|The date and time when the function was last used. +|String + +|**maxServiceTime** + +__required__ +|The maximum service time for the function. +|String + +|**minServiceTime** + +__required__ +|The minimum service time for the function. +|String + +|**node** + +__required__ +|The query node where the function is cached. +|String + +|**undefined_function** + +__required__ +|Whether the function exists or is undefined. +|Boolean + +|**uses** + +__required__ +|The number of uses of the function. +|Number +|=== + +Each query node keeps its own cache of recently-used user-defined functions, so you may see the same function listed for multiple nodes. + +[#sys-tasks-cache] +== Monitor Cached Tasks + +The `system:tasks_cache` catalog maintains a list of recently-used scheduled tasks, such as index advisor sessions. +To see the list of recently-used scheduled tasks, use: + +[source,sqlpp] +---- +SELECT * FROM system:tasks_cache; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "tasks_cache": { + "class": "advisor", + "delay": "1h0m0s", + "id": "bcd9f8e4-b324-504c-a98b-ace90dba869f", + "name": "aa7f688a-bf29-438f-888f-eeaead87ca40", + "node": "10.143.192.101:8091", + "state": "scheduled", + "subClass": "analyze", + "submitTime": "2019-09-17 05:18:12.903122381 -0700 PDT m=+8460.550715992" + } + }, + { + "tasks_cache": { + "class": "advisor", + "delay": "5m0s", + "id": "254abec5-5782-543e-9ee0-d07da146b94e", + "name": "ca2cfe56-01fa-4563-8eb0-a753af76d865", + "node": "10.143.192.101:8091", + "results": [ + // ... + ], + "startTime": "2019-09-17 05:03:31.821597725 -0700 PDT m=+7579.469191487", + "state": "completed", + "stopTime": "2019-09-17 05:03:31.963133954 -0700 PDT m=+7579.610727539", + "subClass": "analyze", + "submitTime": "2019-09-17 04:58:31.821230131 -0700 PDT m=+7279.468823737" + } + } +] +---- + +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**class** + +__required__ +|The class of the task. + +*Example*: ``advisor`` +|string + +|**delay** + +__required__ +|The scheduled duration of the task. +|string + +|**id** + +__required__ +|The internal ID of the task. +|string + +|**name** + +__required__ +|The name of the task. +|string + +|**node** + +__required__ +|The node where the task was started. +|string + +|**state** + +__required__ +|The state of the task. + +*Values*: `scheduled`, `cancelled`, `completed` +|string + +|**subClass** + +__required__ +|The subclass of the task. + +*Example*: `analyze` +|string + +|**submitTime** + +__required__ +|The date and time when the task was submitted. +|string + +|**results** + +__optional__ +|Not scheduled tasks: the results of the task. +|Any array + +|**startTime** + +__optional__ +|Not scheduled tasks: the date and time when the task started. +|string (date-time) + +|**stopTime** + +__optional__ +|Not scheduled tasks: the date and time when the task stopped. +|string (date-time) +|=== + +Refer to xref:n1ql:n1ql-language-reference/advisor.adoc[ADVISOR Function] for more information on index advisor sessions. + +[#sys-transactions] +== Monitor Transactions + +The `system:transactions` catalog maintains a list of active Couchbase transactions. +To see the list of active transactions, use: + +[source,sqlpp] +---- +SELECT * FROM system:transactions; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "transactions": { + "durabilityLevel": "majority", + "durabilityTimeout": "2.5s", + "expiryTime": "2021-04-21T12:53:48.598+01:00", + "id": "85aea637-2288-434b-b7c5-413ad8e7c175", + "isolationLevel": "READ COMMITED", + "lastUse": "2021-04-21T12:51:48.598+01:00", + "node": "127.0.0.1:8091", + "numAtrs": 1024, + "scanConsistency": "unbounded", + "status": 0, + "timeout": "2m0s", + "usedMemory": 960, + "uses": 1 + } + // ... + } +] +---- + +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**durabilityLevel** + +__required__ +|Durability level for all mutations within a transaction. +|string + +|**durabilityTimeout** + +__required__ +|Durability timeout per mutation within the transaction. +|string (duration) + +|**expiryTime** + +__required__ +| +|string (date-time) + +|**id** + +__required__ +|The transaction ID. +|string + +|**isolationLevel** + +__required__ +|The isolation level of the transaction. +|string + +|**lastUse** + +__required__ +| +|string (date-time) + +|**node** + +__required__ +|The node where the transaction was started. +|string + +|**numAtrs** + +__required__ +|The total number of active transaction records. +|integer + +|**scanConsistency** + +__required__ +|The transactional scan consistency. +|string + +|**status** + +__required__ +| +|integer + +|**timeout** + +__required__ +|The transaction timeout duration. +|string (duration) + +|**usedMemory** + +__required__ +| +|integer + +|**uses** + +__required__ +| +|integer +|=== + +Refer to xref:n1ql:n1ql-language-reference/transactions.adoc[{sqlpp} Support for Couchbase Transactions] for more information. + +== Related Links + +* Refer to xref:n1ql:n1ql-intro/sysinfo.adoc[Getting System Information] for more information on the system namespace. diff --git a/modules/n1ql/partials/nav.adoc b/modules/n1ql/partials/nav.adoc index 5d567a2ef..59fdefe17 100644 --- a/modules/n1ql/partials/nav.adoc +++ b/modules/n1ql/partials/nav.adoc @@ -32,6 +32,7 @@ endif::flag-devex-tools[] *** xref:n1ql:n1ql-language-reference/time-series.adoc[] ** xref:n1ql:n1ql-manage/index.adoc[] *** xref:n1ql:n1ql-intro/sysinfo.adoc[] + *** xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc[] *** xref:clusters:index-service/manage-indexes.adoc[] *** xref:n1ql:n1ql-manage/query-settings.adoc[] ** xref:n1ql:n1ql-language-reference/index.adoc[] From aa9fb9fad94511f7d0546cab145b8318448ed988 Mon Sep 17 00:00:00 2001 From: Simon Dew Date: Tue, 28 May 2024 18:26:09 +0100 Subject: [PATCH 2/4] DOC-11904: Document system:sequences, all_sequences & vitals keyspaces (#221) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add system:sequences and system:all_sequences * Add system:vitals * Replace profiling explanation with includes from REST API * Tabs for requests, prepared statements * Outsource prepared statement fields to the REST API docs * Delete Monitor Query Clusters section: marketing bumf that says nothing * Rework sections on query profiling * Add sequence examples * Move Configure the Completed Requests as a top-level section * Move query profiling after manage and monitor sections * Query engine → query node * Reformat ToC * Update system:active_requests * Update system:prepareds * Update system:completed_requests * Update query plan in system catalogs * Include query plan field names and meanings; also, refer to → see * Include profile fields * Link to logging * Add feature flag for REST API * Add feature flag for cbq shell --- .../examples/settings/node-level-settings.sh | 2 +- .../settings/save-node-level-settings.sh | 2 +- .../n1ql-manage/monitoring-n1ql-query.adoc | 2706 ++++++++--------- 3 files changed, 1330 insertions(+), 1380 deletions(-) diff --git a/modules/n1ql/examples/settings/node-level-settings.sh b/modules/n1ql/examples/settings/node-level-settings.sh index c582ed1bd..2a42a58df 100644 --- a/modules/n1ql/examples/settings/node-level-settings.sh +++ b/modules/n1ql/examples/settings/node-level-settings.sh @@ -1,5 +1,5 @@ #!/bin/sh # tag::curl[] -curl http://localhost:8093/admin/settings -u user:pword +curl $BASE_URL/admin/settings -u $USER:$PASSWORD # end::curl[] \ No newline at end of file diff --git a/modules/n1ql/examples/settings/save-node-level-settings.sh b/modules/n1ql/examples/settings/save-node-level-settings.sh index ddbac3034..6d83ed494 100644 --- a/modules/n1ql/examples/settings/save-node-level-settings.sh +++ b/modules/n1ql/examples/settings/save-node-level-settings.sh @@ -1,5 +1,5 @@ #!/bin/sh # tag::curl[] -curl http://localhost:8093/admin/settings -u user:pword -o ./query_settings.json +curl $BASE_URL/admin/settings -u $USER:$PASSWORD -o ./query_settings.json # end::curl[] \ No newline at end of file diff --git a/modules/n1ql/pages/n1ql-manage/monitoring-n1ql-query.adoc b/modules/n1ql/pages/n1ql-manage/monitoring-n1ql-query.adoc index 1fcf85c63..309e3ff6e 100644 --- a/modules/n1ql/pages/n1ql-manage/monitoring-n1ql-query.adoc +++ b/modules/n1ql/pages/n1ql-manage/monitoring-n1ql-query.adoc @@ -1,6 +1,6 @@ = Manage and Monitor Queries -:description: Monitoring and profiling {sqlpp} queries, query service engines, and corresponding system resources is very important for smoother operational performance and efficiency of the system. -:page-aliases: monitoring:monitoring-n1ql-query, manage:monitor/monitoring-n1ql-query +:example-caption!: +:description: Monitoring and profiling {sqlpp} queries, query service nodes, and corresponding system resources is very important for smoother operational performance and efficiency of the system. [abstract] {description} @@ -13,18 +13,22 @@ When running on a cluster with multiple query nodes, stats about all queries on For example, this can help identify: -* The top 10 slow or fast queries running on a particular query engine or the cluster. +* The top 10 slow or fast queries running on a particular query node or the cluster. * Resource usage statistics of the query service, or resources used for a particular query. * Details about the active, completed, and prepared queries. * Find long running queries that are running for more than 2 minutes. These system keyspaces are like virtual keyspaces that are transient in nature, and are not persisted to disk or permanent storage. Hence, the information in the keyspaces pertains to the current instantiation of the query service. + +//// +TODO: Reinstate this for monitoring and managing when other content is moved to the System Information page You can access the keyspaces using any of the following: * {sqlpp} language (from the cbq shell or Query Workbench) * REST API * Monitoring SDK +//// NOTE: All the power of the {sqlpp} query language can be applied on the keyspaces to obtain various insights. @@ -33,48 +37,47 @@ The following diagnostics are provided: [cols="1,3"] |=== | System Catalogs -a| -* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-datastores[system:datastores] -* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-namespaces[system:namespaces] -* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-buckets[system:buckets] -* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-scopes[system:scopes] -* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-keyspaces[system:keyspaces] -* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-indexes[system:indexes] -* xref:n1ql:n1ql-intro/sysinfo.adoc#querying-dual[system:dual] +a| [%hardbreaks] +xref:n1ql:n1ql-intro/sysinfo.adoc#querying-datastores[system:datastores] +xref:n1ql:n1ql-intro/sysinfo.adoc#querying-namespaces[system:namespaces] +xref:n1ql:n1ql-intro/sysinfo.adoc#querying-buckets[system:buckets] +xref:n1ql:n1ql-intro/sysinfo.adoc#querying-scopes[system:scopes] +xref:n1ql:n1ql-intro/sysinfo.adoc#querying-keyspaces[system:keyspaces] +xref:n1ql:n1ql-intro/sysinfo.adoc#querying-indexes[system:indexes] +xref:n1ql:n1ql-intro/sysinfo.adoc#querying-dual[system:dual] | Monitoring Catalogs -a| -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> +a| [%hardbreaks] +<> +<> +<> +<> | Security Catalogs -a| -* <> -* <> -* <> -* <> +a| [%hardbreaks] +<> +<> +<> +<> | Other -a| -* <> - -NOTE: These are only available using REST APIs. - +a| [%hardbreaks] +<> +<> +<> +<> +<> +<> +<> +<> |=== == Authentication and Client Privileges Client applications must be authenticated with sufficient privileges to access system keyspaces. -* Users must have the *Query System Catalog* role to access restricted system keyspaces. -For more details about user roles, see xref:learn:security/authorization-overview.adoc[Authorization]. +* Users must have permission to access restricted system keyspaces. +For more details about database credentials, see xref:clusters:manage-database-users.adoc[]. * In addition, users must have permission to access a bucket, scope, or collection to be able to view that item in the system catalog. Similarly, users must have SELECT permission on the target of an index to be able to view that index in the system catalog. @@ -84,1663 +87,1420 @@ Similarly, users must have SELECT permission on the target of an index to be abl ** `system:datastores` ** `system:namespaces` -== Query Monitoring Settings - -The monitoring settings can be set for each query engine (using the REST API) or for each query statement (using the cbq command line tool). -Both settings are actually set via REST endpoints: using the xref:n1ql:n1ql-rest-api/admin.adoc[Admin REST API] (`/admin/settings` endpoint) and the xref:n1ql:n1ql-rest-api/index.adoc[Query REST API] (`/query/service` endpoint). +ifdef::flag-devex-rest-api[] +== Examples on this Page -The cbq shell and Query Workbench use the Query REST API to set monitoring at the request level. -The Query Workbench automatically enables the profiling and timings. -It can be disabled using the [.ui]*Preferences* option. -For more information refer to the xref:tools:query-workbench.adoc[Query Workbench] section. +In the REST API examples: -Use the following query parameters to enable, disable, and control the monitoring capabilities, and the level of monitoring and profiling details for each query or globally at a query engine level: +* `$BASE_URL` is the protocol, host name or IP address, and port -- for example, `+http://localhost:8093+`. +* `$USER` is the user name. +* `$PASSWORD` is the password. +endif::flag-devex-rest-api[] -* profile -* controls +[#vitals] +== Monitor System Vitals -For more details and examples, see xref:n1ql:n1ql-manage/query-settings.adoc[]. +The `system:vitals` catalog provides data about the running state and health of the query node, such as number of logical cores, active threads, queued threads, CPU utilization, memory usage, network utilization, garbage collection percentage, and so on. +This information can be very useful to assess the current workload and performance characteristics of a query node. -=== Enable Settings for a Query Engine +[#sys-vitals-get] +=== Get System Vitals -You can enable profile settings for each query engine. -These examples use local host IP address and default port numbers. -You need to provide correct credentials, IP address, and port details of your setup. +To view system vitals, use +ifdef::flag-devex-rest-api[] +the Admin REST API or +endif::flag-devex-rest-api[] +a {sqlpp} query. -. Get the current query settings: +[tabs] +==== +ifdef::flag-devex-rest-api[] +REST API:: + +-- +To view system vitals with the Admin REST API: + [source,sh] ---- -include::n1ql:example$settings/save-node-level-settings.sh[tag=curl] +curl -u $USER:$PASSWORD $BASE_URL/admin/vitals ---- +-- +endif::flag-devex-rest-api[] + +{sqlpp}:: + -[source,sh] +-- +To view system vitals with {sqlpp}: + +[source,sqlpp] ---- -cat ./query_settings.json +SELECT * FROM system:vitals; ---- -+ +-- +==== + +[[sys-vital-examples]] +=== System Vitals Details + +Getting system vitals, as described in <>, returns results similar to the following. + +==== [source,json] ---- -include::n1ql:example$settings/node-level-settings.jsonc[] +{ + "uptime": "7h39m32.668577197s", + "local.time": "2021-04-30 18:42:39.517208807 +0000 UTC m=+27573.945319668", + "version": "7.0.0-N1QL", + "total.threads": 191, + "cores": 2, + "gc.num": 669810600, + "gc.pause.time": "57.586373ms", + "gc.pause.percent": 0, + "memory.usage": 247985184, + "memory.total": 11132383704, + "memory.system": 495554808, + "cpu.user.percent": 0, + "cpu.sys.percent": 0, + "request.completed.count": 140, + "request.active.count": 0, + "request.per.sec.1min": 0.0018, + "request.per.sec.5min": 0.0055, + "request.per.sec.15min": 0.0033, + "request_time.mean": "536.348163ms", + "request_time.median": "54.065567ms", + "request_time.80percentile": "981.869933ms", + "request_time.95percentile": "2.543128455s", + "request_time.99percentile": "4.627922799s", + "request.prepared.percent": 0 +} ---- +==== + +This catalog contains the following attributes: + +include::n1ql:partial$n1ql-rest-api/admin/definitions.adoc[tags=vitals] + +[#sys-active-req] +== Monitor and Manage Active Requests + +The `system:active_requests` catalog lists all currently executing active requests or queries. + +[[sys-active-get]] +=== Get Active Requests -. Set current query settings profile: - .. To set the query settings saved in a file [.path]_./query_settings.json_, enter the following query: +To view active requests, use +ifdef::flag-devex-rest-api[] +the Admin REST API or +endif::flag-devex-rest-api[] +a {sqlpp} query. + +[tabs] +==== +ifdef::flag-devex-rest-api[] +REST API:: + +-- +To view active requests with the Admin REST API: + [source,sh] ---- -curl http://localhost:8093/admin/settings -u user:pword \ - -X POST \ - -d@./query_settings.json +curl -u $USER:$PASSWORD $BASE_URL/admin/active_requests ---- +-- +endif::flag-devex-rest-api[] - .. To explicitly specify the settings, enter the following query: +{sqlpp}:: + -[source,sh] +-- +To view active requests with {sqlpp}: + +[source,sqlpp] ---- -curl http://localhost:8093/admin/settings -u user:pword \ - -H 'Content-Type: application/json' \ - -d '{"profile": "phases"}' +SELECT * FROM system:active_requests; ---- +-- +==== + +To get the query plan for active requests, include `meta().plan` in a {sqlpp} query. +See <>. -. Verify the settings are changed as specified: +[tabs] +==== +{sqlpp}:: + -[source,sh] +-- +To view active requests with {sqlpp}, including the query plan: + +[source,sqlpp] ---- -include::n1ql:example$settings/node-level-settings.sh[tag=curl] +SELECT *, meta().plan FROM system:active_requests; ---- +-- +==== + +[[sys-active-delete]] +=== Terminate an Active Request + +The DELETE command can be used to terminate an active request, for instance, a non-responding or a long-running query. + +[tabs] +==== +ifdef::flag-devex-rest-api[] +REST API:: + -[source,json] +-- +To terminate an active request [.var]`uuid` with the Admin REST API: + +[source,sh] ---- -{ - // ... - "profile":"phases", - "request-size-cap": 67108864, - "scan-cap": 512, - "servicers": 4, - "timeout": 0, - "txtimeout": "0s", - "use-cbo": true -} +curl -u $USER:$PASSWORD -X DELETE $BASE_URL/admin/active_requests/uuid ---- +-- +endif::flag-devex-rest-api[] -=== Enable Settings per Session or per Query - -You can enable monitoring and profiling settings for each query statement. -To set query settings using the cbq shell, use the `\SET` command: +{sqlpp}:: ++ +-- +To terminate an active request [.var]`uuid` with {sqlpp}: [source,sqlpp] ---- -\set -profile "timings"; -\set; +DELETE FROM system:active_requests WHERE requestId = "uuid"; ---- +-- +==== -[source,none] ----- - Query Parameters : - Parameter name : profile - Value : ["timings"] - ... ----- +[[sys-active-examples]] +=== Active Request Details -To set query settings using the REST API, specify the parameters in the request body: +Getting active requests, as described in <>, returns results similar to the following. -[source,sh] +==== +[source,json] ---- -curl http://localhost:8093/query/service -u user:pword \ - -d 'profile=timings&statement=SELECT * FROM "world" AS hello' +[ + { + "active_requests": { + "clientContextID": "832adfa0-e9e6-464e-b5e6-b7ec7549d511", + "cpuTime": "111.877µs", + "elapsedTime": "77.631814ms", + "executionTime": "77.517185ms", + "n1qlFeatCtrl": 76, + "node": "127.0.0.1:8091", + "phaseOperators": { + "authorize": 1, + "fetch": 1, + "primaryScan": 1, + "project": 1, + "stream": 1 + }, + "phaseTimes": { + "authorize": "4.998µs", + "fetch": "69.519µs", + "instantiate": "16.28µs", + "parse": "597.435µs", + "plan": "24.141851ms", + "plan.index.metadata": "24.005473ms", + "plan.keyspace.metadata": "2.022µs", + "primaryScan": "23.496033ms", + "project": "824ns", + "stream": "2.242µs" + }, + "queryContext": "default:travel-sample.inventory", + "remoteAddr": "127.0.0.1:37506", + "requestId": "05cc1895-9986-4819-b4d3-8a4311e8f319", + "requestTime": "2024-05-21T13:29:16.864Z", + "scanConsistency": "unbounded", + "state": "running", + "statement": "SELECT * FROM system:active_requests;", + "statementType": "SELECT", + "useCBO": true, + "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:126.0) Gecko/20100101 Firefox/126.0", + "users": "builtin:Administrator" + } + } +] ---- +==== -[#monitor-profile-details] -== Monitoring and Profiling Details +This catalog contains the following attributes: -Couchbase Server provides detailed query monitoring and profiling information. -The profiling and finer query execution timing details can be obtained for any query. +include::n1ql:partial$n1ql-rest-api/admin/definitions.adoc[tags=requests] -When a query executes a user-defined function, profiling information is available for the {sqlpp} queries within the user-defined function as well. +For query plan field names and meanings, see <>. -[[profile]] -=== Attribute Profile in Query Response +[#sys-prepared] +== Monitor and Manage Prepared Statements -When profiling is enabled, a query response includes the profile attribute. -The attribute details are as follows: +The `system:prepareds` catalog provides data about the known prepared statements and their state in a query node's prepared statement cache. +For each prepared statement, this catalog provides information such as name, statement, query plan, last use time, number of uses, and so on. -.Attribute Details -[cols="14,8"] -|=== -| Attribute | Example +A prepared statement is created and stored relative to the current xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context[query context]. +You can create multiple prepared statements with the same name, each stored relative to a different query context. +This enables you to run multiple instances of the same application against different datasets. -a| -`phaseTimes` -- Cumulative execution times for various phases involved in the query execution, such as authorize, indexscan, fetch, parse, plan, run, etc. +When there are multiple prepared statements with the same name in different query contexts, the name of the prepared statement in the `system:prepareds` catalog includes the associated query context in brackets. -[NOTE] -==== -This value will be dynamic, depending on the documents processed by various phases up to this moment in time. +[[sys-prepared-get]] +=== Get Prepared Statements + +To get a list of all known prepared statements, use +ifdef::flag-devex-rest-api[] +the Admin REST API or +endif::flag-devex-rest-api[] +a {sqlpp} query. -A new query on `system:active_requests` will return different values. +[tabs] ==== -a| -[source,json] +ifdef::flag-devex-rest-api[] +REST API:: ++ +-- +To get a list of all known prepared statements with the Admin REST API: + +[source,sh] ---- -"phaseTimes": { - "authorize": "823.631µs", - "fetch": "656.873µs", - "indexScan": "29.146543ms", - "instantiate": "236.221µs", - "parse": "826.382µs", - "plan": "11.831101ms", - "run": "16.892181ms" -} +curl -u $USER:$PASSWORD $BASE_URL/admin/prepareds ---- +-- +endif::flag-devex-rest-api[] -a| -`phaseCounts` -- Count of documents processed at selective phases involved in the query execution, such as authorize, indexscan, fetch, parse, plan, run, etc. +{sqlpp}:: ++ +-- +To get a list of all known prepared statements with a {sqlpp} query: -[NOTE] +[source,sqlpp] +---- +SELECT * FROM system:prepareds; +---- +-- ==== -This value will be dynamic, depending on the documents processed by various phases up to this moment in time. -A new query on `system:active_requests` will return different values. +To get information about a specific prepared statement, use +ifdef::flag-devex-rest-api[] +the Admin REST API or +endif::flag-devex-rest-api[] +a {sqlpp} query. + +[tabs] ==== -a| -[source,json] +ifdef::flag-devex-rest-api[] +REST API:: ++ +-- +To get information about a specific prepared statement [.var]`example1` with the Admin REST API: + +[source,sh] ---- -"phaseCounts": { - "fetch": 16, - "indexScan": 187 -} +curl -u $USER:$PASSWORD $BASE_URL/admin/prepareds/example1 ---- +-- +endif::flag-devex-rest-api[] -a| -`phaseOperators` -- Indicates the number of each kind of query operators involved in different phases of the query processing. -For instance, this example, one non covering index path was taken, which involves 1 indexScan and 1 fetch operator. +{sqlpp}:: ++ +-- +To get information about a specific prepared statement [.var]`example1` with a {sqlpp} query: -A join would have probably involved 2 fetches (1 per keyspace) +[source,sqlpp] +---- +SELECT * FROM system:prepareds WHERE name = "example1"; +---- +-- +==== -A union select would have twice as many operator counts (1 per each branch of the union). +To get the query plan for prepared statements, include `meta().plan` in a {sqlpp} query. +See <>. -This is in essence the count of all the operators in the `executionTimings` field. -a| -[source,json] +[tabs] +==== +{sqlpp}:: ++ +-- +To view prepared statements with {sqlpp}, including the query plan: + +[source,sqlpp] ---- -"phaseOperators": { - "authorize": 1, - "fetch": 1, - "indexScan": 2 -} +SELECT *, meta().plan FROM system:prepareds; ---- +-- +==== -a| -`executionTimings` -- The execution details such as kernel and service execution times, number of documents processed at each query operator in each phase, and number of phase switches, for various phases involved in the query execution. - -The following statistics are collected for each operator: +[[sys-prepared-delete]] +=== Delete Prepared Statements -`#operator`:: -Name of the operator. +To delete a specific prepared statement, use +ifdef::flag-devex-rest-api[] +the Admin REST API or +endif::flag-devex-rest-api[] +a {sqlpp} query. -`#stats`:: -These values will be dynamic, depending on the documents processed by various phases up to this moment in time. +[tabs] +==== +ifdef::flag-devex-rest-api[] +REST API:: + -A new query on `system:active_requests` will return different values. - -`#itemsIn`;; -Number of input documents to the operator. +-- +To delete a prepared statement [.var]`p1` with the Admin REST API: -`#itemsOut`;; -Number of output documents after the operator processing. +[source,sh] +---- +curl -u $USER:$PASSWORD -X DELETE $BASE_URL/admin/prepareds/p1 +---- +-- +endif::flag-devex-rest-api[] -`#phaseSwitches`;; -Number of switches between executing, waiting for services, or waiting for the `goroutine` scheduler. +{sqlpp}:: ++ +-- +To delete a prepared statement [.var]`p1` with a {sqlpp} query: -`execTime`;; -Time spent executing the operator code inside {sqlpp} query engine. +[source,sqlpp] +---- +DELETE FROM system:prepareds WHERE name = "p1"; +---- +-- +==== -`kernTime`;; -Time spent waiting to be scheduled for CPU time. +To delete all the known prepared statements, use a {sqlpp} query. -`servTime`;; -Time spent waiting for another service, such as index or data. +[tabs] +==== +{sqlpp}:: + -* For index scan, it is time spent waiting for GSI/indexer. -* For fetch, it is time spent waiting on the KV store. +-- +To delete all known prepared statements: -a| -[source,json] +[source,sqlpp] ---- -"executionTimings": { - // … - [{ - "#operator": "Fetch", - "#stats": { - "#itemsIn": 187, - "#itemsOut": 16, - "#phaseSwitches": 413, - "execTime": "128.434µs", - "kernTime": "15.027879ms", - "servTime": "1.590934ms", - "state": "services" - }, - "keyspace": "travel-sample", - "namespace": "default" - }, - { - "#operator": "IntersectScan", - "#stats": { - "#itemsIn": 187, - "#itemsOut": 187, - "#phaseSwitches": 749, - "execTime": "449.944µs", - "kernTime": "14.625524ms" - } - // … - ]} +DELETE FROM system:prepareds; ---- +-- +==== -|=== - -These statistics (`kernTime`, `servTime`, and `execTime`) can be very helpful in troubleshooting query performance issues, such as: +[[sys-prepared-examples]] +=== Prepared Statement Details -* A high `servTime` for a low number of items processed is an indication that the indexer or KV store is stressed. -* A high `kernTime` means there is a downstream issue in the query plan or the query server having many requests to process (so the scheduled waiting time will be more for CPU time). +To try the examples in this section, first create a couple of prepared statements. -.Phases Profile +.Create a prepared statement with default query context ==== -The cbq engine must be started with authorization, for example: +include::ROOT:partial$query-context.adoc[tag=unset] -[source,sh] +.Query +[source,sqlpp] ---- -./cbq -engine=http://localhost:8091/ -u Administrator -p pword +PREPARE p1 AS SELECT * FROM `travel-sample`.inventory.airline WHERE iata = "U2"; ---- +==== -Using the cbq shell, show the statistics collected when the `profile` is set to `phases`: +.Create a prepared statement with specified query context +==== +include::ROOT:partial$query-context.adoc[tag=example] +.Query [source,sqlpp] ---- -\set -profile "phases"; -SELECT * FROM `travel-sample`.inventory.airline LIMIT 1; +PREPARE p1 AS SELECT * FROM airline WHERE iata = "U2"; ---- +==== + +Getting prepared statements, as described in <>, returns results similar to the following. +==== [source,json] ---- { - "requestID": "06d6c1c2-1a8a-4989-a856-7314f9eddee5", + "requestID": "d976e59a-d74e-4350-b0df-fa137099d594", "signature": { - "*": "*" + "*": "*", + "plan": "json" }, "results": [ { - "airline": { - "callsign": "MILE-AIR", - "country": "United States", - "iata": "Q5", - "icao": "MLA", - "id": 10, - "name": "40-Mile Air", - "type": "airline" + "prepareds": { + "encoded_plan": "H4sIAAAAAAAA/6RTUW/TPBT9K9H5XrbJ30QBMcmIhzJ1AjG0qh3wwKbEJLedmWt71061UIXfjpxkndYhENpbYt97z7nn+GxAtnQVVbk3ykICAgtSsWY6djayMwHy6JWAthXdjr3+TBy0s5Avh7N5qewHaoJXJQXIDSpaqNpEGVmtyfwf1MobOtR2TTY6bg6VZqMtQS6UCdQKWLUiSPgR+u9uFOTdIAg4T6yi4zT+v/sfjOt45Vj/IAh41mttaNmTONUhQn7d4FzxkuL9tL/SEpiyXkMepQ/nA+Sz9rIV+FleaVPtMpjTTU22TG19AZPtcP+5aMp6pbhJcr6AwLe6vO54P+CLQfR+n3zLPh/Y576fcleXe3bfqYydYxsMt/k1NZCR66T+9eAdJO4l+L0NoXQ+nWxhIVAHbZeQWAaNVjxc6YRiefWnXZ6EvYs2VayMIYMneXWiTSSGQOlspXvhsLdXDPyKw0KrqIr97E12gU/PL7D/iMh7q6NWZtpLDwGmUJuYR+JV6ADp1qfCQGaRVouKBzsu28s2vbYd4pFJrZDuBFJOtyE8Go0EbmriJqWVbmOfYKab86aTaz45nRyfJxC9tF2skyoHkDhAKzC0TGeT6Xg2yfwoG8+zvic7yE5mZx+z4oFpxePEZF/eTWaTLMmyFeV19zLo+O3ZsNivAAAA//+q+jhuaAQAAA==", + "featuresControl": 76, + "indexApiVersion": 4, + "indexScanKeyspaces": { + "default:travel-sample.inventory.airline": false + }, + "name": "p1", // <1> + "namespace": "default", + "node": "127.0.0.1:8091", + "statement": "PREPARE p1 AS SELECT * FROM `travel-sample`.inventory.airline WHERE iata = \"U2\";", + "uses": 0 } - } - ], - "status": "success", - "metrics": { - "elapsedTime": "12.77927ms", - "executionTime": "12.570648ms", - "resultCount": 1, - "resultSize": 254, - "serviceLoad": 12 - }, - "profile": { - "phaseTimes": { - "authorize": "19.629µs", - "fetch": "401.997µs", - "instantiate": "147.686µs", - "parse": "4.545234ms", - "plan": "409.364µs", - "primaryScan": "6.103775ms", - "run": "6.699056ms" - }, - "phaseCounts": { - "fetch": 1, - "primaryScan": 1 - }, - "phaseOperators": { - "authorize": 1, - "fetch": 1, - "primaryScan": 1 }, - "requestTime": "2021-04-30T18:37:56.394Z", - "servicingHost": "127.0.0.1:8091" - } -} ----- -==== - -.Timings Profile -==== -Using the cbq shell, show the statistics collected when `profile` is set to `timings`: - -[source,sqlpp] ----- -\set -profile "timings"; -SELECT * FROM `travel-sample`.inventory.airline LIMIT 1; ----- - -[source,json] ----- -{ - "requestID": "268a1240-6864-43a2-af13-ccb8d1e50abf", - "signature": { - "*": "*" - }, - "results": [ { - "airline": { - "callsign": "MILE-AIR", - "country": "United States", - "iata": "Q5", - "icao": "MLA", - "id": 10, - "name": "40-Mile Air", - "type": "airline" + "prepareds": { + "encoded_plan": "H4sIAAAAAAAA/6STT28TMRDFv8rqcWkrExFAVDLiEKpUIIoaJQUOtNqY3Ulq6tju2Bt1iZbPjry7TWmKQKg3/xnPvPk9zwZkC1dSmXujLCQgsCAVK6YjZyM7EyAPXwloW9LNyOvPxEE7C/myP5sVyn6gOnhVUIDcoKSFqkyUkdWazNOgVt7QQNs12ei4HijNRluCXCgTqBGwakWQ8EN06zYV5G0iCDhPrKLjlP7J3QajKl461j8IAp71WhtadiJOdIiQXzc4U7ykeJftn7IEJqzXkIdp4XyAfNZcNAI/i0ttyl0FM7quyBbpWRfAZNu6/x00Yb1SXCecLyDwrSquWt339KKH3vWTb9Xnvfrcd1lu43LP7jsVsXVsg/42v6IaMnKV6F/13kHiDsGfbQiF8+lkWxYCVdB2CYll0GjE/ZaOKRaXf+vlUbV3q00UK2PI4FFeHWsTiSFQOFvqDhz29ua9vvlgrlVU8/3sTXaOT8/Psf9AyHuro1Zm0qGHAFOoTMwj8Sq0BenGp8BAZpFai4p7Oy6aiyb9th3hkUmtkO4E0pxuh/BwOBS4rojrNK108wDy4HevmK7P6pbibHwyPjpLtfXSttOeYB1A4gCNQJ9pMh1PRtNx5ofZaJZ1b7KD7Hh6+jHreWRf3o2n4ywx2RJ53X4LOnp72nf1KwAA////9+bsZQQAAA==", + "featuresControl": 76, + "indexApiVersion": 4, + "indexScanKeyspaces": { + "default:travel-sample.inventory.airline": false + }, + "name": "p1(travel-sample.inventory)", // <2> + "namespace": "default", + "node": "127.0.0.1:8091", + "statement": "PREPARE p1 AS SELECT * FROM airline WHERE iata = \"U2\";", + "uses": 0 } } ], "status": "success", "metrics": { - "elapsedTime": "2.915245ms", - "executionTime": "2.755355ms", - "resultCount": 1, - "resultSize": 254, + "elapsedTime": "25.323496ms", + "executionTime": "25.173646ms", + "resultCount": 2, + "resultSize": 7891, "serviceLoad": 12 - }, - "profile": { - "phaseTimes": { - "authorize": "18.096µs", - "fetch": "388.122µs", - "instantiate": "31.702µs", - "parse": "646.157µs", - "plan": "120.427µs", - "primaryScan": "1.402918ms", - "run": "1.936852ms" - }, - "phaseCounts": { - "fetch": 1, - "primaryScan": 1 - }, - "phaseOperators": { - "authorize": 1, - "fetch": 1, - "primaryScan": 1 - }, - "requestTime": "2021-04-30T18:40:13.239Z", - "servicingHost": "127.0.0.1:8091", - "executionTimings": { - "#operator": "Authorize", - "#stats": { - "#phaseSwitches": 4, - "execTime": "1.084µs", - "servTime": "17.012µs" - }, - "privileges": { - "List": [ - { - "Target": "default:travel-sample.inventory.airline", - "Priv": 7, - "Props": 0 - } - ] - }, - "~child": { - "#operator": "Sequence", - "#stats": { - "#phaseSwitches": 1, - "execTime": "2.474µs" - }, - "~children": [ - { - "#operator": "PrimaryScan3", - "#stats": { - "#itemsOut": 1, - "#phaseSwitches": 7, - "execTime": "18.584µs", - "kernTime": "8.869µs", - "servTime": "1.384334ms" - }, - "bucket": "travel-sample", - "index": "def_inventory_airline_primary", - "index_projection": { - "primary_key": true - }, - "keyspace": "airline", - "limit": "1", - "namespace": "default", - "scope": "inventory", - "using": "gsi" - }, - { - "#operator": "Fetch", - "#stats": { - "#itemsIn": 1, - "#itemsOut": 1, - "#phaseSwitches": 10, - "execTime": "25.64µs", - "kernTime": "1.427752ms", - "servTime": "362.482µs" - }, - "bucket": "travel-sample", - "keyspace": "airline", - "namespace": "default", - "scope": "inventory" - }, - { - "#operator": "InitialProject", - "#stats": { - "#itemsIn": 1, - "#itemsOut": 1, - "#phaseSwitches": 9, - "execTime": "6.006µs", - "kernTime": "1.825917ms" - }, - "result_terms": [ - { - "expr": "self", - "star": true - } - ] - }, - { - "#operator": "Limit", - "#stats": { - "#itemsIn": 1, - "#itemsOut": 1, - "#phaseSwitches": 4, - "execTime": "2.409µs", - "kernTime": "2.094µs" - }, - "expr": "1" - }, - { - "#operator": "Stream", - "#stats": { - "#itemsIn": 1, - "#itemsOut": 1, - "#phaseSwitches": 6, - "execTime": "46.964µs", - "kernTime": "1.844828ms" - } - } - ] - }, - "~versions": [ - "7.0.0-N1QL", - "7.0.0-4960-enterprise" - ] - } } } ---- -==== - -[[plan]] -=== Attribute Meta in System Keyspaces - -The `meta().plan` virtual attribute captures the whole query plan, and monitoring stats of various phases and involved query operators. -The `meta().plan` must be explicitly called in the SELECT query projection list. -The `meta().plan` attribute is enabled only for individual requests that are running (`active_requests`) or completed (`completed_requests`) when the profile is set to timings (`profile ="timings"`) for each individual request. -If at the engine level, the profile is set to off and individual requests have been run with `profile ="timings"`, then the system keyspaces will return the plan only for those requests. +In this example, the names of the prepared statements are identical, but they are associated with different query contexts. -Since there may be a combination of profile settings for all of the requests reported by the system keyspaces, not all requests returned will have a `meta().plan` attachment. - -NOTE: For the `system:prepareds` requests, the `meta().plan` is available at all times since the `PREPARE` statement is not dependant on the profile setting. - -This attribute is enabled for the following system keyspaces: +<.> The name of the prepared statement for the default query context +<.> The name of the prepared statement showing the associated query context +==== -* <> -* <> -* <> +This catalog contains the following attributes: -For a detailed example, see <>. +include::n1ql:partial$n1ql-rest-api/admin/definitions.adoc[tags=statements] -== Monitor Query Clusters +For query plan field names and meanings, see <>. -Couchbase Server allows you to monitor many aspects of an active cluster: cluster-aware operations, diagnostics, and more system keyspaces features that cover multiple nodes. -Functionalities include: +[#sys-completed-req] +== Monitor and Manage Completed Requests -* Ability to access active / completed / prepared requests across all Query nodes from {sqlpp}. -* Ability to list nodes by type and with status from {sqlpp}. -* Ability to list system keyspaces from `system:keyspaces`. -* Extra fields in `system:active_requests` and `system:completed_requests`. -* Counters to keep track of specific requests, such as cancelled requests. -* Killing request for CREATE INDEX. +By default, the `system:completed_requests` catalog maintains a list of the most recent completed requests that have run longer than a predefined threshold of time. +(You can also log completed requests that meet other conditions that you define.) -=== System Keyspaces +For each completed request, this catalog maintains information such as requestId, statement text, prepared name (if prepared statement), request time, service time, and so on. +This information provides a general insight into the health and performance of the query node and the cluster. -* The `system:keyspaces` keyspace can be augmented to list system keyspaces with a static map. -The small disadvantage of this is that it has to be maintained as new system keyspaces are added. -* The `system:active_requests` and `system:completed_requests` keyspaces can report scan consistency. -* The `system:prepareds` keyspace can list min and max execution and service times, as well as average times. +[[sys-completed-get]] +=== Get Completed Requests -=== cbq-engine-cbauth +To get a list of all logged completed requests, use +ifdef::flag-devex-rest-api[] +the Admin REST API or +endif::flag-devex-rest-api[] +a {sqlpp} query. -`cbq-engine-cbauth` is an internal user that the query service uses to allow Query Workbench clients to query across multiple query nodes. +[tabs] +==== +ifdef::flag-devex-rest-api[] +REST API:: ++ +-- +To get a list of all logged completed requests using the Admin REST API: -Since Query Workbench can connect to the same node only when cbq-engine is running, Query Workbench cannot do any query-clustered operations. +[source,sh] +---- +curl -u $USER:$PASSWORD $BASE_URL/admin/completed_requests +---- +-- +endif::flag-devex-rest-api[] -To get around this block, once the Query Workbench clients connect to a query node, this internal user (cbq-engine-cbauth) will be used to do any further inter-node user verification. +{sqlpp}:: ++ +-- +To get a list of all logged completed requests using {sqlpp}: -[#vitals] -== System Vitals +[source,sqlpp] +---- +SELECT * FROM system:completed_requests; +---- +-- +==== -The [.cmd]`Vitals` API provides data about the running state and health of the query engine, such as number of logical cores, active threads, queued threads, CPU utilization, memory usage, network utilization, garbage collection percentage, and so on. -This information can be very useful to assess the current workload and performance characteristics of a query engine, and hence load-balance the requests being sent to various query engines. +Note that the `completed` state means that the request was started and completed by the Query service, but it does not mean that it was necessarily successful. +The request could have been successful, or completed with errors. -For field names and meanings, refer to xref:n1ql:n1ql-rest-api/admin.adoc#_vitals[Vitals]. +To find requests that completed successfully, search for completed requests whose `state` is `completed` and whose `errorCount` field has the value `0`. -=== Get System Vitals +[tabs] +==== +{sqlpp}:: ++ +-- +To get a list of all logged completed requests, including only successful requests: -[source,sh] +[source,sqlpp] ---- -curl -u Administrator:pword http://localhost:8093/admin/vitals +SELECT * FROM system:completed_requests +WHERE state = "completed" AND errorCount = 0; ---- +-- +==== -[source,json] ----- -{ - "uptime": "7h39m32.668577197s", - "local.time": "2021-04-30 18:42:39.517208807 +0000 UTC m=+27573.945319668", - "version": "7.0.0-N1QL", - "total.threads": 191, - "cores": 2, - "gc.num": 669810600, - "gc.pause.time": "57.586373ms", - "gc.pause.percent": 0, - "memory.usage": 247985184, - "memory.total": 11132383704, - "memory.system": 495554808, - "cpu.user.percent": 0, - "cpu.sys.percent": 0, - "request.completed.count": 140, - "request.active.count": 0, - "request.per.sec.1min": 0.0018, - "request.per.sec.5min": 0.0055, - "request.per.sec.15min": 0.0033, - "request_time.mean": "536.348163ms", - "request_time.median": "54.065567ms", - "request_time.80percentile": "981.869933ms", - "request_time.95percentile": "2.543128455s", - "request_time.99percentile": "4.627922799s", - "request.prepared.percent": 0 -} ----- +To get the query plan for completed requests, include `meta().plan` in a {sqlpp} query. +See <>. -[#sys-active-req] -== Monitor and Manage Active Requests +[tabs] +==== +{sqlpp}:: ++ +-- +To view completed requests with {sqlpp}, including the query plan: -The `system:active_requests` catalog lists all currently executing active requests or queries. +[source,sqlpp] +---- +SELECT *, meta().plan FROM system:completed_requests; +---- +-- +==== -For field names and meanings, refer to xref:n1ql:n1ql-rest-api/admin.adoc#_requests[Requests]. -The profile related attributes are described in the section <> +[[sys-completed-delete]] +=== Purge the Completed Requests -[[sys-active-get]] -=== Get Active Requests +To purge a specific completed request, use +ifdef::flag-devex-rest-api[] +the Admin REST API or +endif::flag-devex-rest-api[] +a {sqlpp} query. -To view active requests with Admin REST API: +[tabs] +==== +ifdef::flag-devex-rest-api[] +REST API:: ++ +-- +To purge a completed request [.var]`uuid` with the Admin REST API: [source,sh] ---- -curl -u Administrator:pword http://localhost:8093/admin/active_requests +curl -u $USER:$PASSWORD -X DELETE $BASE_URL/admin/completed_requests/uuid ---- +-- +endif::flag-devex-rest-api[] -To view active requests with {sqlpp}, including the query plan: +{sqlpp}:: ++ +-- +To purge a completed request [.var]`uuid` with {sqlpp}: [source,sqlpp] ---- -SELECT *, meta().plan FROM system:active_requests; +DELETE FROM system:completed_requests WHERE requestId = "uuid"; ---- +-- +==== -[[sys-active-delete]] -=== Terminate an Active Request - -The DELETE command can be used to terminate an active request, for instance, a non-responding or a long-running query. - -To terminate an active request [.var]`uuid` with the Admin REST API: - -[source,sh] ----- -curl -u Administrator:pword -X DELETE http://localhost:8093/admin/active_requests/uuid ----- +To purge completed requests for a given time period, use a {sqlpp} query. -To terminate an active request [.var]`uuid` with {sqlpp}: +[tabs] +==== +{sqlpp}:: ++ +-- +To purge the completed requests for a given time period: [source,sqlpp] ---- -DELETE FROM system:active_requests WHERE requestId = "uuid"; +DELETE FROM system:completed_requests WHERE requestTime LIKE "2015-09-09%"; ---- +-- +==== -[[sys-active-examples]] -=== Examples +[[sys-completed-examples]] +=== Completed Request Details -.Get Active +To try the examples in this section, first run a query which takes at least 1000ms (the default value of the `completed-threshold` query setting) to get registered in the `system:completed_requests` keyspace. + +.Run a long query ==== +include::ROOT:partial$query-context.adoc[tag=example] + +.Query [source,sqlpp] ---- -SELECT *, meta().plan FROM system:active_requests; +SELECT * FROM route ORDER BY sourceairport LIMIT 5; ---- +==== +Getting completed requests, as described in <>, returns results similar to the following. + +==== [source,json] ---- [ + // ... { - "active_requests": { - "clientContextID": "81984707-a9cd-4b78-9110-d130eb580d7f", - "elapsedTime": "65.543946ms", - "executionTime": "65.507111ms", + "completed_requests": { + "clientContextID": "a19a61ab-cd9e-46c9-be71-92623ff85741", + "cpuTime": "912.408423ms", + "elapsedTime": "3.762926948s", + "errorCount": 0, + "errors": [], + "n1qlFeatCtrl": 76, "node": "127.0.0.1:8091", "phaseCounts": { - "primaryScan": 1 + "fetch": 24023, + "primaryScan": 24023, + "primaryScan.GSI": 24023, + "sort": 24028 }, "phaseOperators": { "authorize": 1, "fetch": 1, - "primaryScan": 1 + "primaryScan": 1, + "primaryScan.GSI": 1, + "project": 1, + "sort": 2, + "stream": 1 }, "phaseTimes": { - "authorize": "2.361862ms", - "fetch": "7.222µs", - "instantiate": "13.233µs", - "parse": "660.048µs", - "plan": "52.877µs", - "primaryScan": "41.125271ms" + "authorize": "15.111µs", + "fetch": "3.641125449s", + "instantiate": "332.963µs", + "parse": "1.04015ms", + "plan": "602.878µs", + "plan.index.metadata": "25.849µs", + "plan.keyspace.metadata": "11.586µs", + "primaryScan": "101.118572ms", + "primaryScan.GSI": "101.118572ms", + "project": "33.273783ms", + "run": "3.760767643s", + "sort": "666.364325ms", + "stream": "1.617688ms" }, - "remoteAddr": "127.0.0.1:57065", - "requestId": "27e73286-c6cc-4c26-8977-ef8d68e91c8f", - "requestTime": "2019-05-06 09:08:42.431161361 -0700 PDT m=+6976.301141271", + "queryContext": "default:travel-sample.inventory", + "remoteAddr": "127.0.0.1:37684", + "requestId": "e170bf67-d364-4ed7-9698-784bbb779d18", + "requestTime": "2024-05-21T14:31:46.882Z", + "resultCount": 5, + "resultSize": 17714, "scanConsistency": "unbounded", - "state": "running", - "statement": "SELECT *, meta().plan FROM system:active_requests;", - "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:66.0) Gecko/20100101 Firefox/66.0 (Couchbase Query Workbench (6.5.0-2859-enterprise))", - "users": "Administrator" - }, - "plan": { // <1> - "#operator": "Sequence", - "#stats": { - "#phaseSwitches": 1, - "execTime": "1.661µs" - }, - "~children": [ - { - "#operator": "Authorize", - "#stats": { - "#phaseSwitches": 3, - "execTime": "4.844µs", - "servTime": "2.357018ms" - }, - // ... - } - ] - }, - "plan": { - "#operator": "Sequence", - "#stats": { - "#phaseSwitches": 1, - "execTime": "1.661µs" - }, - "~children": [ - { - "#operator": "Authorize", - "#stats": { - "#phaseSwitches": 3, - "execTime": "4.844µs", - "servTime": "2.357018ms" - }, - "privileges": { - "List": [ - { - "Priv": 4, - "Target": "#system:active_requests" - } - ] - }, - "~child": { - "#operator": "Sequence", - "#stats": { - "#phaseSwitches": 1, - "execTime": "2.71µs" - }, - "~children": [ - { - "#operator": "PrimaryScan", - "#stats": { - "#itemsOut": 1, - "#phaseSwitches": 7, - "execTime": "22.314µs", - "kernTime": "2.13µs", - "servTime": "41.102957ms" - }, - "index": "#primary", - "keyspace": "active_requests", - "namespace": "#system", - "using": "system" - }, - { - "#operator": "Fetch", - "#stats": { - "#itemsIn": 1, - "#phaseSwitches": 5, - "execTime": "7.222µs", - "kernTime": "41.130913ms", - "servTime": "22.888285ms", - "state": "services" - }, - "keyspace": "active_requests", - "namespace": "#system" - }, - { - "#operator": "Sequence", - "#stats": { - "#phaseSwitches": 1, - "execTime": "1.544µs" - }, - "~children": [ - { - "#operator": "InitialProject", - "#stats": { - "#phaseSwitches": 1, - "execTime": "980ns", - "kernTime": "64.066618ms", - "state": "kernel" - }, - "result_terms": [ - { - "expr": "self", - "star": true - }, - { - "expr": "(meta(`active_requests`).`plan`)" - } - ] - }, - { - "#operator": "FinalProject", - "#stats": { - "#phaseSwitches": 1, - "execTime": "827ns" - } - } - ] - } - ] - } - }, - { - "#operator": "Stream", - "#stats": { - "#phaseSwitches": 1, - "execTime": "1.29µs", - "kernTime": "66.511806ms", - "state": "kernel" - } - } - ], - "~versions": [ - "2.0.0-N1QL", - "6.5.0-2859-enterprise" - ] + "serviceTime": "3.762768429s", + "state": "completed", + "statement": "SELECT * FROM route ORDER BY sourceairport LIMIT 5;", + "statementType": "SELECT", + "useCBO": true, + "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:126.0) Gecko/20100101 Firefox/126.0", + "users": "builtin:Administrator", + "~qualifier": "threshold" } } ] ---- - -<1> The *plan* section contains a tree of operators that combine to execute the {sqlpp} query. -The root operator is a Sequence, which itself has a collection of child operators like Authorize, PrimaryScan, Fetch, and possibly even more Sequences. ==== -[#sys-prepared] -== Monitor and Manage Prepared Statements +For field names and meanings, see <>. -The `system:prepareds` catalog provides data about the known prepared statements and their state in a query engine's prepared statement cache. -For each prepared statement, this catalog provides information such as name, statement, query plan, last use time, number of uses, and so on. +For query plan field names and meanings, see <>. -For field names and meanings, refer to xref:n1ql:n1ql-rest-api/admin.adoc#_statements[Statements]. -The `system:prepareds` catalog returns all the properties that the {sqlpp} Admin REST API would return for a specific prepared statement. -In addition, the `system:prepareds` catalog also returns the following properties. +ifdef::flag-devex-rest-api[] +[[sys-completed-config]] +== Configure the Completed Requests -[options="header", cols=".^3a,.^11a,.^4a"] -|=== -|Name|Description|Schema -|**node** + -__required__ -|The node on which the prepared statement is stored. -|string +You can configure the `system:completed_requests` keyspace by specifying parameters through the Admin API `/admin/settings` endpoint. -|**namespace** + -__required__ -|The namespace in which the prepared statement is stored. -Currently, only the `default` namespace is available. -|string -|=== +You can specify the conditions for completed request logging using the `completed` field. +This field takes a JSON object containing the names and values of logging qualifiers. +Completed requests that meet the defined qualifiers are logged. -A prepared statement is created and stored relative to the current xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context[query context]. -You can create multiple prepared statements with the same name, each stored relative to a different query context. -This enables you to run multiple instances of the same application against different datasets. +[source,sh] +---- +curl $BASE_URL/admin/settings -u $USER:$PASSWORD \ + -H 'Content-Type: application/json' \ + -d '{"completed": {"user": "marco", "error": 12003}}' +---- -When there are multiple prepared statements with the same name in different query contexts, the name of the prepared statement in the `system:prepareds` catalog includes the associated query context in brackets. +=== Logging Qualifiers -[[sys-prepared-get]] -=== Get Prepared Statements +You can specify the following logging qualifiers. +A completed request is logged if _any_ of the qualifiers are met (logical OR). + +[horizontal] +`threshold`:: The execution time threshold in milliseconds. +`aborted`:: Whether to log requests that generate a panic. +`error`:: Log requests returning this error number. +`client`:: Log requests from this IP address. +`user`:: Log requests with this user name. +`context`:: Log requests with this client context ID. + +For full details, see xref:n1ql:n1ql-rest-api/admin.adoc#_logging_parameters[Logging parameters]. + +The basic syntax adds a qualifier to the logging parameters, i.e. any existing qualifiers are not removed. +You can change the value of a logging qualifier by specifying the same qualifier again with a new value. + +To add a new instance of an existing qualifier, use a plus sign (`+`) before the qualifier name, e.g. `+user`. +To remove a qualifier, use a minus sign (`-`) before the qualifier name, e.g. `-user`. -To get a list of all known prepared statements, you can use the Admin REST API or a {sqlpp} query: +For example, the following request will add user `simon` to those tracked, and remove error `12003`. [source,sh] ---- -curl -u Administrator:pword http://localhost:8093/admin/prepareds +curl $BASE_URL/admin/settings -u $USER:$PASSWORD \ + -H 'Content-Type: application/json' \ + -d '{"completed": {"+user": "simon", "-error": 12003}}' ---- -[source,sqlpp] +Similarly, you could remove all logging by execution time with the following request, as long as the value matches the existing threshold. + +[source,sh] ---- -SELECT * FROM system:prepareds; +curl $BASE_URL/admin/settings -u $USER:$PASSWORD \ + -H 'Content-Type: application/json' \ + -d '{"completed": {"-threshold": 1000}}' ---- -To get information about a specific prepared statement [.var]`example1`, you can use the Admin REST API or a {sqlpp} query: +=== Tagged Sets + +You can also specify qualifiers that have to be met as a group for the completed request to be logged (logical AND). + +To do this, specify the `tag` field along with a set of qualifiers, like so: [source,sh] ---- -curl -u Administrator:pword http://localhost:8093/admin/prepareds/example1 +curl $BASE_URL/admin/settings -u $USER:$PASSWORD \ + -H 'Content-Type: application/json' \ + -d '{"completed": {"user": "marco", "error": 12003, "tag": "both_user_and_error"}}' ---- -[source,sqlpp] ----- -SELECT * FROM system:prepareds WHERE name = "example1"; ----- +In this case, the request will be logged when both user and error match. -[[sys-prepared-delete]] -=== Delete Prepared Statements +The tag name can be any string that is meaningful and unique. +Requests that match a tagged set of conditions are logged with a field `~tag`, which is set to the name of the tag. -To delete a specific prepared statement [.var]`p1`, you can use the Admin REST API or a {sqlpp} query: +To add a qualifier to a tagged set, specify the tag name again along with the new qualifier: [source,sh] ---- -curl -u Administrator:pword -X DELETE http://localhost:8093/admin/prepareds/p1 +curl $BASE_URL/admin/settings -u $USER:$PASSWORD \ + -H 'Content-Type: application/json' \ + -d '{"completed": {"client": "172.1.2.3", "tag": "both_user_and_error"}}' ---- -[source,sqlpp] +You cannot add a new instance of an existing qualifier to a tagged set using a plus sign (`+`) before the qualifier name. +For example, you cannot add a `user` qualifier to a tagged set that already contains a `user` qualifier. +If you need to track two users with the same error, create two tagged sets, one per user. + +You can remove a qualifier from a tagged set using a minus sign (`-`) before the qualifier name, e.g. `-user`. +When you remove the last qualifier from a tagged set, the tagged set is removed. + +[NOTE] +-- +You can specify multiple tagged sets. +In this case, completed requests are logged if they match all of the qualifiers in any of the tagged sets. + +You can also specify a mixture of tagged sets and individual qualifiers. +In this case, completed requests are logged if they match any of the individual qualifiers, or all of the qualifiers in any of the tagged sets. +-- + +=== Completed Threshold + +The [.param]`completed-threshold` field provides another way of specifying the `threshold` qualifier within the `completed` field. + +This field sets the minimum request duration after which requests are added to the `system:completed_requests` catalog. +The default value is 1000ms. +Specify [.in]`0` to log all requests and [.in]`-1` to not log any requests to the keyspace. + +To specify a different value, use: + +[source,sh] ---- -DELETE FROM system:prepareds WHERE name = "p1"; +curl $BASE_URL/admin/settings -u $USER:$PASSWORD \ + -H 'Content-Type: application/json' \ + -d '{"completed-threshold":0}' ---- -To delete all the known prepared statements, you must use a {sqlpp} query: +=== Completed Limit -[source,sqlpp] +The [.param]`completed-limit` field sets the number of most recent requests to be tracked in the `system:completed_requests` catalog. +The default value is 4000. +Specify [.in]`0` to not track any requests and [.in]`-1` to set no limit. + +To specify a different value, use: + +[source,sh] ---- -DELETE FROM system:prepareds; +curl $BASE_URL/admin/settings -u $USER:$PASSWORD \ + -H 'Content-Type: application/json' \ + -d '{"completed-limit":1000}' ---- +endif::flag-devex-rest-api[] -[[sys-prepared-examples]] -=== Examples +[#query-monitoring-settings] +== Query Profiling -.Get Prepared +Query profiling enables you to obtain more detailed monitoring information and finer execution timings for any query. +You can set query profiling to the following levels: + +ifdef::flag-devex-tools[] +* `off` -- query profiling is disabled. +endif::flag-devex-tools[] +* `phases` -- query profiling is enabled, including information about the phases of query execution. +* `timings` -- query profiling is enabled, including information about the phases of query execution, and detailed timing information. + +You can set query profiling: + +ifdef::flag-devex-rest-api[] +* At the <>, so that it is enabled for all queries on that node. +endif::flag-devex-rest-api[] +* At the <>, for individual queries. + +For more information about Query settings and parameters, see xref:n1ql:n1ql-manage/query-settings.adoc[]. + +ifdef::flag-devex-rest-api[] +[#enable-settings-for-a-query-engine] +=== Enable Query Profiling for a Query Node + +To activate query profiling at the node level, specify the `profile` setting using the xref:n1ql:n1ql-rest-api/admin.adoc[Admin REST API] (`/admin/settings` endpoint). + +.See the current node-level query settings ==== -.Prepared statement with default query context -- using cbq -[source,sqlpp] +The following request gets the current node-level query settings. + +.Request +[source,sh] ---- -\UNSET -query_context; -PREPARE p1 AS SELECT * FROM `travel-sample`.inventory.airline WHERE iata = "U2"; +include::n1ql:example$settings/node-level-settings.sh[tag=curl] ---- +.Results [source,json] ---- -{ - "requestID": "64069886-eb17-4fa6-8cc8-e60ebe93d97c", - "signature": "json", - "results": [ - { - "encoded_plan": "H4sIAAAAAAAA/wEAAP//AAAAAAAAAAA=", - "featureControls": 76, - "indexApiVersion": 4, - "indexScanKeyspaces": { - "default:travel-sample.inventory.airline": false - }, - "name": "[127.0.0.1:8091]p1", - "namespace": "default", - "operator": { - "#operator": "Authorize", - "privileges": { - "List": [ - { - "Priv": 7, - "Props": 0, - "Target": "default:travel-sample.inventory.airline" - } - ] - }, - "~child": { - "#operator": "Sequence", - "~children": [ - { - "#operator": "Sequence", - "~children": [ - { - "#operator": "PrimaryScan3", - "bucket": "travel-sample", - "index": "def_inventory_airline_primary", - "index_projection": { - "primary_key": true - }, - "keyspace": "airline", - "namespace": "default", - "scope": "inventory", - "using": "gsi" - }, - { - "#operator": "Fetch", - "bucket": "travel-sample", - "keyspace": "airline", - "namespace": "default", - "scope": "inventory" - }, - { - "#operator": "Parallel", - "~child": { - "#operator": "Sequence", - "~children": [ - { - "#operator": "Filter", - "condition": "((`airline`.`iata`) = \"U2\")" - }, - { - "#operator": "InitialProject", - "result_terms": [ - { - "expr": "self", - "star": true - } - ] - } - ] - } - } - ] - }, - { - "#operator": "Stream" - } - ] - } - }, - "queryContext": "", - "reqType": "SELECT", - "signature": { - "*": "*" - }, - "text": "PREPARE p1 AS SELECT * FROM `travel-sample`.inventory.airline WHERE iata = \"U2\";", - "useCBO": true - } - ], - "status": "success", - "metrics": { - "elapsedTime": "105.814848ms", - "executionTime": "105.648798ms", - "resultCount": 1, - "resultSize": 3301, - "serviceLoad": 12 - } -} +include::n1ql:example$settings/node-level-settings.jsonc[] ---- +==== -.Prepared statement with specified query context -- using cbq -[source,sqlpp] +.Save node-level query settings to a file +==== +The following request saves the current node-level query settings to the file `query_settings.json`. + +.Request +[source,sh] ---- -\SET -query_context travel-sample.inventory; -PREPARE p1 AS SELECT * FROM airline WHERE iata = "U2"; +include::n1ql:example$settings/save-node-level-settings.sh[tag=curl] ---- +==== -[source,json] +.Set node-level query settings from a file +==== +Assuming that you have edited the file `query_settings.json` to specify the query settings you want, the following request +sets the node-level query settings according to the file. + +.Request +[source,sh] ---- -{ - "requestID": "1c90603e-5e42-42b4-9362-4fc96bf895ac", - "signature": "json", - "results": [ - { - "encoded_plan": "H4sIAAAAAAAA/wEAAP//AAAAAAAAAAA=", - "featureControls": 76, - "indexApiVersion": 4, - "indexScanKeyspaces": { - "default:travel-sample.inventory.airline": false - }, - "name": "[127.0.0.1:8091]p1", - "namespace": "default", - "operator": { - "#operator": "Authorize", - "privileges": { - "List": [ - { - "Priv": 7, - "Props": 0, - "Target": "default:travel-sample.inventory.airline" - } - ] - }, - "~child": { - "#operator": "Sequence", - "~children": [ - { - "#operator": "Sequence", - "~children": [ - { - "#operator": "PrimaryScan3", - "bucket": "travel-sample", - "index": "def_inventory_airline_primary", - "index_projection": { - "primary_key": true - }, - "keyspace": "airline", - "namespace": "default", - "scope": "inventory", - "using": "gsi" - }, - { - "#operator": "Fetch", - "bucket": "travel-sample", - "keyspace": "airline", - "namespace": "default", - "scope": "inventory" - }, - { - "#operator": "Parallel", - "~child": { - "#operator": "Sequence", - "~children": [ - { - "#operator": "Filter", - "condition": "((`airline`.`iata`) = \"U2\")" - }, - { - "#operator": "InitialProject", - "result_terms": [ - { - "expr": "self", - "star": true - } - ] - } - ] - } - } - ] - }, - { - "#operator": "Stream" - } - ] - } - }, - "queryContext": "travel-sample.inventory", - "reqType": "SELECT", - "signature": { - "*": "*" - }, - "text": "PREPARE p1 AS SELECT * FROM airline WHERE iata = \"U2\";", - "useCBO": true - } - ], - "status": "success", - "metrics": { - "elapsedTime": "17.476424ms", - "executionTime": "17.187836ms", - "resultCount": 1, - "resultSize": 3298, - "serviceLoad": 12 - } -} +curl $BASE_URL/admin/settings -u $USER:$PASSWORD \ + -X POST \ + -d@./query_settings.json ---- +==== -.List prepared statements -[source,sqlpp] +.Set node-level query settings explicitly +==== +The following request explicitly sets query profiling at the node level. + +.Request +[source,sh] ---- -SELECT *, meta().plan FROM system:prepareds; +curl $BASE_URL/admin/settings -u $USER:$PASSWORD \ + -H 'Content-Type: application/json' \ + -d '{"profile": "phases"}' ---- +.Results [source,json] ---- { - "requestID": "d976e59a-d74e-4350-b0df-fa137099d594", - "signature": { - "*": "*", - "plan": "json" - }, - "results": [ - { - "plan": { - // ... - }, - "prepareds": { - "encoded_plan": "H4sIAAAAAAAA/6RTUW/TPBT9K9H5XrbJ30QBMcmIhzJ1AjG0qh3wwKbEJLedmWt71061UIXfjpxkndYhENpbYt97z7nn+GxAtnQVVbk3ykICAgtSsWY6djayMwHy6JWAthXdjr3+TBy0s5Avh7N5qewHaoJXJQXIDSpaqNpEGVmtyfwf1MobOtR2TTY6bg6VZqMtQS6UCdQKWLUiSPgR+u9uFOTdIAg4T6yi4zT+v/sfjOt45Vj/IAh41mttaNmTONUhQn7d4FzxkuL9tL/SEpiyXkMepQ/nA+Sz9rIV+FleaVPtMpjTTU22TG19AZPtcP+5aMp6pbhJcr6AwLe6vO54P+CLQfR+n3zLPh/Y576fcleXe3bfqYydYxsMt/k1NZCR66T+9eAdJO4l+L0NoXQ+nWxhIVAHbZeQWAaNVjxc6YRiefWnXZ6EvYs2VayMIYMneXWiTSSGQOlspXvhsLdXDPyKw0KrqIr97E12gU/PL7D/iMh7q6NWZtpLDwGmUJuYR+JV6ADp1qfCQGaRVouKBzsu28s2vbYd4pFJrZDuBFJOtyE8Go0EbmriJqWVbmOfYKab86aTaz45nRyfJxC9tF2skyoHkDhAKzC0TGeT6Xg2yfwoG8+zvic7yE5mZx+z4oFpxePEZF/eTWaTLMmyFeV19zLo+O3ZsNivAAAA//+q+jhuaAQAAA==", - "featuresControl": 76, - "indexApiVersion": 4, - "indexScanKeyspaces": { - "default:travel-sample.inventory.airline": false - }, - "name": "p1", // <1> - "namespace": "default", - "node": "127.0.0.1:8091", - "statement": "PREPARE p1 AS SELECT * FROM `travel-sample`.inventory.airline WHERE iata = \"U2\";", - "uses": 0 - } - }, - { - "plan": { - // ... - }, - "prepareds": { - "encoded_plan": "H4sIAAAAAAAA/6STT28TMRDFv8rqcWkrExFAVDLiEKpUIIoaJQUOtNqY3Ulq6tju2Bt1iZbPjry7TWmKQKg3/xnPvPk9zwZkC1dSmXujLCQgsCAVK6YjZyM7EyAPXwloW9LNyOvPxEE7C/myP5sVyn6gOnhVUIDcoKSFqkyUkdWazNOgVt7QQNs12ei4HijNRluCXCgTqBGwakWQ8EN06zYV5G0iCDhPrKLjlP7J3QajKl461j8IAp71WhtadiJOdIiQXzc4U7ykeJftn7IEJqzXkIdp4XyAfNZcNAI/i0ttyl0FM7quyBbpWRfAZNu6/x00Yb1SXCecLyDwrSquWt339KKH3vWTb9Xnvfrcd1lu43LP7jsVsXVsg/42v6IaMnKV6F/13kHiDsGfbQiF8+lkWxYCVdB2CYll0GjE/ZaOKRaXf+vlUbV3q00UK2PI4FFeHWsTiSFQOFvqDhz29ua9vvlgrlVU8/3sTXaOT8/Psf9AyHuro1Zm0qGHAFOoTMwj8Sq0BenGp8BAZpFai4p7Oy6aiyb9th3hkUmtkO4E0pxuh/BwOBS4rojrNK108wDy4HevmK7P6pbibHwyPjpLtfXSttOeYB1A4gCNQJ9pMh1PRtNx5ofZaJZ1b7KD7Hh6+jHreWRf3o2n4ywx2RJ53X4LOnp72nf1KwAA////9+bsZQQAAA==", - "featuresControl": 76, - "indexApiVersion": 4, - "indexScanKeyspaces": { - "default:travel-sample.inventory.airline": false - }, - "name": "p1(travel-sample.inventory)", // <2> - "namespace": "default", - "node": "127.0.0.1:8091", - "statement": "PREPARE p1 AS SELECT * FROM airline WHERE iata = \"U2\";", - "uses": 0 - } - } - ], - "status": "success", - "metrics": { - "elapsedTime": "25.323496ms", - "executionTime": "25.173646ms", - "resultCount": 2, - "resultSize": 7891, - "serviceLoad": 12 - } + // ... + "profile":"phases", + "request-size-cap": 67108864, + "scan-cap": 512, + "servicers": 4, + "timeout": 0, + "txtimeout": "0s", + "use-cbo": true } ---- - -Note that the names of the prepared statements are identical, but they are associated with different query contexts. - -<.> The name of the prepared statement for the default query context -<.> The name of the prepared statement showing the associated query context ==== +endif::flag-devex-rest-api[] -[#sys-completed-req] -== Monitor and Manage Completed Requests +[#enable-settings-per-session-or-per-query] +=== Enable Query Profiling for a Request -By default, the `system:completed_requests` catalog maintains a list of the most recent completed requests that have run longer than a predefined threshold of time. -(You can also log completed requests that meet other conditions that you define.) - -For each completed request, this catalog maintains information such as requestId, statement text, prepared name (if prepared statement), request time, service time, and so on. -This information provides a general insight into the health and performance of the query engine and the cluster. - -For field names and meanings, refer to xref:n1ql:n1ql-rest-api/admin.adoc#_requests[Requests]. -Most field names and meanings match exactly those of `system:active_requests`. - -Note that the `completed` state means that the request was started and completed by the Query service, but it does not mean that it was necessarily successful. -The request could have been successful, or completed with errors. +ifdef::flag-devex-tools[] +To activate profiling at the request level, you can: -To find requests that completed successfully, search for completed requests whose `state` is `completed` and whose `errorCount` field has the value `0`. +ifdef::flag-devex-rest-api[] +* Specify the `profile` setting using the xref:n1ql:n1ql-rest-api/index.adoc[Query REST API] (`/query/service` endpoint). +endif::flag-devex-rest-api[] +* Specify the `profile` setting using the xref:n1ql:n1ql-intro/cbq.adoc[cbq] command line tool. -[NOTE] -==== -Request profiling affects the `system:completed_requests` keyspace in the following ways: - -* When the feature is turned on, completed requests are stored with their execution plan. -* Profiling information is likely to use 100KB+ per entry. -* Due to the added overhead of running both profiling and logging, we recommend turning on both of them only when needed. -Running only one of them continuously has no noticeable affect on performance. -* Profiling does not carry any extra cost beyond memory for completed requests, so it's fine to run it continuously. +[tabs] ==== +ifdef::flag-devex-rest-api[] +REST API:: ++ +-- +To set query settings using the REST API, specify the parameters in the request body. -[[sys-completed-get]] -=== Get Completed Requests +''' -To get a list of all logged completed requests using the Admin REST API: +The following statement sets the profiling to phases: [source,sh] ---- -curl -u Administrator:pword http://localhost:8093/admin/completed_requests +curl $BASE_URL/query/service -u $USER:$PASSWORD \ + -d 'profile=phases&statement=SELECT * FROM `travel-sample`.inventory.airline LIMIT 1' ---- -To get a list of all logged completed requests using {sqlpp}, including the query plan: - -[source,sqlpp] ----- -SELECT *, meta().plan FROM system:completed_requests; ----- - -To get a list of all logged completed requests using {sqlpp}, including only successful requests: - -[source,sqlpp] ----- -SELECT * FROM system:completed_requests -WHERE state = "completed" AND errorCount = 0; ----- - -[[sys-completed-delete]] -=== Purge the Completed Requests - -To purge a completed request [.var]`uuid` with the Admin REST API: +The following statement sets the profiling to timings: [source,sh] ---- -curl -u Administrator:pword -X DELETE http://localhost:8093/admin/completed_requests/uuid +curl $BASE_URL/query/service -u $USER:$PASSWORD \ + -d 'profile=timings&statement=SELECT * FROM `travel-sample`.inventory.airline LIMIT 1' ---- +-- +endif::flag-devex-rest-api[] -To purge a completed request [.var]`uuid` with {sqlpp}: +{sqlpp}:: ++ +-- +To set query settings using the cbq shell, use the `\SET` command. -[source,sqlpp] ----- -DELETE FROM system:completed_requests WHERE requestId = "uuid"; ----- +''' -To purge the completed requests for a given time period, use: +The following statement sets the profiling to phases: [source,sqlpp] ---- -DELETE FROM system:completed_requests WHERE requestTime LIKE "2015-09-09%"; +\set -profile "phases"; +SELECT * FROM `travel-sample`.inventory.airline LIMIT 1; ---- -[[sys-completed-config]] -=== Configure the Completed Requests - -You can configure the `system:completed_requests` keyspace by specifying parameters through the Admin API settings endpoint. +The following statement sets the profiling to timings: -In Couchbase Server 6.5 and later, you can specify the conditions for completed request logging using the `completed` field. - -This field takes a JSON object containing the names and values of logging qualifiers. -Completed requests that meet the defined qualifiers are logged. - -[source,sh] +[source,sqlpp] ---- -curl http://localhost:8093/admin/settings -u Administrator:password \ - -H 'Content-Type: application/json' \ - -d '{"completed": {"user": "marco", "error": 12003}}' +\set -profile "timings"; +SELECT * FROM `travel-sample`.inventory.airline LIMIT 1; ---- +-- +==== +endif::flag-devex-tools[] -==== Logging Qualifiers - -You can specify the following logging qualifiers. -A completed request is logged if _any_ of the qualifiers are met (logical OR). - -[horizontal] -`threshold`:: The execution time threshold in milliseconds. -`aborted`:: Whether to log requests that generate a panic. -`error`:: Log requests returning this error number. -`client`:: Log requests from this IP address. -`user`:: Log requests with this user name. -`context`:: Log requests with this client context ID. - -For full details, refer to xref:n1ql:n1ql-rest-api/admin.adoc#_logging_parameters[Logging parameters]. - -The basic syntax adds a qualifier to the logging parameters, i.e. any existing qualifiers are not removed. -You can change the value of a logging qualifier by specifying the same qualifier again with a new value. +The Query tab automatically enables Query profiling, with detailed timing information. +To disable or enable Query profiling with the Query tab, specify the *Collect query timings* option using the xref:clusters:query-service/query-workbench.adoc#query-settings[Query Settings]. -To add a new instance of an existing qualifier, use a plus sign (`+`) before the qualifier name, e.g. `+user`. -To remove a qualifier, use a minus sign (`-`) before the qualifier name, e.g. `-user`. +[#monitor-profile-details] +== Query Profiling Details -For example, the following request will add user `simon` to those tracked, and remove error `12003`. +You can access the profiling information: -[source,sh] ----- -curl http://localhost:8093/admin/settings -u Administrator:password \ - -H 'Content-Type: application/json' \ - -d '{"completed": {"+user": "simon", "-error": 12003}}' ----- +ifdef::flag-devex-tools[] +* In the <>. +endif::flag-devex-tools[] +* In the <>. -Similarly, you could remove all logging by execution time with the following request, as long as the value matches the existing threshold. +When a query executes a user-defined function, profiling information is available for the {sqlpp} queries within the user-defined function as well. -[source,sh] ----- -curl http://localhost:8093/admin/settings -u Administrator:password \ - -H 'Content-Type: application/json' \ - -d '{"completed": {"-threshold": 1000}}' ----- +ifdef::flag-devex-tools[] +[[profile]] +=== Profiling Details in Query Responses -==== Tagged Sets +When profiling is enabled: -You can also specify qualifiers that have to be met as a group for the completed request to be logged (logical AND). +* If you are using +ifdef::flag-devex-rest-api[] +the Query REST API or +endif::flag-devex-rest-api[] +the cbq shell, query profiling information is returned with the query results. +* If you are using the Query workbench, query profiling information is not returned with the query results. -To do this, specify the `tag` field along with a set of qualifiers, like so: +.Phases Profile +==== +If you are using +ifdef::flag-devex-rest-api[] +the Query REST API or +endif::flag-devex-rest-api[] +the cbq shell, the following statistics are returned when `profile` is set to `phases`: -[source,sh] +[source,json] ---- -curl http://localhost:8093/admin/settings -u Administrator:password \ - -H 'Content-Type: application/json' \ - -d '{"completed": {"user": "marco", "error": 12003, "tag": "both_user_and_error"}}' +{ + "requestID": "06d6c1c2-1a8a-4989-a856-7314f9eddee5", + "signature": { + "*": "*" + }, + "results": [ + { + "airline": { + "callsign": "MILE-AIR", + "country": "United States", + "iata": "Q5", + "icao": "MLA", + "id": 10, + "name": "40-Mile Air", + "type": "airline" + } + } + ], + "status": "success", + "metrics": { + "elapsedTime": "12.77927ms", + "executionTime": "12.570648ms", + "resultCount": 1, + "resultSize": 254, + "serviceLoad": 12 + }, + "profile": { + "phaseTimes": { + "authorize": "19.629µs", + "fetch": "401.997µs", + "instantiate": "147.686µs", + "parse": "4.545234ms", + "plan": "409.364µs", + "primaryScan": "6.103775ms", + "run": "6.699056ms" + }, + "phaseCounts": { + "fetch": 1, + "primaryScan": 1 + }, + "phaseOperators": { + "authorize": 1, + "fetch": 1, + "primaryScan": 1 + }, + "requestTime": "2021-04-30T18:37:56.394Z", + "servicingHost": "127.0.0.1:8091" + } +} ---- +==== -In this case, the request will be logged when both user and error match. - -The tag name can be any string that is meaningful and unique. -Requests that match a tagged set of conditions are logged with a field `~tag`, which is set to the name of the tag. - -To add a qualifier to a tagged set, specify the tag name again along with the new qualifier: +.Timings Profile +==== +If you are using +ifdef::flag-devex-rest-api[] +the Query REST API or +endif::flag-devex-rest-api[] +the cbq shell, the following statistics are returned when `profile` is set to `timings`: -[source,sh] +[source,json] ---- -curl http://localhost:8093/admin/settings -u Administrator:password \ - -H 'Content-Type: application/json' \ - -d '{"completed": {"client": "172.1.2.3", "tag": "both_user_and_error"}}' +{ + "requestID": "268a1240-6864-43a2-af13-ccb8d1e50abf", + "signature": { + "*": "*" + }, + "results": [ + { + "airline": { + "callsign": "MILE-AIR", + "country": "United States", + "iata": "Q5", + "icao": "MLA", + "id": 10, + "name": "40-Mile Air", + "type": "airline" + } + } + ], + "status": "success", + "metrics": { + "elapsedTime": "2.915245ms", + "executionTime": "2.755355ms", + "resultCount": 1, + "resultSize": 254, + "serviceLoad": 12 + }, + "profile": { + "phaseTimes": { + "authorize": "18.096µs", + "fetch": "388.122µs", + "instantiate": "31.702µs", + "parse": "646.157µs", + "plan": "120.427µs", + "primaryScan": "1.402918ms", + "run": "1.936852ms" + }, + "phaseCounts": { + "fetch": 1, + "primaryScan": 1 + }, + "phaseOperators": { + "authorize": 1, + "fetch": 1, + "primaryScan": 1 + }, + "requestTime": "2021-04-30T18:40:13.239Z", + "servicingHost": "127.0.0.1:8091", + "executionTimings": { + "#operator": "Authorize", + "#stats": { + "#phaseSwitches": 4, + "execTime": "1.084µs", + "servTime": "17.012µs" + }, + "privileges": { + "List": [ + { + "Target": "default:travel-sample.inventory.airline", + "Priv": 7, + "Props": 0 + } + ] + }, + "~child": { + "#operator": "Sequence", + "#stats": { + "#phaseSwitches": 1, + "execTime": "2.474µs" + }, + "~children": [ + { + "#operator": "PrimaryScan3", + "#stats": { + "#itemsOut": 1, + "#phaseSwitches": 7, + "execTime": "18.584µs", + "kernTime": "8.869µs", + "servTime": "1.384334ms" + }, + "bucket": "travel-sample", + "index": "def_inventory_airline_primary", + "index_projection": { + "primary_key": true + }, + "keyspace": "airline", + "limit": "1", + "namespace": "default", + "scope": "inventory", + "using": "gsi" + }, + { + "#operator": "Fetch", + "#stats": { + "#itemsIn": 1, + "#itemsOut": 1, + "#phaseSwitches": 10, + "execTime": "25.64µs", + "kernTime": "1.427752ms", + "servTime": "362.482µs" + }, + "bucket": "travel-sample", + "keyspace": "airline", + "namespace": "default", + "scope": "inventory" + }, + { + "#operator": "InitialProject", + "#stats": { + "#itemsIn": 1, + "#itemsOut": 1, + "#phaseSwitches": 9, + "execTime": "6.006µs", + "kernTime": "1.825917ms" + }, + "result_terms": [ + { + "expr": "self", + "star": true + } + ] + }, + { + "#operator": "Limit", + "#stats": { + "#itemsIn": 1, + "#itemsOut": 1, + "#phaseSwitches": 4, + "execTime": "2.409µs", + "kernTime": "2.094µs" + }, + "expr": "1" + }, + { + "#operator": "Stream", + "#stats": { + "#itemsIn": 1, + "#itemsOut": 1, + "#phaseSwitches": 6, + "execTime": "46.964µs", + "kernTime": "1.844828ms" + } + } + ] + }, + "~versions": [ + "7.0.0-N1QL", + "7.0.0-4960-enterprise" + ] + } + } +} ---- +==== -You cannot add a new instance of an existing qualifier to a tagged set using a plus sign (`+`) before the qualifier name. -For example, you cannot add a `user` qualifier to a tagged set that already contains a `user` qualifier. -If you need to track two users with the same error, create two tagged sets, one per user. - -You can remove a qualifier from a tagged set using a minus sign (`-`) before the qualifier name, e.g. `-user`. -When you remove the last qualifier from a tagged set, the tagged set is removed. - -[NOTE] --- -You can specify multiple tagged sets. -In this case, completed requests are logged if they match all of the qualifiers in any of the tagged sets. - -You can also specify a mixture of tagged sets and individual qualifiers. -In this case, completed requests are logged if they match any of the individual qualifiers, or all of the qualifiers in any of the tagged sets. --- - -==== Completed Threshold - -The [.param]`completed-threshold` field provides another way of specifying the `threshold` qualifier within the `completed` field. +The `profile` object contains the following attributes: -This field sets the minimum request duration after which requests are added to the `system:completed_requests` catalog. -The default value is 1000ms. -Specify [.in]`0` to log all requests and [.in]`-1` to not log any requests to the keyspace. +include::n1ql:partial$n1ql-rest-api/query/definitions.adoc[tags=profile] +endif::flag-devex-tools[] -To specify a different value, use: +[[plan]] +=== Profiling Details in System Catalogs -[source,sh] ----- -curl http://localhost:port/admin/settings -u user:pword \ - -H 'Content-Type: application/json' \ - -d '{"completed-threshold":0}' ----- +The <> and <> system catalogs always return profiling information regarding query phases: that is, phase times, phase counts, and phase operators. -==== Completed Limit +The <>, <>, and <> system catalogs also support the `meta().plan` virtual attribute. +This captures the whole query plan, and includes profiling information regarding execution timings. -The [.param]`completed-limit` field sets the number of most recent requests to be tracked in the `system:completed_requests` catalog. -The default value is 4000. -Specify [.in]`0` to not track any requests and [.in]`-1` to set no limit. +To get execution timing information from these system catalogs, you must explicitly specify `meta().plan` in the projection list for the SELECT query. -To specify a different value, use: +Within these system catalogs, not all statements have a `meta().plan` attribute. -[source,sh] ----- -curl http://localhost:port/admin/settings -u user:pword \ - -H 'Content-Type: application/json' \ - -d '{"completed-limit":1000}' ----- +* With <> and <>, the `meta().plan` attribute is only available for statements that you run when profile is set to `timings`. -[[sys-completed-examples]] -=== Examples +* With <>, the `meta().plan` attribute is available for all statements. -[[example-2]] -.Completed Request +[NOTE] ==== -First, using the cbq shell, we set `profile = "timings"` and run a long query which takes at least 1000ms (the default value of the `completed-threshold` query setting) to get registered in the `system:completed_requests` keyspace: - -.Query 1 -[source,sqlpp] ----- -\set -profile "timings"; -SELECT * FROM `travel-sample`.inventory.route ORDER BY sourceairport; ----- - -Now, using the cbq shell, we change the profile setting to "phases" and rerun another long query: - -.Query 2 -[source,sqlpp] ----- -\set -profile "phases"; -SELECT * FROM `travel-sample`.inventory.route ORDER BY destinationairport; ----- +When request profiling is set to `timings`, profiling information is likely to use 100KB+ per entry in the `system:completed_requests` keyspace. -Run a query `system:completed_requests` keyspace with `meta().plan`. +* Due to the added overhead of running both profiling and xref:clusters:monitoring/monitoring.adoc#activity-logs[logging], turn on both of them only when needed. +Running only one of them continuously has no noticeable affect on performance. +* Profiling does not carry any extra cost beyond memory for completed requests, so it's fine to run it continuously. +==== -.Query 3 -[source,sqlpp] ----- -SELECT meta().plan, * from system:completed_requests; ----- +[[example-2]] +.Plan Details +==== +Getting the plan for a statement that you ran when the profile was set to `timings` returns results similar to the following. -.Result [source,json] ---- -{ - "requestID": "4a36e1dc-cea0-4ba2-a428-258511d50582", - "signature": { - "*": "*", - "plan": "json" - }, - "results": [ - // ... - { // <1> - "completed_requests": { - "elapsedTime": "15.641879295s", - "errorCount": 0, - "node": "127.0.0.1:8091", - "phaseCounts": { - "fetch": 24024, - "primaryScan": 24024, - "sort": 24024 - }, - "phaseOperators": { - "authorize": 1, - "fetch": 1, - "primaryScan": 1, - "sort": 1 - }, - "phaseTimes": { - "authorize": "51.305µs", - "fetch": "3.27276723s", - "instantiate": "60.662µs", - "parse": "66.701943ms", - "plan": "15.12951ms", - "primaryScan": "171.439769ms", - "run": "15.548781894s", - "sort": "153.767638ms" - }, - "remoteAddr": "172.17.0.1:56962", - "requestId": "08445bae-66ef-4ccd-8b2d-ea899b453a1b", - "requestTime": "2021-04-30T21:14:57.576Z", - "resultCount": 24024, - "resultSize": 81821919, - "scanConsistency": "unbounded", - "serviceTime": "15.630714144s", - "state": "completed", - "statement": "SELECT * FROM `travel-sample`.inventory.route ORDER BY destinationairport;", - "useCBO": true, - "userAgent": "Go-http-client/1.1 (CBQ/2.0)", - "users": "Administrator" - } - }, - // ... - { // <2> - "completed_requests": { - "elapsedTime": "15.321128463s", - "errorCount": 0, - "node": "127.0.0.1:8091", - "phaseCounts": { - "fetch": 24024, - "primaryScan": 24024, - "sort": 24024 - }, - "phaseOperators": { - "authorize": 1, - "fetch": 1, - "primaryScan": 1, - "sort": 1 - }, - "phaseTimes": { - "authorize": "23.037µs", - "fetch": "3.092306635s", - "instantiate": "7.313569ms", - "parse": "579.368µs", - "plan": "2.449143ms", - "primaryScan": "153.686873ms", - "run": "15.29433203s", - "sort": "147.889352ms" - }, - "remoteAddr": "172.17.0.1:56900", - "requestId": "5eefc9e5-bdaa-4824-bcd7-47977eb1f08a", - "requestTime": "2021-04-30T21:13:58.707Z", - "resultCount": 24024, - "resultSize": 81821919, - "scanConsistency": "unbounded", - "serviceTime": "15.30510306s", - "state": "completed", - "statement": "SELECT * FROM `travel-sample`.inventory.route ORDER BY sourceairport;", - "useCBO": true, - "userAgent": "Go-http-client/1.1 (CBQ/2.0)", - "users": "Administrator" +[ + { + // ... + "plan": { + "#operator": "Authorize", + "#stats": { + "#phaseSwitches": 4, + "execTime": "1.725µs", + "servTime": "21.312µs" + }, + "privileges": { + "List": [ + { + "Priv": 7, + "Props": 0, + "Target": "default:travel-sample.inventory.route" + } + ] }, - "plan": { - "#operator": "Authorize", + "~child": { + "#operator": "Sequence", "#stats": { - "#phaseSwitches": 4, - "execTime": "1.725µs", - "servTime": "21.312µs" - }, - "privileges": { - "List": [ - { - "Priv": 7, - "Props": 0, - "Target": "default:travel-sample.inventory.route" - } - ] + "#phaseSwitches": 2, + "execTime": "1.499µs" }, - "~child": { - "#operator": "Sequence", - "#stats": { - "#phaseSwitches": 2, - "execTime": "1.499µs" - }, - "~children": [ - { - "#operator": "PrimaryScan3", - "#stats": { - "#heartbeatYields": 6, - "#itemsOut": 24024, - "#phaseSwitches": 96099, - "execTime": "84.366121ms", - "kernTime": "3.021901421s", - "servTime": "69.320752ms" - }, - "bucket": "travel-sample", - "index": "def_inventory_route_primary", - "index_projection": { - "primary_key": true - }, - "keyspace": "route", - "namespace": "default", - "scope": "inventory", - "using": "gsi" + "~children": [ + { + "#operator": "PrimaryScan3", + "#stats": { + "#heartbeatYields": 6, + "#itemsOut": 24024, + "#phaseSwitches": 96099, + "execTime": "84.366121ms", + "kernTime": "3.021901421s", + "servTime": "69.320752ms" }, - { - "#operator": "Fetch", - "#stats": { - "#heartbeatYields": 7258, - "#itemsIn": 24024, - "#itemsOut": 24024, - "#phaseSwitches": 99104, - "execTime": "70.34694ms", - "kernTime": "142.630196ms", - "servTime": "3.021959695s" - }, - "bucket": "travel-sample", - "keyspace": "route", - "namespace": "default", - "scope": "inventory" + "bucket": "travel-sample", + "index": "def_inventory_route_primary", + "index_projection": { + "primary_key": true }, - { - "#operator": "InitialProject", - "#stats": { - "#itemsIn": 24024, - "#itemsOut": 24024, - "#phaseSwitches": 96100, - "execTime": "15.331951ms", - "kernTime": "3.219612458s" - }, - "result_terms": [ - { - "expr": "self", - "star": true - } - ] + "keyspace": "route", + "namespace": "default", + "scope": "inventory", + "using": "gsi" + }, + { + "#operator": "Fetch", + "#stats": { + "#heartbeatYields": 7258, + "#itemsIn": 24024, + "#itemsOut": 24024, + "#phaseSwitches": 99104, + "execTime": "70.34694ms", + "kernTime": "142.630196ms", + "servTime": "3.021959695s" }, - { - "#operator": "Order", - "#stats": { - "#itemsIn": 24024, - "#itemsOut": 24024, - "#phaseSwitches": 72078, - "execTime": "147.889352ms", - "kernTime": "3.229055752s" - }, - "sort_terms": [ - { - "expr": "(`route`.`sourceairport`)" - } - ] + "bucket": "travel-sample", + "keyspace": "route", + "namespace": "default", + "scope": "inventory" + }, + { + "#operator": "InitialProject", + "#stats": { + "#itemsIn": 24024, + "#itemsOut": 24024, + "#phaseSwitches": 96100, + "execTime": "15.331951ms", + "kernTime": "3.219612458s" }, - { - "#operator": "Stream", - "#stats": { - "#itemsIn": 24024, - "#itemsOut": 24024, - "#phaseSwitches": 24025, - "execTime": "11.851634134s" + "result_terms": [ + { + "expr": "self", + "star": true + } + ] + }, + { + "#operator": "Order", + "#stats": { + "#itemsIn": 24024, + "#itemsOut": 24024, + "#phaseSwitches": 72078, + "execTime": "147.889352ms", + "kernTime": "3.229055752s" + }, + "sort_terms": [ + { + "expr": "(`route`.`sourceairport`)" } + ] + }, + { + "#operator": "Stream", + "#stats": { + "#itemsIn": 24024, + "#itemsOut": 24024, + "#phaseSwitches": 24025, + "execTime": "11.851634134s" } - ] - }, - "~versions": [ - "7.0.0-N1QL", - "7.0.0-4960-enterprise" + } ] - } + }, + "~versions": [ + "7.0.0-N1QL", + "7.0.0-4960-enterprise" + ] } - ], - "status": "success", - "metrics": { - "elapsedTime": "172.831251ms", - "executionTime": "172.586836ms", - "resultCount": 40, - "resultSize": 65181, - "serviceLoad": 12 - }, - "profile": { // <3> - "phaseTimes": { - "authorize": "19.912µs", - "fetch": "123.022426ms", - "instantiate": "29.424µs", - "parse": "6.414711ms", - "plan": "3.19076ms", - "primaryScan": "55.521683ms", - "run": "158.514001ms" - }, - "phaseCounts": { - "fetch": 40, - "primaryScan": 40 - }, - "phaseOperators": { - "authorize": 1, - "fetch": 1, - "primaryScan": 1 - }, - "requestTime": "2021-04-30T21:15:18.104Z", - "servicingHost": "127.0.0.1:8091" } -} +] ---- +==== -This example shows: +ifdef::flag-devex-tools[] +For field names and meanings, see <<_execution_timings,Execution Timings>>. +endif::flag-devex-tools[] +ifndef::flag-devex-tools[] +The query plan contains the following attributes: -<1> The profile attribute with all phases-related statistics for Query 2. -<2> The `meta().plan` with all detailed statistics collected for Query 1. -<3> The profile attribute with all phases-related statistics for this query itself, which is querying the `system:completed_requests` keyspace. -==== +include::n1ql:partial$n1ql-rest-api/query/definitions.adoc[lines=699..749] +endif::flag-devex-tools[] [#sys_my-user-info] == Monitor Your User Info @@ -2106,7 +1866,7 @@ __required__ |Integer |=== -For further details, refer to xref:n1ql:n1ql-language-reference/updatestatistics.adoc[UPDATE STATISTICS]. +For further details, see xref:n1ql:n1ql-language-reference/updatestatistics.adoc[]. [#sys-dictionary-cache] == Monitor Cached Statistics @@ -2168,7 +1928,7 @@ This will result in a list similar to: This catalog contains an array of dictionary caches, one for each keyspace for which optimizer statistics are available. Each dictionary cache gives the same information as the <> catalog. -For further details, refer to xref:n1ql:n1ql-language-reference/updatestatistics.adoc[UPDATE STATISTICS]. +For further details, see xref:n1ql:n1ql-language-reference/updatestatistics.adoc[]. [#sys-functions] == Monitor Functions @@ -2738,6 +2498,196 @@ __required__ Refer to xref:n1ql:n1ql-language-reference/transactions.adoc[{sqlpp} Support for Couchbase Transactions] for more information. +[#sys-sequences] +== Monitor Sequences + +The `system:sequences` catalog maintains a list of loaded sequences on any node: that is, sequences that have been accessed since the last restart. +To see the list of loaded sequences, use: + +[source,sqlpp] +---- +SELECT * FROM system:sequences; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "sequences": { + "bucket": "travel-sample", + "cache": 50, + "cycle": false, + "increment": 1, + "max": 9223372036854776000, + "min": -9223372036854776000, + "name": "seq1", + "namespace": "default", + "namespace_id": "default", + "path": "`default`:`travel-sample`.`inventory`.`seq1`", + "scope_id": "inventory", + "value": { + "73428daec3c68d8632ae66b09b70f14d": null, + "~next_block": 0 + } + } + }, +// ... +] +---- + +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**bucket** + +__required__ +|The bucket containing the sequence. +|string + +|**cache** + +__required__ +|The sequence's cache size. +|integer + +|**cycle** + +__required__ +|Whether the sequence is set to cycle. +|boolean + +|**increment** + +__required__ +|The sequence step value. +|integer + +|**min** + +__required__ +|The minimum value permitted for the sequence. +|integer + +|**max** + +__required__ +|The maximum value permitted for the sequence. +|integer + +|**name** + +__required__ +|The name of the sequence. +|string + +|**namespace** + +__required__ +|Namespace to which the sequence belongs. +|string + +|**namespace_id** + +__required__ +|ID of the namespace to which the sequence belongs. +|string + +|**path** + +__required__ +|The fully qualified sequence name. +|string + +|**scope_id** + +__required__ +|ID of the scope to which the sequence belongs. +|string + +|**value** + +__required__ +|The current value of the sequence on each Query node. +| <> object +|=== + +[[value]] +**Values** + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**** + +__required__ +|The name of this property is the UUID of a Query node. + +The value of this property is the current value of the sequence on that node. +|Integer + +|**~next_block** + +__optional__ +|The starting vale of the next block of values that can be reserved for the sequence. +|Integer +|=== + +For further details, see xref:n1ql:n1ql-language-reference/createsequence.adoc[]. + +[#sys-all-sequences] +== Monitor All Sequences + +The `system:all_sequences` catalog maintains a list of all defined sequences. +To see the list of all defined sequences, use: + +[source,sqlpp] +---- +SELECT * FROM system:all_sequences; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "sequences": { + "bucket": "travel-sample", + "cache": 50, + "cycle": false, + "increment": -1, + "max": 9223372036854776000, + "min": 0, + "name": "seq4", + "namespace": "default", + "namespace_id": "default", + "path": "`default`:`travel-sample`.`inventory`.`seq4`", + "scope_id": "inventory", + "value": { + "73428daec3c68d8632ae66b09b70f14d": 10, + "~next_block": -40 + } + } + }, + { + "sequences": { + "bucket": "travel-sample", + "cache": 50, + "cycle": true, + "increment": 5, + "max": 1000, + "min": 0, + "name": "seq3", + "namespace": "default", + "namespace_id": "default", + "path": "`default`:`travel-sample`.`inventory`.`seq3`", + "scope_id": "inventory", + "value": { + "73428daec3c68d8632ae66b09b70f14d": 5, + "~next_block": 255 + } + } + }, +// ... +] +---- + +This catalog gives the same information as the <> catalog. + +For further details, see xref:n1ql:n1ql-language-reference/createsequence.adoc[]. + == Related Links * Refer to xref:n1ql:n1ql-intro/sysinfo.adoc[Getting System Information] for more information on the system namespace. From d547cdb55f7005a31e31f9cf4fd9edd0243be453 Mon Sep 17 00:00:00 2001 From: Simon Dew <39966290+simon-dew@users.noreply.github.com> Date: Thu, 14 Mar 2024 11:38:36 +0000 Subject: [PATCH 3/4] DOC-11437: Add last index scan time to system:indexes (#149) * Update keyspace attributes to use tables * Add system index metadata * Update titles: action first --- modules/n1ql/pages/n1ql-intro/sysinfo.adoc | 352 ++++++++++++++++----- 1 file changed, 275 insertions(+), 77 deletions(-) diff --git a/modules/n1ql/pages/n1ql-intro/sysinfo.adoc b/modules/n1ql/pages/n1ql-intro/sysinfo.adoc index c9d0a3b10..d0ed22e02 100644 --- a/modules/n1ql/pages/n1ql-intro/sysinfo.adoc +++ b/modules/n1ql/pages/n1ql-intro/sysinfo.adoc @@ -71,7 +71,7 @@ The dual catalog has been added for evaluating constant expressions. It contains a single entry with no attributes. [#querying-datastores] -== Querying Datastores +== Query Datastores You can query datastores using the `system:datastores` keyspace as follows: @@ -80,14 +80,25 @@ You can query datastores using the `system:datastores` keyspace as follows: SELECT * FROM system:datastores ---- -The query returns the following attributes: +This catalog contains the following attributes: -[horizontal] -`id`:: (string) ID of the datastore -`url`:: (string) URL of the datastore instance +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**id** + +__required__ +|ID of the datastore. +|String + +|**url** + +__required__ +|URL of the datastore instance. +|String +|=== [#querying-namespaces] -== Querying Namespaces +== Query Namespaces You can query namespaces using the `system:namespaces` keyspace as follows: @@ -96,15 +107,30 @@ You can query namespaces using the `system:namespaces` keyspace as follows: SELECT * FROM system:namespaces ---- -The query returns the following attributes: +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**id** + +__required__ +|ID of the namespace. +|String -[horizontal] -`id`:: (string) ID of the namespace -`name`:: (string) Name of the namespace -`datastore_id`:: (string) ID of the datastore to which the namespace belongs +|**name** + +__required__ +|Name of the namespace. +|String + +|**datastore_id** + +__required__ +|ID of the datastore to which the namespace belongs. +|String +|=== [#querying-buckets] -== Querying Buckets +== Query Buckets You can query buckets using the `system:buckets` keyspace as follows: @@ -113,17 +139,40 @@ You can query buckets using the `system:buckets` keyspace as follows: SELECT * FROM system:buckets ---- -The query returns the following attributes: +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**datastore_id** + +__required__ +|ID of the datastore to which the bucket belongs. +|String -[horizontal] -`datastore_id`:: (string) ID of the datastore to which the bucket belongs -`name`:: (string) Name of the bucket -`namespace`:: (string) Namespace to which the bucket belongs -`namespace_id`:: (string) ID of the namespace to which the bucket belongs -`path`:: (string) Path of the bucket +|**name** + +__required__ +|Name of the bucket. +|String + +|**namespace** + +__required__ +|Namespace to which the bucket belongs. +|String + +|**namespace_id** + +__required__ +|ID of the namespace to which the bucket belongs. +|String + +|**path** + +__required__ +|Path of the bucket. +|String +|=== [#querying-scopes] -== Querying Scopes +== Query Scopes You can query scopes using the `system:scopes` keyspace as follows: @@ -132,21 +181,48 @@ You can query scopes using the `system:scopes` keyspace as follows: SELECT * FROM system:scopes ---- -The query returns the following attributes: +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**bucket** + +__required__ +|Bucket to which the scope belongs. +|String + +|**datastore_id** + +__required__ +|ID of the datastore to which the scope belongs. +|String -[horizontal] -`bucket`:: (string) Bucket to which the scope belongs -`datastore_id`:: (string) ID of the datastore to which the scope belongs -`name`:: (string) Name of the scope -`namespace`:: (string) Namespace to which the scope belongs -`namespace_id`:: (string) ID of the namespace to which the scope belongs -`path`:: (string) Path of the scope +|**name** + +__required__ +|Name of the scope. +|String + +|**namespace** + +__required__ +|Namespace to which the scope belongs. +|String + +|**namespace_id** + +__required__ +|ID of the namespace to which the scope belongs. +|String + +|**path** + +__required__ +|Path of the scope. +|String +|=== NOTE: Querying `system:scopes` only returns named scopes -- that is, non-default scopes. To return all scopes, including the default scopes, you can query `system:all_scopes`. [#querying-keyspaces] -== Querying Collections +== Query Collections You can query collections using the `system:keyspaces` keyspace as follows: @@ -155,33 +231,72 @@ You can query collections using the `system:keyspaces` keyspace as follows: SELECT * FROM system:keyspaces ---- -For the default collection in the default scope, the query returns the following attributes: +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**bucket** + +__optional__ +|For a named, non-default collection: +Bucket to which the keyspace belongs. +|String + +|**datastore_id** + +__required__ +|ID of the datastore to which the keyspace belongs. +|String + +|**id** + +__required__ +|For the default collection in the default scope: +ID of the bucket to which the keyspace belongs. + +''' + +For a named, non-default collection: +ID of the keyspace. +|String + +|**name** + +__required__ +|For the default collection in the default scope: +Bucket to which the keyspace belongs. + +''' + +For a named, non-default collection: +Name of the keyspace. +|String + +|**namespace** + +__required__ +|Namespace to which the keyspace belongs. +|String -[horizontal] -`datastore_id`:: (string) ID of the datastore to which the keyspace belongs -`id`:: (string) ID of the bucket to which the keyspace belongs -`name`:: (string) Bucket to which the keyspace belongs -`namespace`:: (string) Namespace to which the keyspace belongs -`namespace_id`:: (string) ID of the namespace to which the keyspace belongs -`path`:: (string) Path of the keyspace +|**namespace_id** + +__required__ +|ID of the namespace to which the keyspace belongs. +|String -For a named, non-default collection, the query returns the following attributes: +|**path** + +__required__ +|Path of the keyspace. +|String -[horizontal] -`bucket`:: (string) Bucket to which the keyspace belongs -`datastore_id`:: (string) ID of the datastore to which the keyspace belongs -`id`:: (string) ID of the keyspace -`name`:: (string) Name of the keyspace -`namespace`:: (string) Namespace to which the keyspace belongs -`namespace_id`:: (string) ID of the namespace to which the keyspace belongs -`path`:: (string) Path of the keyspace -`scope`:: (string) Scope to which the keyspace belongs +|**scope** + +__optional__ +|For a named, non-default collection: +Scope to which the keyspace belongs. +|String +|=== NOTE: Querying `system:keyspaces` only returns non-system keyspaces. To return all keyspaces, including the system keyspaces, you can query `system:all_keyspaces`. [#querying-indexes] -== Querying Indexes +== Query Indexes You can query indexes using the `system:indexes` keyspace as follows: @@ -190,40 +305,123 @@ You can query indexes using the `system:indexes` keyspace as follows: SELECT * FROM system:indexes ---- -For an index on the default collection in the default scope, the query returns the following attributes: - -[horizontal] -`condition`:: (string) Index filter, if present -`datastore_id`:: (string) ID of the datastore to which the index belongs -`id`:: (string) ID of the index -`index_key`:: (array of strings) List of index keys -`is_primary`:: (boolean) True if the index is a primary index -`keyspace_id`:: (string) ID of the bucket to which the index belongs -`name`:: (string) Name of the index -`namespace_id`:: (string) ID of the namespace to which the index belongs -`state`:: (string) State of index, for example, online -`using`:: (string) Type of index, for example, gsi - -For an index on a named, non-default collection, the query returns the following attributes: - -[horizontal] -`bucket_id`:: (string) ID of the bucket to which the index belongs -`condition`:: (string) Index filter, if present -`datastore_id`:: (string) ID of the datastore to which the index belongs -`id`:: (string) ID of the index -`index_key`:: (array of strings) List of index keys -`is_primary`:: (boolean) True if the index is a primary index -`keyspace_id`:: (string) ID of the keyspace to which the index belongs -`name`:: (string) Name of the index -`namespace_id`:: (string) ID of the namespace to which the index belongs -`state`:: (string) State of index, for example, online -`using`:: (string) Type of index, for example, gsi +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**bucket_id** + +__optional__ +|For an index on a named, non-default collection: +ID of the bucket to which the index belongs. +|String + +|**condition** + +__optional__ +|Index filter, if present. +|String + +|**datastore_id** + +__required__ +|ID of the datastore to which the index belongs. +|String + +|**id** + +__required__ +|ID of the index. +|String + +|**index_key** + +__required__ +|List of index keys. +|String array + +|**is_primary** + +__required__ +|True if the index is a primary index. +|Boolean + +|**keyspace_id** + +__required__ +|For an index on the default collection in the default scope: +ID of the bucket to which the index belongs. + +''' + +For an index on a named, non-default collection: +ID of the keyspace to which the index belongs. +|String + +|**name** + +__required__ +|Name of the index. +|String + +|**metadata** + +__required__ +|Metadata for the index. +|<> object + +|**namespace_id** + +__required__ +|ID of the namespace to which the index belongs. +|String + +|**state** + +__required__ +|State of index. + +*Example*: `online` +|String + +|**using** + +__required__ +|Type of index. + +*Example*: `gsi` +|String +|=== + +[[metadata]] +**Metadata** +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**last_scan_time** + +__required__ +|The last scan timestamp of the index. +|String + +|**num_replica** + +__required__ +|The index replica count. +|String + +|**stats** + +__required__ +|Statistics for the index. +|<> object +|=== + +[[stats]] +**Stats** +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**last_known_scan_time** + +__required__ +|The index last scan time from the indexer, in UNIX Epoch format. +|Number +|=== NOTE: Querying `system:indexes` only returns indexes on non-system keyspaces. To return all indexes, including indexes on system keyspaces, you can query `system:all_indexes`. [#querying-dual] -== Querying Dual +== Query Dual You can use dual to evaluate constant expressions. From 3d5755d3fbb8dfd9a84841d7ce4dea1db39cba20 Mon Sep 17 00:00:00 2001 From: Simon Dew Date: Wed, 29 May 2024 00:46:36 +0100 Subject: [PATCH 4/4] =?UTF-8?q?DOC-11228:=20IA=20Query=20Rework=20?= =?UTF-8?q?=E2=80=94=20Getting=20System=20Information=20page=20(#224)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Move non-monitoring catalogs to the system information page * Move Logical Hierarchy from System Information to Query Concepts * Add Profiling Summary table * Move table of catalogs from Manage and Monitor Queries to System Info * Copy Authentication from Manage and Monitor Queries to System Information * Update intro and reinstate information on how to access the keyspaces * Tidy up Query Concepts * Tidy up Get System Information * Update links to system catalogs * Update links to logical hierarchy * Update N1QL Management overview page * Add table styles to Manage and Monitor Queries --- modules/guides/pages/cbo.adoc | 2 +- .../pages/n1ql-intro/queriesandresults.adoc | 68 +- modules/n1ql/pages/n1ql-intro/sysinfo.adoc | 1292 +++++++++++++++- .../n1ql-language-reference/alterindex.adoc | 6 +- .../altersequence.adoc | 3 +- .../n1ql-language-reference/build-index.adoc | 2 +- .../createcollection.adoc | 2 +- .../createfunction.adoc | 4 +- .../n1ql-language-reference/createindex.adoc | 6 +- .../createprimaryindex.adoc | 2 +- .../n1ql-language-reference/createscope.adoc | 6 +- .../createsequence.adoc | 3 +- .../pages/n1ql-language-reference/delete.adoc | 7 +- .../dropcollection.adoc | 2 +- .../n1ql-language-reference/dropfunction.adoc | 2 +- .../n1ql-language-reference/dropindex.adoc | 2 +- .../dropprimaryindex.adoc | 2 +- .../n1ql-language-reference/dropscope.adoc | 6 +- .../n1ql-language-reference/dropsequence.adoc | 5 +- .../n1ql-language-reference/execfunction.adoc | 2 +- .../explainfunction.adoc | 2 +- .../pages/n1ql-language-reference/from.adoc | 2 +- .../pages/n1ql-language-reference/grant.adoc | 6 +- .../pages/n1ql-language-reference/infer.adoc | 7 +- .../pages/n1ql-language-reference/insert.adoc | 14 +- .../pages/n1ql-language-reference/merge.adoc | 2 +- .../pages/n1ql-language-reference/revoke.adoc | 7 +- .../n1ql-language-reference/sequenceops.adoc | 5 +- .../statistics-delete.adoc | 2 +- .../statistics-expressions.adoc | 2 +- .../statistics-index.adoc | 2 +- .../statistics-indexes.adoc | 2 +- .../n1ql-language-reference/transactions.adoc | 6 +- .../pages/n1ql-language-reference/update.adoc | 7 +- .../pages/n1ql-language-reference/upsert.adoc | 2 +- .../n1ql-language-reference/userfun.adoc | 2 +- modules/n1ql/pages/n1ql-manage/index.adoc | 4 +- .../n1ql-manage/monitoring-n1ql-query.adoc | 1349 ++--------------- 38 files changed, 1465 insertions(+), 1382 deletions(-) diff --git a/modules/guides/pages/cbo.adoc b/modules/guides/pages/cbo.adoc index 9720fc75e..c4ad754de 100644 --- a/modules/guides/pages/cbo.adoc +++ b/modules/guides/pages/cbo.adoc @@ -225,7 +225,7 @@ Reference: Administrator guides: * xref:clusters:query-service/query-workbench.adoc#query-settings[Modify Query Settings] -* xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-dictionary[Monitor Statistics] +* xref:n1ql:n1ql-intro/sysinfo.adoc#sys-dictionary[Monitor Statistics] Querying with SDKs: diff --git a/modules/n1ql/pages/n1ql-intro/queriesandresults.adoc b/modules/n1ql/pages/n1ql-intro/queriesandresults.adoc index 2784df3d2..15fd0397b 100644 --- a/modules/n1ql/pages/n1ql-intro/queriesandresults.adoc +++ b/modules/n1ql/pages/n1ql-intro/queriesandresults.adoc @@ -72,18 +72,62 @@ SELECT name, brewery_id from `beer-sample` WHERE brewery_id IS NOT MISSING LIMIT } ---- -[#keyspace-reference] -== Keyspace References +[#logical-hierarchy] +== Logical Hierarchy Couchbase stores data within a xref:clusters:data-service/about-buckets-scopes-collections.adoc[logical hierarchy] of buckets, scopes, and collections. This enables separation between documents of different types. + +Datastores:: + +Datastores are similar to sites. +A datastore is a database deployment, for example, a server cluster, cloud service, or mobile installation. +It is analogous to a relation database instance. + +Namespaces:: + +Namespaces are similar to pools. +A namespace is a unit of authorization, resource allocation, and tenancy. +It is analogous to a relational database or schema. +Currently, only the `default` and `system` namespaces are available. + +Buckets:: + +A bucket is the fundamental space for storing data in Couchbase Capella. +Each bucket contains at least one scope by default, and you can create more scopes as required. +It is analogous to a relational database table space or file group. + +Scopes:: + +A scope is a set of one or more collections. +Each scope contains at least one collection by default, and you can create more collections as required. +It is analogous to a group of relational database tables. + +Collections:: + +A collection is a set of documents that may vary in structure. +It is a unit of authorization and resource allocation. +It is analogous to a relational database table. + +Keyspaces:: + +Keyspaces map to collections in Couchbase Capella. +You must refer to a keyspace using a <>. ++ +The term keyspace is also used to refer to any of the catalogs in the xref:n1ql:n1ql-intro/sysinfo.adoc[system] namespace. + +[#keyspace-reference] +== Keyspace References + For most queries, you must provide one or more keyspace references to specify the data sources for the query. A keyspace reference may be: -* A _full_ keyspace reference: a path comprising the bucket, the scope, and the collection which contains the documents you want. +* A [dfn]#full keyspace reference#: a path comprising the optional namespace, the bucket, the scope, and the collection which contains the documents you want. +For example, `default:{backtick}travel-sample{backtick}.inventory.airline`. -* A _partial_ keyspace reference, comprising the collection name only. +* A [dfn]#partial keyspace reference#, comprising the collection name only. +For example, `airline`. When a query contains partial keyspace references, you must set the <> to specify a bucket and scope before running a query. Partial keyspace references are resolved using the bucket and scope supplied by the query context. @@ -94,7 +138,7 @@ In this case, the <> must not be set. [#query-context] == Query Context -The [def]_query context_ specifies the the namespace, bucket, and scope used to resolve partial keyspace references. +The [dfn]#query context# specifies the the namespace, bucket, and scope used to resolve partial keyspace references. When the query context is set, you can refer to a collection using just the collection name, without having to enter the keyspace path. When the query context is not set, it defaults to the `default` namespace, with no bucket, scope, or collection. @@ -113,10 +157,10 @@ This can be used to support the separation of tenant data in a multi-tenancy env -- [#paths] -== Paths +== Sub-Document Paths One of the main differences between JSON and flat rows is that JSON supports a nested structure, allowing documents to contain other documents, also known as sub-documents. -{sqlpp} provides [.term]_paths_ to support nested data. +{sqlpp} provides [dfn]#sub-document paths# to support nested data. Paths use dot notation syntax to identify the logical location of an attribute within a document. For example, to get the street from a customer order, use the path `orders.billTo.street`. This path refers to the value for `street` in the `billTo` object. @@ -160,13 +204,13 @@ the Query tab. For more information and examples, refer to xref:n1ql:n1ql-manage/query-settings.adoc#section_srh_tlm_n1b[Named Parameters and Positional Parameters]. [#prepare-stmts] -== Query Optimization Using Prepared Statements +== Prepared Statements When a {sqlpp} query is sent to the server, the server inspects the query and parses it, planning which indexes to use, if any. -Once this is done, it generates a _query plan_. +Once this is done, it generates a [dfn]#query plan#. The computation for the plan adds some additional processing time and overhead for the query. -A frequently-used query can be _prepared_ so that its plan is generated only once. +You can [dfn]#prepare# a frequently-used query so that its plan is generated only once. Subsequent queries using the same query will use the pre-generated plan instead, saving on the overhead and processing of the plan each time. NOTE: Parameterized queries are considered the same query for caching and planning purposes, even if the supplied parameters are different. @@ -181,10 +225,10 @@ This allows specific data (for example, specific fields) to be retrieved quickly The Index service enables you to create two types of index: primary indexes and global secondary indexes. -* You can define a _primary index_ on a keyspace. +* You can define a [dfn]#primary index# on a keyspace. Primary indexes are based on the unique key of every item in a specified collection. A primary index is intended to be used for simple queries, which have no filters or predicates. -* You can also create a _secondary index_ on specific fields in a keyspace. +* You can also create a [dfn]#secondary index# on specific fields in a keyspace. Secondary indexes, often referred to as Global Secondary Indexes or GSIs, constitute the principal means of indexing documents to be accessed by the Query service. + For example, creating a secondary index on the `name` and `email` fields in the `users` keyspace would allow you to query the keyspace regarding a document's `name` or `email` properties. diff --git a/modules/n1ql/pages/n1ql-intro/sysinfo.adoc b/modules/n1ql/pages/n1ql-intro/sysinfo.adoc index d0ed22e02..e5622b50f 100644 --- a/modules/n1ql/pages/n1ql-intro/sysinfo.adoc +++ b/modules/n1ql/pages/n1ql-intro/sysinfo.adoc @@ -1,74 +1,66 @@ = Get System Information :page-topic-type: concept -:description: {sqlpp} has a system namespace that stores metadata about data containers. \ -You can query the system namespace to get information about the data containers. +:description: {sqlpp} has a system namespace that stores metadata about data containers, the Query service, and the system as a whole. \ +You can query the system namespace to get this information. [abstract] {description} include::ROOT:partial$component-signpost.adoc[] -There is a catalog for each type of data container. -The catalog names are plural in order to avoid conflicting with {sqlpp} keywords. +Within the `system` namespace, the following catalogs are provided. +Most catalog names are plural in order to avoid conflicting with {sqlpp} keywords. -== Logical Hierarchy - -With the `system` namespace, you can get information about following types of data container: - -Datastores:: - -Datastores are similar to sites. -A datastore is a database deployment, for example, a server cluster, cloud service, or mobile installation. -It is analogous to a relation database instance. - -Namespaces:: - -Namespaces are similar to pools. -A namespace is a unit of authorization, resource allocation, and tenancy. -It is analogous to a relational database or schema. -Currently, only the `default` and `system` namespaces are available. - -Buckets:: - -A bucket is the fundamental space for storing data in Couchbase Capella. -Each bucket contains at least one scope by default, and you can create more scopes as required. -It is analogous to a relational database table space or file group. - -Scopes:: - -A scope is a set of one or more collections. -Each scope contains at least one collection by default, and you can create more collections as required. -It is analogous to a group of relational database tables. - -Collections:: - -A collection is a set of documents that may vary in structure. -It is a unit of authorization and resource allocation. -It is analogous to a relational database table. - -Keyspaces:: +[cols="4"] +|=== +| Data Containers | Monitoring Catalogs | Security Catalogs | Other + +a| [%hardbreaks] +<> +<> +<> +<> +<> +<> +<> + +a| [%hardbreaks] +xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#vitals[system:vitals] +xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-active-req[system:active_requests] +xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-prepared[system:prepareds] +xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-completed-req[system:completed_requests] + +a| [%hardbreaks] +<> +<> +<> +<> + +a| [%hardbreaks] +<> +<> +<> +<> +<> +<> +<> +<> +|=== -Keyspaces map to collections in Couchbase Capella. -A keyspace may be referred to using a full keyspace reference. -For example, `default:{backtick}travel-sample{backtick}.inventory.airline` refers to the `airline` collection in the `inventory` scope in the `travel-sample` bucket. -A keyspace may also be referred to using a partial keyspace reference, comprising just the collection name. -In this case, the namespace, bucket, and scope must be implied by the current xref:n1ql-intro/queriesandresults.adoc#query-context[query context]. -+ -For compatibility with legacy versions of Couchbase Server, the name of the scope and collection may be omitted when referring to the default collection in the default scope within a bucket. -So for example, we may refer to the `travel-sample` keyspace, meaning the default collection in the default scope in the `travel-sample` bucket. -+ -The term is also used to refer to any of the catalogs in the `system` namespace. +== Authentication and Client Privileges -Indexes:: +Client applications must be authenticated with sufficient privileges to access system keyspaces. -An index on a keyspace. -It is analogous to a relational database index. -Types of indexes include b-tree (ForestDB or MOI) and view indexes. +* Users must have permission to access restricted system keyspaces. +For more details about database credentials, see xref:clusters:manage-database-users.adoc[]. -Dual:: +* In addition, users must have permission to access a bucket, scope, or collection to be able to view that item in the system catalog. +Similarly, users must have SELECT permission on the target of an index to be able to view that index in the system catalog. -The dual catalog has been added for evaluating constant expressions. -It contains a single entry with no attributes. +* The following system keyspaces are considered open, that is, all users can access them without any special privileges: + ** `system:dual` + ** `system:datastores` + ** `system:namespaces` [#querying-datastores] == Query Datastores @@ -432,6 +424,1192 @@ SELECT 2+5 FROM system:dual The query returns the result of the expression, 7 in this case. +[#sys_my-user-info] +== Monitor Your User Info + +The `system:my_user_info` catalog maintains a list of all information of your profile. + +To see your current information, use: + +[source,sqlpp] +---- +SELECT * FROM system:my_user_info; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "my_user_info": { + "domain": "local", + "external_groups": [], + "groups": [], + "id": "jane", + "name": "Jane Doe", + "password_change_date": "2019-05-07T02:31:53.000Z", + "roles": [ + { + "origins": [ + { + "type": "user" + } + ], + "role": "admin" + } + ] + } + } +] +---- + +[#sys-user-info] +== Monitor All User Info + +The `system:user_info` catalog maintains a list of all current users in your bucket and their information. + +To see the list of all current users, use: + +[source,sqlpp] +---- +SELECT * FROM system:user_info; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "user_info": { + "domain": "local", + "external_groups": [], + "groups": [], + "id": "jane", + "name": "Jane Doe", + "password_change_date": "2019-05-07T02:31:53.000Z", + "roles": [ + { + "origins": [ + { + "type": "user" + } + ], + "role": "admin" + } + ] + } + }, + { + "user_info": { + "domain": "ns_server", + "id": "Administrator", + "name": "Administrator", + "roles": [ + { + "role": "admin" + } + ] + } + } +] +---- + +[#sys-nodes] +== Monitor Nodes + +The `system:nodes` catalog shows the datastore topology information. +This is separate from the Query clustering view, in that Query clustering shows a map of the Query cluster, as provided by the cluster manager, while `system:nodes` shows a view of the nodes and services that make up the actual datastore, which may or may not include Query nodes. + +* The dichotomy is important in that Query nodes could be clustered by one entity (e.g. Zookeeper) and be connected to a clustered datastore (e.g. Couchbase) such that each does not have visibility of the other. +* Should {sqlpp} be extended to be able to concurrently connect to multiple datastores, each datastore will report its own topology, so that `system:nodes` offers a complete view of all the storage nodes, whatever those may be. +* The `system:nodes` keyspace provides a way to report services advertised by each node as well as services that are actually running. +This is datastore dependent. +* Query clustering is still reported by the `/admin` endpoints. + +To see the list of all current node information, use: + +[source,sqlpp] +---- +SELECT * FROM system:nodes; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "nodes": { + "name": "127.0.0.1:8091", + "ports": { + "cbas": 8095, + "cbasAdmin": 9110, + "cbasCc": 9111, + "cbasSSL": 18095, + "eventingAdminPort": 8096, + "eventingSSL": 18096, + "fts": 8094, + "ftsSSL": 18094, + "indexAdmin": 9100, + "indexHttp": 9102, + "indexHttps": 19102, + "indexScan": 9101, + "indexStreamCatchup": 9104, + "indexStreamInit": 9103, + "indexStreamMaint": 9105, + "kv": 11210, + "kvSSL": 11207, + "n1ql": 8093, + "n1qlSSL": 18093 + }, + "services": [ + "cbas", + "eventing", + "fts", + "index", + "kv", + "n1ql" + ] + } + } +] +---- + +[#sys-app-roles] +== Monitor Applicable Roles + +The `system:applicable_roles` catalog maintains a list of all applicable roles and grantee of each bucket. + +To see the list of all current applicable role information, use: + +[source,sqlpp] +---- +SELECT * FROM system:applicable_roles; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "applicable_roles": { + "grantee": "anil", + "role": "replication_admin" + } + }, + { + "applicable_roles": { + "bucket_name": "travel-sample", + "grantee": "anil", + "role": "select" + } + }, + { + "applicable_roles": { + "bucket_name": "*", + "grantee": "anil", + "role": "select" + } + } +] +---- + +For more examples, take a look at the blog: https://blog.couchbase.com/optimize-n1ql-performance-using-request-profiling/[Optimize {sqlpp} performance using request profiling^]. + +[#sys-dictionary] +== Monitor Statistics + +The `system:dictionary` catalog maintains a list of the on-disk optimizer statistics stored in the `_query` collection within the `_system` scope. + +If you have multiple query nodes, the data retrieved from this catalog will be the same, regardless of the node on which you run the query. + +To see the list of on-disk optimizer statistics, use: + +[source,sqlpp] +---- +SELECT * FROM system:dictionary; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "dictionary": { + "avgDocKeySize": 12, + "avgDocSize": 278, + "bucket": "travel-sample", + "distributionKeys": [ + "airportname", + "faa", + "city" + ], + "docCount": 1968, + "indexes": [ + { + "indexId": "bc3048e87bf84828", + "indexName": "def_inventory_airport_primary", + "indexStats": [ + { + "avgItemSize": 24, + "avgPageSize": 11760, + "numItems": 1968, + "numPages": 4, + "resRatio": 1 + } + ] + }, + // ... + ], + "keyspace": "airport", + "namespace": "default", + "scope": "inventory" + } + }, + // ... +] +---- + +This catalog contains an array of dictionaries, one for each keyspace for which optimizer statistics are available. +Each dictionary gives the following information: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**avgDocKeySize** + +__required__ +|Average doc key size. +|Integer + +|**avgDocSize** + +__required__ +|Average doc size. +|Integer + +|**bucket** + +__required__ +|The bucket for which statistics are available. +|String + +|**keyspace** + +__required__ +|The keyspace for which statistics are available. +|String + +|**namespace** + +__required__ +|The namespace for which statistics are available. +|String + +|**scope** + +__required__ +|The scope for which statistics are available. +|String + +|**distributionKeys** + +__required__ +|Distribution keys for which histograms are available. +|String array + +|**docCount** + +__required__ +|Document count. +|Integer + +|**indexes** + +__required__ +|An array of indexes in this keyspace for which statistics are available. +|<> array + +|**node** + +__required__ +|The query node where this dictionary cache is resident. +|String +|=== + +[[indexes]] +**Indexes** + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**indexId** + +__required__ +|The index ID. +|String + +|**indexName** + +__required__ +|The index name. +|String + +|**indexStats** + +__required__ +|An array of statistics for each index, with one element for each index partition. +|<> array +|=== + +[[indexStats]] +**Index Statistics** + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**avgItemSize** + +__required__ +|Average item size. +|Integer + +|**avgPageSize** + +__required__ +|Average page size. +|Integer + +|**numItems** + +__required__ +|Number of items. +|Integer + +|**numPages** + +__required__ +|Number of pages. +|Integer + +|**resRatio** + +__required__ +|Resident ratio. +|Integer +|=== + +For further details, see xref:n1ql:n1ql-language-reference/updatestatistics.adoc[]. + +[#sys-dictionary-cache] +== Monitor Cached Statistics + +The `system:dictionary_cache` catalog maintains a list of the in-memory cached subset of the optimizer statistics. + +If you have multiple query nodes, the data retrieved from this node shows cached optimizer statistics from all nodes. +Individual nodes may have a different subset of cached information. + +To see the list of in-memory optimizer statistics, use: + +[source,sqlpp] +---- +SELECT * FROM system:dictionary_cache; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "dictionary_cache": { + "avgDocKeySize": 12, + "avgDocSize": 278, + "bucket": "travel-sample", + "distributionKeys": [ + "airportname", + "faa", + "city" + ], + "docCount": 1968, + "indexes": [ + { + "indexId": "bc3048e87bf84828", + "indexName": "def_inventory_airport_primary", + "indexStats": [ + { + "avgItemSize": 24, + "avgPageSize": 11760, + "numItems": 1968, + "numPages": 4, + "resRatio": 1 + } + ] + }, + // ... + ], + "keyspace": "airport", + "namespace": "default", + "node": "172.23.0.3:8091", + "scope": "inventory" + } + }, + // ... +] +---- + +This catalog contains an array of dictionary caches, one for each keyspace for which optimizer statistics are available. +Each dictionary cache gives the same information as the <> catalog. + +For further details, see xref:n1ql:n1ql-language-reference/updatestatistics.adoc[]. + +[#sys-functions] +== Monitor Functions + +The `system:functions` catalog maintains a list of all user-defined functions across all nodes. +To see the list of all user-defined functions, use: + +[source,sqlpp] +---- +SELECT * FROM system:functions; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "functions": { + "definition": { + "#language": "inline", + "expression": "(((`fahrenheit` - 32) * 5) / 9)", + "parameters": [ + "fahrenheit" + ], + "text": "((fahrenheit - 32) * 5/9)" + }, + "identity": { + "bucket": "travel-sample", + "name": "celsius", + "namespace": "default", + "scope": "inventory", + "type": "scope" + } + } + }, + { + "functions": { + "definition": { + "#language": "javascript", + "library": "geohash-js", + "name": "geohash-js", + "object": "calculateAdjacent", + "parameters": [ + "src", + "dir" + ] + }, + "identity": { + "name": "adjacent", + "namespace": "default", + "type": "global" + } + } + }, + // ... +] +---- + +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**definition** + +__required__ +|The definition of the function. +|<> object + +|**identity** + +__required__ +|The identity of the function. +|<> object +|=== + +[[definition]] +**Definition** + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**#language** + +__required__ +|The language of the function. + +*Example*: `inline` +|String + +|**parameters** + +__required__ +|The parameters required by the function. +|String array + +|**expression** + +__optional__ +|For inline functions only: the expression defining the function. +|String + +|**text** + +__optional__ +|For inline functions: the verbatim text of the function. + +''' + +For {sqlpp} managed user-defined functions: the external code defining the function. +|String + +|**library** + +__optional__ +|For external functions only: the library containing the function. +|String + +|**name** + +__optional__ +|For external functions only: the relative name of the library. +|String + +|**object** + +__optional__ +|For external functions only: the object defining the function. +|String +|=== + +[[identity]] +**Identity** + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**name** + +__required__ +|The name of the function. +|String + +|**namespace** + +__required__ +|The namespace of the function. + +*Example*: `default` +|String + +|**type** + +__required__ +|The type of the function. + +*Example*: `global` +|String + +|**bucket** + +__optional__ +|For scoped functions only: the bucket containing the function. +|String + +|**scope** + +__optional__ +|For scoped functions only: the scope containing the function. +|String +|=== + +[#sys-functions-cache] +== Monitor Cached Functions + +The `system:functions_cache` catalog maintains a list of recently-used user-defined functions across all nodes. +The catalog also lists user-defined functions that have been called recently, but do not exist. +To see the list of recently-used user-defined functions, use: + +[source,sqlpp] +---- +SELECT * FROM system:functions_cache; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "functions_cache": { + "#language": "inline", + "avgServiceTime": "3.066847ms", + "expression": "(((`fahrenheit` - 32) * 5) / 9)", + "lastUse": "2022-03-09 00:17:59.60659793 +0000 UTC m=+35951.429537902", + "maxServiceTime": "3.066847ms", + "minServiceTime": "0s", + "name": "celsius", + "namespace": "default", + "node": "127.0.0.1:8091", + "parameters": [ + "fahrenheit" + ], + "scope": "inventory", + "text": "((fahrenheit - 32) * 5/9)", + "type": "scope", + "uses": 1 + } + }, + { + "functions_cache": { + "#language": "javascript", + "avgServiceTime": "56.892636ms", + "lastUse": "2022-03-09 00:15:46.289934029 +0000 UTC m=+35818.007560703", + "library": "geohash-js", + "maxServiceTime": "146.025426ms", + "minServiceTime": "0s", + "name": "geohash-js", + "namespace": "default", + "node": "127.0.0.1:8091", + "object": "calculateAdjacent", + "parameters": [ + "src", + "dir" + ], + "type": "global", + "uses": 4 + } + }, + { + "functions_cache": { + "avgServiceTime": "3.057421ms", + "lastUse": "2022-03-09 00:17:25.396840275 +0000 UTC m=+35917.199008929", + "maxServiceTime": "3.057421ms", + "minServiceTime": "0s", + "name": "notFound", + "namespace": "default", + "node": "127.0.0.1:8091", + "type": "global", + "undefined_function": true, + "uses": 1 + } + } +] +---- + +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**#language** + +__required__ +|The language of the function. + +*Example*: `inline` +|String + +|**name** + +__required__ +|The name of the function. +|String + +|**namespace** + +__required__ +|The namespace of the function. + +*Example*: `default` +|String + +|**parameters** + +__required__ +|The parameters required by the function. +|String array + +|**type** + +__required__ +|The type of the function. + +*Example*: `global` +|String + +|**scope** + +__optional__ +|For scoped functions only: the scope containing the function. +|String + +|**expression** + +__optional__ +|For inline functions only: the expression defining the function. +|String + +|**text** + +__optional__ +|For inline functions: the verbatim text of the function. + +''' + +For {sqlpp} managed user-defined functions: the external code defining the function. +|String + +|**library** + +__optional__ +|For external functions only: the library containing the function. +|String + +|**object** + +__optional__ +|For external functions only: the object defining the function. +|String + +|**avgServiceTime** + +__required__ +|The mean service time for the function. +|String + +|**lastUse** + +__required__ +|The date and time when the function was last used. +|String + +|**maxServiceTime** + +__required__ +|The maximum service time for the function. +|String + +|**minServiceTime** + +__required__ +|The minimum service time for the function. +|String + +|**node** + +__required__ +|The query node where the function is cached. +|String + +|**undefined_function** + +__required__ +|Whether the function exists or is undefined. +|Boolean + +|**uses** + +__required__ +|The number of uses of the function. +|Number +|=== + +Each query node keeps its own cache of recently-used user-defined functions, so you may see the same function listed for multiple nodes. + +[#sys-tasks-cache] +== Monitor Cached Tasks + +The `system:tasks_cache` catalog maintains a list of recently-used scheduled tasks, such as index advisor sessions. +To see the list of recently-used scheduled tasks, use: + +[source,sqlpp] +---- +SELECT * FROM system:tasks_cache; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "tasks_cache": { + "class": "advisor", + "delay": "1h0m0s", + "id": "bcd9f8e4-b324-504c-a98b-ace90dba869f", + "name": "aa7f688a-bf29-438f-888f-eeaead87ca40", + "node": "10.143.192.101:8091", + "state": "scheduled", + "subClass": "analyze", + "submitTime": "2019-09-17 05:18:12.903122381 -0700 PDT m=+8460.550715992" + } + }, + { + "tasks_cache": { + "class": "advisor", + "delay": "5m0s", + "id": "254abec5-5782-543e-9ee0-d07da146b94e", + "name": "ca2cfe56-01fa-4563-8eb0-a753af76d865", + "node": "10.143.192.101:8091", + "results": [ + // ... + ], + "startTime": "2019-09-17 05:03:31.821597725 -0700 PDT m=+7579.469191487", + "state": "completed", + "stopTime": "2019-09-17 05:03:31.963133954 -0700 PDT m=+7579.610727539", + "subClass": "analyze", + "submitTime": "2019-09-17 04:58:31.821230131 -0700 PDT m=+7279.468823737" + } + } +] +---- + +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**class** + +__required__ +|The class of the task. + +*Example*: ``advisor`` +|string + +|**delay** + +__required__ +|The scheduled duration of the task. +|string + +|**id** + +__required__ +|The internal ID of the task. +|string + +|**name** + +__required__ +|The name of the task. +|string + +|**node** + +__required__ +|The node where the task was started. +|string + +|**state** + +__required__ +|The state of the task. + +*Values*: `scheduled`, `cancelled`, `completed` +|string + +|**subClass** + +__required__ +|The subclass of the task. + +*Example*: `analyze` +|string + +|**submitTime** + +__required__ +|The date and time when the task was submitted. +|string + +|**results** + +__optional__ +|Not scheduled tasks: the results of the task. +|Any array + +|**startTime** + +__optional__ +|Not scheduled tasks: the date and time when the task started. +|string (date-time) + +|**stopTime** + +__optional__ +|Not scheduled tasks: the date and time when the task stopped. +|string (date-time) +|=== + +Refer to xref:n1ql:n1ql-language-reference/advisor.adoc[ADVISOR Function] for more information on index advisor sessions. + +[#sys-transactions] +== Monitor Transactions + +The `system:transactions` catalog maintains a list of active Couchbase transactions. +To see the list of active transactions, use: + +[source,sqlpp] +---- +SELECT * FROM system:transactions; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "transactions": { + "durabilityLevel": "majority", + "durabilityTimeout": "2.5s", + "expiryTime": "2021-04-21T12:53:48.598+01:00", + "id": "85aea637-2288-434b-b7c5-413ad8e7c175", + "isolationLevel": "READ COMMITED", + "lastUse": "2021-04-21T12:51:48.598+01:00", + "node": "127.0.0.1:8091", + "numAtrs": 1024, + "scanConsistency": "unbounded", + "status": 0, + "timeout": "2m0s", + "usedMemory": 960, + "uses": 1 + } + // ... + } +] +---- + +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**durabilityLevel** + +__required__ +|Durability level for all mutations within a transaction. +|string + +|**durabilityTimeout** + +__required__ +|Durability timeout per mutation within the transaction. +|string (duration) + +|**expiryTime** + +__required__ +| +|string (date-time) + +|**id** + +__required__ +|The transaction ID. +|string + +|**isolationLevel** + +__required__ +|The isolation level of the transaction. +|string + +|**lastUse** + +__required__ +| +|string (date-time) + +|**node** + +__required__ +|The node where the transaction was started. +|string + +|**numAtrs** + +__required__ +|The total number of active transaction records. +|integer + +|**scanConsistency** + +__required__ +|The transactional scan consistency. +|string + +|**status** + +__required__ +| +|integer + +|**timeout** + +__required__ +|The transaction timeout duration. +|string (duration) + +|**usedMemory** + +__required__ +| +|integer + +|**uses** + +__required__ +| +|integer +|=== + +Refer to xref:n1ql:n1ql-language-reference/transactions.adoc[{sqlpp} Support for Couchbase Transactions] for more information. + +[#sys-sequences] +== Monitor Sequences + +The `system:sequences` catalog maintains a list of loaded sequences on any node: that is, sequences that have been accessed since the last restart. +To see the list of loaded sequences, use: + +[source,sqlpp] +---- +SELECT * FROM system:sequences; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "sequences": { + "bucket": "travel-sample", + "cache": 50, + "cycle": false, + "increment": 1, + "max": 9223372036854776000, + "min": -9223372036854776000, + "name": "seq1", + "namespace": "default", + "namespace_id": "default", + "path": "`default`:`travel-sample`.`inventory`.`seq1`", + "scope_id": "inventory", + "value": { + "73428daec3c68d8632ae66b09b70f14d": null, + "~next_block": 0 + } + } + }, +// ... +] +---- + +This catalog contains the following attributes: + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**bucket** + +__required__ +|The bucket containing the sequence. +|string + +|**cache** + +__required__ +|The sequence's cache size. +|integer + +|**cycle** + +__required__ +|Whether the sequence is set to cycle. +|boolean + +|**increment** + +__required__ +|The sequence step value. +|integer + +|**min** + +__required__ +|The minimum value permitted for the sequence. +|integer + +|**max** + +__required__ +|The maximum value permitted for the sequence. +|integer + +|**name** + +__required__ +|The name of the sequence. +|string + +|**namespace** + +__required__ +|Namespace to which the sequence belongs. +|string + +|**namespace_id** + +__required__ +|ID of the namespace to which the sequence belongs. +|string + +|**path** + +__required__ +|The fully qualified sequence name. +|string + +|**scope_id** + +__required__ +|ID of the scope to which the sequence belongs. +|string + +|**value** + +__required__ +|The current value of the sequence on each Query node. +| <> object +|=== + +[[value]] +**Values** + +[options="header", cols="~a,~a,~a"] +|=== +|Name|Description|Schema + +|**** + +__required__ +|The name of this property is the UUID of a Query node. + +The value of this property is the current value of the sequence on that node. +|Integer + +|**~next_block** + +__optional__ +|The starting vale of the next block of values that can be reserved for the sequence. +|Integer +|=== + +For further details, see xref:n1ql:n1ql-language-reference/createsequence.adoc[]. + +[#sys-all-sequences] +== Monitor All Sequences + +The `system:all_sequences` catalog maintains a list of all defined sequences. +To see the list of all defined sequences, use: + +[source,sqlpp] +---- +SELECT * FROM system:all_sequences; +---- + +This will result in a list similar to: + +[source,json] +---- +[ + { + "sequences": { + "bucket": "travel-sample", + "cache": 50, + "cycle": false, + "increment": -1, + "max": 9223372036854776000, + "min": 0, + "name": "seq4", + "namespace": "default", + "namespace_id": "default", + "path": "`default`:`travel-sample`.`inventory`.`seq4`", + "scope_id": "inventory", + "value": { + "73428daec3c68d8632ae66b09b70f14d": 10, + "~next_block": -40 + } + } + }, + { + "sequences": { + "bucket": "travel-sample", + "cache": 50, + "cycle": true, + "increment": 5, + "max": 1000, + "min": 0, + "name": "seq3", + "namespace": "default", + "namespace_id": "default", + "path": "`default`:`travel-sample`.`inventory`.`seq3`", + "scope_id": "inventory", + "value": { + "73428daec3c68d8632ae66b09b70f14d": 5, + "~next_block": 255 + } + } + }, +// ... +] +---- + +This catalog gives the same information as the <> catalog. + +For further details, see xref:n1ql:n1ql-language-reference/createsequence.adoc[]. + == Related Links * Refer to xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc[] for more information on the system namespace. \ No newline at end of file diff --git a/modules/n1ql/pages/n1ql-language-reference/alterindex.adoc b/modules/n1ql/pages/n1ql-language-reference/alterindex.adoc index 9e9ff8020..0b059e1b9 100644 --- a/modules/n1ql/pages/n1ql-language-reference/alterindex.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/alterindex.adoc @@ -6,10 +6,8 @@ :console-indexes: xref:server:manage:manage-ui/manage-ui.adoc#console-indexes :query-context: xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context :identifiers: xref:n1ql-language-reference/identifiers.adoc - -:sysinfo: xref:n1ql-intro/sysinfo.adoc -:logical-hierarchy: {sysinfo}#logical-hierarchy -:querying-indexes: {sysinfo}#querying-indexes +:logical-hierarchy: xref:n1ql:n1ql-intro/queriesandresults.adoc#logical-hierarchy +:querying-indexes: xref:n1ql-intro/sysinfo.adoc#querying-indexes [abstract] {description} diff --git a/modules/n1ql/pages/n1ql-language-reference/altersequence.adoc b/modules/n1ql/pages/n1ql-language-reference/altersequence.adoc index 51c1f0c8a..23381eaff 100644 --- a/modules/n1ql/pages/n1ql-language-reference/altersequence.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/altersequence.adoc @@ -7,7 +7,7 @@ :authorization-overview: xref:server:learn:security/authorization-overview.adoc :query-context: xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context :identifiers: xref:n1ql-language-reference/identifiers.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy // TEMP include::partial$n1ql-language-reference/horizontal-style.adoc[] @@ -415,3 +415,4 @@ If you didn't do this, the sequence would cycle to the highest possible value, ` * To create a sequence, see xref:n1ql-language-reference/createsequence.adoc[]. * To drop a sequence, see xref:n1ql-language-reference/dropsequence.adoc[]. * To use a sequence in an expression, see xref:n1ql-language-reference/sequenceops.adoc[]. +* To monitor sequences, see xref:n1ql:n1ql-intro/sysinfo.adoc#sys-sequences[Monitor Sequences]. diff --git a/modules/n1ql/pages/n1ql-language-reference/build-index.adoc b/modules/n1ql/pages/n1ql-language-reference/build-index.adoc index c1aaabc6a..f2948ed32 100644 --- a/modules/n1ql/pages/n1ql-language-reference/build-index.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/build-index.adoc @@ -8,7 +8,7 @@ :expression: xref:n1ql-language-reference/index.adoc :selectclause: xref:n1ql-language-reference/selectclause.adoc :subqueries: xref:n1ql-language-reference/subqueries.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :querying-indexes: xref:n1ql-intro/sysinfo.adoc#querying-indexes :keyspace-ref: xref:n1ql-language-reference/createindex.adoc#keyspace-ref diff --git a/modules/n1ql/pages/n1ql-language-reference/createcollection.adoc b/modules/n1ql/pages/n1ql-language-reference/createcollection.adoc index 121a66c63..d4fc0295a 100644 --- a/modules/n1ql/pages/n1ql-language-reference/createcollection.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/createcollection.adoc @@ -5,7 +5,7 @@ :page-partial: :identifier: xref:n1ql-language-reference/identifiers.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :naming-for-scopes-and-collections: xref:server:learn:data/scopes-and-collections.adoc#naming-for-scopes-and-collections :query-context: xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context :scopes-and-collections: xref:server:learn:data/scopes-and-collections.adoc diff --git a/modules/n1ql/pages/n1ql-language-reference/createfunction.adoc b/modules/n1ql/pages/n1ql-language-reference/createfunction.adoc index cb66d5e5c..dc75c71b4 100644 --- a/modules/n1ql/pages/n1ql-language-reference/createfunction.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/createfunction.adoc @@ -29,7 +29,7 @@ For more information about the restrictions and extensions that come with the Co [[context]] === Global Functions and Scoped Functions -You can create user-defined functions at two different levels of the {sqlpp} xref:n1ql-intro/sysinfo.adoc#logical-hierarchy[logical hierarchy]. +You can create user-defined functions at two different levels of the {sqlpp} xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy[logical hierarchy]. * A [def]_global function_ is created within a namespace, at the same level as the buckets within the namespace. When you call a global function, any partial keyspace references within the function definition are resolved against the function's namespace, regardless of the current xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context[query context]. @@ -673,5 +673,5 @@ endif::flag-devex-javascript-udfs[] * To execute a user-defined function, see xref:n1ql-language-reference/execfunction.adoc[]. * To see the execution plan for a user-defined function, see xref:n1ql-language-reference/explainfunction.adoc[]. * To include a user-defined function in an expression, see xref:n1ql-language-reference/userfun.adoc[]. -* To monitor user-defined functions, see xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. +* To monitor user-defined functions, see xref:n1ql:n1ql-intro/sysinfo.adoc#sys-functions[Monitor Functions]. * To drop a user-defined function, see xref:n1ql-language-reference/dropfunction.adoc[]. diff --git a/modules/n1ql/pages/n1ql-language-reference/createindex.adoc b/modules/n1ql/pages/n1ql-language-reference/createindex.adoc index cdcb1ca36..2703f3df4 100644 --- a/modules/n1ql/pages/n1ql-language-reference/createindex.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/createindex.adoc @@ -17,10 +17,8 @@ Secondary indexes contain a filtered or a full set of keys in a given keyspace. :index-partitioning: xref:n1ql-language-reference/index-partitioning.adoc :indexing-meta-info: xref:n1ql-language-reference/indexing-meta-info.adoc :operator-pushdowns: xref:learn:services-and-indexes/indexes/index_pushdowns.adoc#operator-pushdowns - -:sysinfo: xref:n1ql-intro/sysinfo.adoc -:logical-hierarchy: {sysinfo}#logical-hierarchy -:querying-indexes: {sysinfo}#querying-indexes +:logical-hierarchy: xref:n1ql:n1ql-intro/queriesandresults.adoc#logical-hierarchy +:querying-indexes: xref:n1ql-intro/sysinfo.adoc#querying-indexes The `CREATE INDEX` statement allows you to create a secondary index. Secondary indexes contain a filtered or a full set of keys in a given keyspace. diff --git a/modules/n1ql/pages/n1ql-language-reference/createprimaryindex.adoc b/modules/n1ql/pages/n1ql-language-reference/createprimaryindex.adoc index 784ffacff..239148afa 100644 --- a/modules/n1ql/pages/n1ql-language-reference/createprimaryindex.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/createprimaryindex.adoc @@ -9,7 +9,7 @@ Primary indexes contain a full set of keys in a given keyspace. :build-index: xref:n1ql-language-reference/build-index.adoc :identifiers: xref:n1ql-language-reference/identifiers.adoc :query-context: xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :querying-indexes: xref:n1ql-intro/sysinfo.adoc#querying-indexes :index-replication: xref:learn:services-and-indexes/indexes/index-replication.adoc#index-replication :query-settings: xref:server:manage:manage-settings/query-settings.adoc diff --git a/modules/n1ql/pages/n1ql-language-reference/createscope.adoc b/modules/n1ql/pages/n1ql-language-reference/createscope.adoc index 285a7db0f..6b13be527 100644 --- a/modules/n1ql/pages/n1ql-language-reference/createscope.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/createscope.adoc @@ -5,10 +5,10 @@ :page-partial: :identifier: xref:n1ql-language-reference/identifiers.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :naming-for-scopes-and-collections: xref:server:learn:data/scopes-and-collections.adoc#naming-for-scopes-and-collections -:scopes-and-collections: xref:server:learn:data/scopes-and-collections.adoc -:manage-scopes-and-collections: xref:server:manage:manage-scopes-and-collections/manage-scopes-and-collections.adoc +:scopes-and-collections: xref:clusters:data-service/about-buckets-scopes-collections.adoc +:manage-scopes-and-collections: xref:clusters:data-service/scopes-collections.adoc :scopes-and-collections-api: xref:server:rest-api:scopes-and-collections-api.adoc :couchbase-cli-collection-manage: xref:server:cli:cbcli/couchbase-cli-collection-manage.adoc diff --git a/modules/n1ql/pages/n1ql-language-reference/createsequence.adoc b/modules/n1ql/pages/n1ql-language-reference/createsequence.adoc index f86be7339..a70aace8d 100644 --- a/modules/n1ql/pages/n1ql-language-reference/createsequence.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/createsequence.adoc @@ -7,7 +7,7 @@ :authorization-overview: xref:server:learn:security/authorization-overview.adoc :query-context: xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context :identifiers: xref:n1ql-language-reference/identifiers.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy // TEMP include::partial$n1ql-language-reference/horizontal-style.adoc[] @@ -411,3 +411,4 @@ See xref:n1ql-language-reference/sequenceops.adoc[] for detailed examples using * To alter a sequence, see xref:n1ql-language-reference/altersequence.adoc[]. * To drop a sequence, see xref:n1ql-language-reference/dropsequence.adoc[]. * To use a sequence in an expression, see xref:n1ql-language-reference/sequenceops.adoc[]. +* To monitor sequences, see xref:n1ql:n1ql-intro/sysinfo.adoc#sys-sequences[Monitor Sequences]. diff --git a/modules/n1ql/pages/n1ql-language-reference/delete.adoc b/modules/n1ql/pages/n1ql-language-reference/delete.adoc index 3b40c60c9..00b4d01ef 100644 --- a/modules/n1ql/pages/n1ql-language-reference/delete.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/delete.adoc @@ -3,8 +3,8 @@ :page-topic-type: reference :imagesdir: ../../assets/images -:authorization-overview: xref:server:learn:security/authorization-overview.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:authorization-overview: xref:clusters:manage-database-users.adoc +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :returning-clause: xref:n1ql-language-reference/insert.adoc#returning-clause :use-keys-clause: xref:n1ql-language-reference/hints.adoc#use-keys-clause @@ -23,8 +23,7 @@ include::partial$n1ql-language-reference/horizontal-style.adoc[] To execute the DELETE statement, you must have the _Query Delete_ privilege granted on the target keyspace. If the statement has any RETURNING clauses that need data read, then the _Query Select_ privilege is also required on the keyspaces referred in the respective clauses. -For more details about user roles, see -{authorization-overview}[Authorization]. +For more details about user roles, see {authorization-overview}[]. .RBAC Examples [%collapsible] diff --git a/modules/n1ql/pages/n1ql-language-reference/dropcollection.adoc b/modules/n1ql/pages/n1ql-language-reference/dropcollection.adoc index aa7056155..c0c8e7395 100644 --- a/modules/n1ql/pages/n1ql-language-reference/dropcollection.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/dropcollection.adoc @@ -5,7 +5,7 @@ :page-partial: :identifier: xref:n1ql-language-reference/identifiers.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :query-context: xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context :scopes-and-collections: xref:server:learn:data/scopes-and-collections.adoc :manage-scopes-and-collections: xref:server:manage:manage-scopes-and-collections/manage-scopes-and-collections.adoc diff --git a/modules/n1ql/pages/n1ql-language-reference/dropfunction.adoc b/modules/n1ql/pages/n1ql-language-reference/dropfunction.adoc index 0d694570f..43aa153ab 100644 --- a/modules/n1ql/pages/n1ql-language-reference/dropfunction.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/dropfunction.adoc @@ -160,4 +160,4 @@ endif::flag-devex-javascript-udfs[] * To execute a user-defined function, see xref:n1ql-language-reference/execfunction.adoc[]. * To see the execution plan for a user-defined function, see xref:n1ql-language-reference/explainfunction.adoc[]. * To include a user-defined function in an expression, see xref:n1ql-language-reference/userfun.adoc[]. -* To monitor user-defined functions, see xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. +* To monitor user-defined functions, see xref:n1ql:n1ql-intro/sysinfo.adoc#sys-functions[Monitor Functions]. diff --git a/modules/n1ql/pages/n1ql-language-reference/dropindex.adoc b/modules/n1ql/pages/n1ql-language-reference/dropindex.adoc index 613bc0941..578b75eda 100644 --- a/modules/n1ql/pages/n1ql-language-reference/dropindex.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/dropindex.adoc @@ -5,7 +5,7 @@ :authorization-overview: xref:server:learn:security/authorization-overview.adoc :query-context: xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :identifiers: xref:n1ql-language-reference/identifiers.adoc :drop-primary-index: xref:n1ql-language-reference/dropprimaryindex.adoc diff --git a/modules/n1ql/pages/n1ql-language-reference/dropprimaryindex.adoc b/modules/n1ql/pages/n1ql-language-reference/dropprimaryindex.adoc index 6cf28492b..be02765cc 100644 --- a/modules/n1ql/pages/n1ql-language-reference/dropprimaryindex.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/dropprimaryindex.adoc @@ -6,7 +6,7 @@ :roles: xref:server:learn:security/roles.adoc :query-context: xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context :identifiers: xref:n1ql-language-reference/identifiers.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy {description} diff --git a/modules/n1ql/pages/n1ql-language-reference/dropscope.adoc b/modules/n1ql/pages/n1ql-language-reference/dropscope.adoc index 1bdcf7eea..a2cb4b261 100644 --- a/modules/n1ql/pages/n1ql-language-reference/dropscope.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/dropscope.adoc @@ -5,9 +5,9 @@ :page-partial: :identifier: xref:n1ql-language-reference/identifiers.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy -:scopes-and-collections: xref:server:learn:data/scopes-and-collections.adoc -:manage-scopes-and-collections: xref:server:manage:manage-scopes-and-collections/manage-scopes-and-collections.adoc +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy +:scopes-and-collections: xref:clusters:data-service/about-buckets-scopes-collections.adoc +:manage-scopes-and-collections: xref:clusters:data-service/scopes-collections.adoc :scopes-and-collections-api: xref:server:rest-api:scopes-and-collections-api.adoc :couchbase-cli-collection-manage: xref:server:cli:cbcli/couchbase-cli-collection-manage.adoc diff --git a/modules/n1ql/pages/n1ql-language-reference/dropsequence.adoc b/modules/n1ql/pages/n1ql-language-reference/dropsequence.adoc index f09209947..4da496e21 100644 --- a/modules/n1ql/pages/n1ql-language-reference/dropsequence.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/dropsequence.adoc @@ -7,7 +7,7 @@ :authorization-overview: xref:server:learn:security/authorization-overview.adoc :query-context: xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context :identifiers: xref:n1ql-language-reference/identifiers.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy // TEMP include::partial$n1ql-language-reference/horizontal-style.adoc[] @@ -118,4 +118,5 @@ DROP SEQUENCE seq2 IF EXISTS; * To create a sequence, see xref:n1ql-language-reference/createsequence.adoc[]. * To alter a sequence, see xref:n1ql-language-reference/altersequence.adoc[]. -* To use a sequence in an expression, see xref:n1ql-language-reference/sequenceops.adoc[]. \ No newline at end of file +* To use a sequence in an expression, see xref:n1ql-language-reference/sequenceops.adoc[]. +* To monitor sequences, see xref:n1ql:n1ql-intro/sysinfo.adoc#sys-sequences[Monitor Sequences]. diff --git a/modules/n1ql/pages/n1ql-language-reference/execfunction.adoc b/modules/n1ql/pages/n1ql-language-reference/execfunction.adoc index 5d570a638..c822c18f8 100644 --- a/modules/n1ql/pages/n1ql-language-reference/execfunction.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/execfunction.adoc @@ -95,5 +95,5 @@ For examples, refer to xref:n1ql-language-reference/createfunction.adoc#examples * To manage external libraries and external functions, see xref:server:n1ql:n1ql-rest-api/functions.adoc[]. * To see the execution plan for a user-defined function, see xref:n1ql-language-reference/explainfunction.adoc[]. * To include a user-defined function in an expression, see xref:n1ql-language-reference/userfun.adoc[]. -* To monitor user-defined functions, see xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. +* To monitor user-defined functions, see xref:n1ql:n1ql-intro/sysinfo.adoc#sys-functions[Monitor Functions]. * To drop a user-defined function, see xref:n1ql-language-reference/dropfunction.adoc[]. diff --git a/modules/n1ql/pages/n1ql-language-reference/explainfunction.adoc b/modules/n1ql/pages/n1ql-language-reference/explainfunction.adoc index 9264a44dc..6f9af0a76 100644 --- a/modules/n1ql/pages/n1ql-language-reference/explainfunction.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/explainfunction.adoc @@ -140,5 +140,5 @@ include::n1ql:example$utility/explainfunctionjs.jsonc[] * To manage external libraries and external functions, see xref:server:n1ql:n1ql-rest-api/functions.adoc[]. * To execute a user-defined function, see xref:n1ql-language-reference/execfunction.adoc[]. * To include a user-defined function in an expression, see xref:n1ql-language-reference/userfun.adoc[]. -* To monitor user-defined functions, see xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. +* To monitor user-defined functions, see xref:n1ql:n1ql-intro/sysinfo.adoc#sys-functions[Monitor Functions]. * To drop a user-defined function, see xref:n1ql-language-reference/dropfunction.adoc[]. diff --git a/modules/n1ql/pages/n1ql-language-reference/from.adoc b/modules/n1ql/pages/n1ql-language-reference/from.adoc index d1be28d87..ec85641e5 100644 --- a/modules/n1ql/pages/n1ql-language-reference/from.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/from.adoc @@ -5,7 +5,7 @@ :authorization-overview: xref:server:learn:security/authorization-overview.adoc :query-context: xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :selectclause: xref:n1ql-language-reference/selectclause.adoc :join: xref:n1ql-language-reference/join.adoc :nest: xref:n1ql-language-reference/nest.adoc diff --git a/modules/n1ql/pages/n1ql-language-reference/grant.adoc b/modules/n1ql/pages/n1ql-language-reference/grant.adoc index f25231990..3064c39f0 100644 --- a/modules/n1ql/pages/n1ql-language-reference/grant.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/grant.adoc @@ -3,8 +3,8 @@ :page-topic-type: reference :imagesdir: ../../assets/images -:authorization-overview: xref:server:learn:security/authorization-overview.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:authorization-overview: xref:clusters:manage-database-users.adoc +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :keyspace-ref: xref:n1ql-language-reference/createindex.adoc#keyspace-ref [abstract] @@ -25,7 +25,7 @@ For example: `pass:c[DataReader ON `travel-sample`]` + or `pass:c[Query_Select ON `travel-sample`]` NOTE: Only Full Administrators can run the GRANT statement. -For more details about user roles, see {authorization-overview}[Authorization]. +For more details about user roles, see {authorization-overview}[]. == Syntax diff --git a/modules/n1ql/pages/n1ql-language-reference/infer.adoc b/modules/n1ql/pages/n1ql-language-reference/infer.adoc index 5a5d87832..13b5a9708 100644 --- a/modules/n1ql/pages/n1ql-language-reference/infer.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/infer.adoc @@ -3,8 +3,8 @@ :imagesdir: ../../assets/images :page-topic-type: reference -:authorization-overview: xref:server:learn:security/authorization-overview.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:authorization-overview: xref:clusters:manage-database-users.adoc +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :keyspace-ref: xref:n1ql-language-reference/createindex.adoc#keyspace-ref :bucket-analyzer: xref:clusters:query-service/query-workbench.adoc#bucket-analyzer @@ -22,8 +22,7 @@ The Query Workbench in the Couchbase Capella UI (available under the [.ui]*Data == RBAC Privileges To execute the INFER statement, you must have the _Query Select_ privilege granted on the target keyspace. -For more details about user roles, see -{authorization-overview}[Authorization]. +For more details about user roles, see {authorization-overview}[]. For example, to execute <> below, you must have the _Query Select_ privilege on the `route` keyspace. diff --git a/modules/n1ql/pages/n1ql-language-reference/insert.adoc b/modules/n1ql/pages/n1ql-language-reference/insert.adoc index 2f833346a..de4cd136d 100644 --- a/modules/n1ql/pages/n1ql-language-reference/insert.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/insert.adoc @@ -3,11 +3,11 @@ :imagesdir: ../../assets/images :page-topic-type: reference -:authorization-overview: xref:server:learn:security/authorization-overview.adoc +:authorization-overview: xref:clusters:manage-database-users.adoc :bucket-expiration: xref:server:learn:data/expiration.adoc -:roles: xref:server:learn:security/roles.adoc -:install-sample-buckets: xref:server:manage:manage-settings/install-sample-buckets.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:roles: xref:organizations:organization-projects-overview.adoc +:install-sample-buckets: xref:clusters:data-service/import-data-documents.adoc#import-sample-data +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :paths: xref:n1ql-intro/queriesandresults.adoc#paths :explain: xref:n1ql-language-reference/explain.adoc :upsert: xref:n1ql-language-reference/upsert.adoc @@ -46,7 +46,7 @@ TIP: Use the {upsert}[UPSERT] statement if you want to overwrite a document with WARNING: Please note that the examples below will alter the data in your sample buckets. To restore your sample data, remove and reinstall the `travel-sample` bucket. -Refer to xref:server:manage:manage-settings/install-sample-buckets.adoc[Sample Buckets] for details. +Refer to xref:server:manage:manage-settings/install-sample-buckets.adoc[Import Sample Data] for details. [#insert-prerequisites] == Prerequisites @@ -64,7 +64,7 @@ See and for details. You should have read-write permission to the keyspace, to be able to insert documents into a keyspace. Any user who has the keyspace credentials or any Couchbase administrator should be able to insert documents into a keyspace. This includes the keyspace administrator for the specified keyspace, the cluster administrator, and the full administrator roles. -See {roles}[Roles] for details about access privileges for various administrators. +See {roles}[] for details about access privileges for various administrators. WARNING: You cannot insert documents into a SASL bucket if you have a read-only role for the SASL bucket. @@ -73,7 +73,7 @@ WARNING: You cannot insert documents into a SASL bucket if you have a read-only To execute the INSERT statement, you must have the _Query Insert_ privilege on the target keyspace. If the statement has any SELECT or RETURNING data-read clauses, then the _Query Select_ privilege is also required on the keyspaces referred in the respective clauses. -For more details about roles and privileges, see {authorization-overview}[Authorization]. +For more details about roles and privileges, see {authorization-overview}[]. .RBAC Examples [%collapsible] diff --git a/modules/n1ql/pages/n1ql-language-reference/merge.adoc b/modules/n1ql/pages/n1ql-language-reference/merge.adoc index 83197408f..aac22f2eb 100644 --- a/modules/n1ql/pages/n1ql-language-reference/merge.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/merge.adoc @@ -6,7 +6,7 @@ :authorization-overview: xref:server:learn:security/authorization-overview.adoc :bucket-expiration: xref:server:learn:data/expiration.adoc :preserve_expiry: xref:n1ql:n1ql-manage/query-settings.adoc#preserve_expiry -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :selectclause: xref:n1ql-language-reference/selectclause.adoc :subqueries: xref:n1ql-language-reference/subqueries.adoc :expression: xref:n1ql-language-reference/index.adoc diff --git a/modules/n1ql/pages/n1ql-language-reference/revoke.adoc b/modules/n1ql/pages/n1ql-language-reference/revoke.adoc index 85d52edd3..a9f1e978b 100644 --- a/modules/n1ql/pages/n1ql-language-reference/revoke.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/revoke.adoc @@ -3,8 +3,8 @@ :page-topic-type: reference :imagesdir: ../../assets/images -:authorization-overview: xref:server:learn:security/authorization-overview.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:authorization-overview: xref:clusters:manage-database-users.adoc +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :keyspace-ref: xref:n1ql-language-reference/createindex.adoc#keyspace-ref [abstract] @@ -25,8 +25,7 @@ For example: `pass:c[DataReader ON `travel-sample`]` + or `pass:c[Query_Select ON `travel-sample`]` NOTE: Only Full Administrators can run the REVOKE statement. -For more details about user roles, see -{authorization-overview}[Authorization]. +For more details about user roles, see {authorization-overview}[]. == Syntax diff --git a/modules/n1ql/pages/n1ql-language-reference/sequenceops.adoc b/modules/n1ql/pages/n1ql-language-reference/sequenceops.adoc index 71c4ab9e1..9134956d9 100644 --- a/modules/n1ql/pages/n1ql-language-reference/sequenceops.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/sequenceops.adoc @@ -8,7 +8,7 @@ :authorization-overview: xref:server:learn:security/authorization-overview.adoc :query-context: xref:n1ql:n1ql-intro/queriesandresults.adoc#query-context :identifiers: xref:n1ql-language-reference/identifiers.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy // TEMP include::partial$n1ql-language-reference/horizontal-style.adoc[] @@ -396,4 +396,5 @@ GROUP BY o.num, o.user; * To create a sequence, see xref:n1ql-language-reference/createsequence.adoc[]. * To alter a sequence, see xref:n1ql-language-reference/altersequence.adoc[]. -* To drop a sequence, see xref:n1ql-language-reference/dropsequence.adoc[]. \ No newline at end of file +* To drop a sequence, see xref:n1ql-language-reference/dropsequence.adoc[]. +* To monitor sequences, see xref:n1ql:n1ql-intro/sysinfo.adoc#sys-sequences[Monitor Sequences]. diff --git a/modules/n1ql/pages/n1ql-language-reference/statistics-delete.adoc b/modules/n1ql/pages/n1ql-language-reference/statistics-delete.adoc index fe5c89787..adf585c4b 100644 --- a/modules/n1ql/pages/n1ql-language-reference/statistics-delete.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/statistics-delete.adoc @@ -10,7 +10,7 @@ :array-expr: {n1ql}/indexing-arrays.adoc#array-expr :meta-info-expr: {n1ql}/indexing-meta-info.adoc#metakeyspace_expr-property :keyspace-ref: {n1ql}/createindex.adoc#keyspace-ref -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy //Related links :updatestatistics: {n1ql}/updatestatistics.adoc diff --git a/modules/n1ql/pages/n1ql-language-reference/statistics-expressions.adoc b/modules/n1ql/pages/n1ql-language-reference/statistics-expressions.adoc index 988807869..dd43bca56 100644 --- a/modules/n1ql/pages/n1ql-language-reference/statistics-expressions.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/statistics-expressions.adoc @@ -13,7 +13,7 @@ :adaptive-expr: {n1ql}/adaptive-indexing.adoc#index-key :meta-info-expr: {n1ql}/indexing-meta-info.adoc#metakeyspace_expr-property :distribution-stats: {cbo}#distribution-stats -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy //Related links :updatestatistics: {n1ql}/updatestatistics.adoc diff --git a/modules/n1ql/pages/n1ql-language-reference/statistics-index.adoc b/modules/n1ql/pages/n1ql-language-reference/statistics-index.adoc index 919698686..a5e4d550c 100644 --- a/modules/n1ql/pages/n1ql-language-reference/statistics-index.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/statistics-index.adoc @@ -11,7 +11,7 @@ :index-path-alter: {n1ql}/alterindex.adoc#index-path :index-path-drop: {n1ql}/dropindex.adoc#index-path :distribution-stats: {cbo}#distribution-stats -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy //Related links :updatestatistics: {n1ql}/updatestatistics.adoc diff --git a/modules/n1ql/pages/n1ql-language-reference/statistics-indexes.adoc b/modules/n1ql/pages/n1ql-language-reference/statistics-indexes.adoc index 496e6f141..4cc2cbbfa 100644 --- a/modules/n1ql/pages/n1ql-language-reference/statistics-indexes.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/statistics-indexes.adoc @@ -9,7 +9,7 @@ :selectclause: {n1ql}/selectclause.adoc :keyspace-ref: {n1ql}/createindex.adoc#keyspace-ref :distribution-stats: {cbo}#distribution-stats -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy //Related links :updatestatistics: {n1ql}/updatestatistics.adoc diff --git a/modules/n1ql/pages/n1ql-language-reference/transactions.adoc b/modules/n1ql/pages/n1ql-language-reference/transactions.adoc index 8951a0805..282051029 100644 --- a/modules/n1ql/pages/n1ql-language-reference/transactions.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/transactions.adoc @@ -21,8 +21,8 @@ :roles: xref:server:learn:security/roles.adoc // Manage cross-references -:install-sample-buckets: xref:clusters:data-service/import-data-documents.adoc#import-overview -:sys-transactions: xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-transactions +:install-sample-buckets: xref:clusters:data-service/import-data-documents.adoc#import-sample-data +:sys-transactions: xref:n1ql:n1ql-intro/sysinfo.adoc#sys-transactions // Tools cross-references :query-workbench: xref:clusters:query-service/query-workbench.adoc @@ -221,7 +221,7 @@ This worked example guides you through a complete Couchbase transaction session === Preparation The worked example assumes that the supplied `travel-sample` bucket is installed. -Refer to {install-sample-buckets}[Sample Buckets] for installation details. +Refer to {install-sample-buckets}[Import Sample Data] for installation details. .Context For this worked example, set the query context to the `tenant_agent_00` scope in the travel sample dataset. diff --git a/modules/n1ql/pages/n1ql-language-reference/update.adoc b/modules/n1ql/pages/n1ql-language-reference/update.adoc index 873a7e7d7..1f4ef4e39 100644 --- a/modules/n1ql/pages/n1ql-language-reference/update.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/update.adoc @@ -3,9 +3,9 @@ :page-topic-type: reference :imagesdir: ../../assets/images -:authorization-overview: xref:server:learn:security/authorization-overview.adoc +:authorization-overview: xref:clusters:manage-database-users.adoc :bucket-expiration: xref:server:learn:data/expiration.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :document-expiration: xref:java-sdk:howtos:kv-operations.adoc#document-expiration :meta: xref:n1ql-language-reference/metafun.adoc#meta :returning-clause: xref:n1ql-language-reference/insert.adoc#returning-clause @@ -33,8 +33,7 @@ Refer to xref:server:manage:manage-settings/install-sample-buckets.adoc[Sample B User executing the UPDATE statement must have the _Query Update_ privilege on the target keyspace. If the statement has any clauses that needs data read, such as SELECT clause, or RETURNING clause, then _Query Select_ privilege is also required on the keyspaces referred in the respective clauses. -For more details about user roles, see -{authorization-overview}[Authorization]. +For more details about user roles, see {authorization-overview}[]. [NOTE] A user with the _Data Writer_ privilege may set documents to expire. diff --git a/modules/n1ql/pages/n1ql-language-reference/upsert.adoc b/modules/n1ql/pages/n1ql-language-reference/upsert.adoc index 5e016e915..b7a543829 100644 --- a/modules/n1ql/pages/n1ql-language-reference/upsert.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/upsert.adoc @@ -6,7 +6,7 @@ :authorization-overview: xref:server:learn:security/authorization-overview.adoc :bucket-expiration: xref:server:learn:data/expiration.adoc :select-syntax: xref:n1ql-language-reference/select-syntax.adoc -:logical-hierarchy: xref:n1ql-intro/sysinfo.adoc#logical-hierarchy +:logical-hierarchy: xref:n1ql-intro/queriesandresults.adoc#logical-hierarchy :paths: xref:n1ql-intro/queriesandresults.adoc#paths :document-expiration: xref:java-sdk:howtos:kv-operations.adoc#document-expiration :preserve_expiry: xref:n1ql:n1ql-manage/query-settings.adoc#preserve_expiry diff --git a/modules/n1ql/pages/n1ql-language-reference/userfun.adoc b/modules/n1ql/pages/n1ql-language-reference/userfun.adoc index 854718a2c..0ec53dc78 100644 --- a/modules/n1ql/pages/n1ql-language-reference/userfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/userfun.adoc @@ -189,5 +189,5 @@ At the latitude of the selected hotel, each geohash represents an area of approx * To manage external libraries and external functions, see xref:server:n1ql:n1ql-rest-api/functions.adoc[]. * To execute a user-defined function, see xref:n1ql-language-reference/execfunction.adoc[]. * To see the execution plan for a user-defined function, see xref:n1ql-language-reference/explainfunction.adoc[]. -* To monitor user-defined functions, see xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-functions[Monitor Functions]. +* To monitor user-defined functions, see xref:n1ql:n1ql-intro/sysinfo.adoc#sys-functions[Monitor Functions]. * To drop a user-defined function, see xref:n1ql-language-reference/dropfunction.adoc[]. diff --git a/modules/n1ql/pages/n1ql-manage/index.adoc b/modules/n1ql/pages/n1ql-manage/index.adoc index fa329c828..5b6a0ffa5 100644 --- a/modules/n1ql/pages/n1ql-manage/index.adoc +++ b/modules/n1ql/pages/n1ql-manage/index.adoc @@ -19,8 +19,8 @@ include::ROOT:partial$component-signpost.adoc[] == Get System Information -{sqlpp} has a system namespace that stores metadata about data containers. -You can query the system namespace to get information about the data containers. +{sqlpp} has a system namespace that stores metadata about data containers, the Query service, and the system as a whole. +You can query the system namespace to get this information. * xref:n1ql:n1ql-intro/sysinfo.adoc[] diff --git a/modules/n1ql/pages/n1ql-manage/monitoring-n1ql-query.adoc b/modules/n1ql/pages/n1ql-manage/monitoring-n1ql-query.adoc index 309e3ff6e..8ef6e05d8 100644 --- a/modules/n1ql/pages/n1ql-manage/monitoring-n1ql-query.adoc +++ b/modules/n1ql/pages/n1ql-manage/monitoring-n1ql-query.adoc @@ -2,14 +2,47 @@ :example-caption!: :description: Monitoring and profiling {sqlpp} queries, query service nodes, and corresponding system resources is very important for smoother operational performance and efficiency of the system. +ifdef::basebackend-html[] +++++ + +++++ +endif::[] + [abstract] {description} In fact, often it is vital for diagnosing and troubleshooting issues such as query performance, resource bottlenecks, and overloading of various services. include::ROOT:partial$component-signpost.adoc[] -System keyspaces provide various monitoring details and statistics about individual queries and query service. -When running on a cluster with multiple query nodes, stats about all queries on all query nodes are collected in these system keyspaces. +System keyspaces provide various monitoring details and statistics about individual queries and the Query service. +When running on a cluster with multiple query nodes, stats about all queries on all query nodes are collected in the Query management and monitoring system keyspaces. For example, this can help identify: @@ -18,75 +51,24 @@ For example, this can help identify: * Details about the active, completed, and prepared queries. * Find long running queries that are running for more than 2 minutes. -These system keyspaces are like virtual keyspaces that are transient in nature, and are not persisted to disk or permanent storage. -Hence, the information in the keyspaces pertains to the current instantiation of the query service. - -//// -TODO: Reinstate this for monitoring and managing when other content is moved to the System Information page -You can access the keyspaces using any of the following: - -* {sqlpp} language (from the cbq shell or Query Workbench) -* REST API -* Monitoring SDK -//// - -NOTE: All the power of the {sqlpp} query language can be applied on the keyspaces to obtain various insights. +These system keyspaces are transient in nature, and are not persisted to disk or permanent storage. +Hence, the information in the keyspaces pertains to the current instantiation of the Query service. -The following diagnostics are provided: +You can access the Query management and monitoring system keyspaces using any of the following: -[cols="1,3"] -|=== -| System Catalogs -a| [%hardbreaks] -xref:n1ql:n1ql-intro/sysinfo.adoc#querying-datastores[system:datastores] -xref:n1ql:n1ql-intro/sysinfo.adoc#querying-namespaces[system:namespaces] -xref:n1ql:n1ql-intro/sysinfo.adoc#querying-buckets[system:buckets] -xref:n1ql:n1ql-intro/sysinfo.adoc#querying-scopes[system:scopes] -xref:n1ql:n1ql-intro/sysinfo.adoc#querying-keyspaces[system:keyspaces] -xref:n1ql:n1ql-intro/sysinfo.adoc#querying-indexes[system:indexes] -xref:n1ql:n1ql-intro/sysinfo.adoc#querying-dual[system:dual] - -| Monitoring Catalogs -a| [%hardbreaks] -<> -<> -<> -<> +* {sqlpp} from +ifdef::flag-devex-tools[the cbq shell or] +the Query Workbench +* The Query Admin REST API +* A monitoring SDK -| Security Catalogs -a| [%hardbreaks] -<> -<> -<> -<> - -| Other -a| [%hardbreaks] -<> -<> -<> -<> -<> -<> -<> -<> -|=== +Using {sqlpp} enables you to obtain further insights from the keyspaces. == Authentication and Client Privileges -Client applications must be authenticated with sufficient privileges to access system keyspaces. - -* Users must have permission to access restricted system keyspaces. +Users must have permission to access restricted system keyspaces. For more details about database credentials, see xref:clusters:manage-database-users.adoc[]. -* In addition, users must have permission to access a bucket, scope, or collection to be able to view that item in the system catalog. -Similarly, users must have SELECT permission on the target of an index to be able to view that index in the system catalog. - -* The following system keyspaces are considered open, that is, all users can access them without any special privileges: - ** `system:dual` - ** `system:datastores` - ** `system:namespaces` - ifdef::flag-devex-rest-api[] == Examples on this Page @@ -100,7 +82,7 @@ endif::flag-devex-rest-api[] [#vitals] == Monitor System Vitals -The `system:vitals` catalog provides data about the running state and health of the query node, such as number of logical cores, active threads, queued threads, CPU utilization, memory usage, network utilization, garbage collection percentage, and so on. +The `system:vitals` catalog provides data about the running state and health of the query nodes, such as number of logical cores, active threads, queued threads, CPU utilization, memory usage, network utilization, garbage collection percentage, and so on. This information can be very useful to assess the current workload and performance characteristics of a query node. [#sys-vitals-get] @@ -1502,1192 +1484,75 @@ The query plan contains the following attributes: include::n1ql:partial$n1ql-rest-api/query/definitions.adoc[lines=699..749] endif::flag-devex-tools[] -[#sys_my-user-info] -== Monitor Your User Info - -The `system:my_user_info` catalog maintains a list of all information of your profile. - -To see your current information, use: - -[source,sqlpp] ----- -SELECT * FROM system:my_user_info; ----- - -This will result in a list similar to: - -[source,json] ----- -[ - { - "my_user_info": { - "domain": "local", - "external_groups": [], - "groups": [], - "id": "jane", - "name": "Jane Doe", - "password_change_date": "2019-05-07T02:31:53.000Z", - "roles": [ - { - "origins": [ - { - "type": "user" - } - ], - "role": "admin" - } - ] - } - } -] ----- - -[#sys-user-info] -== Monitor All User Info - -The `system:user_info` catalog maintains a list of all current users in your bucket and their information. - -To see the list of all current users, use: - -[source,sqlpp] ----- -SELECT * FROM system:user_info; ----- - -This will result in a list similar to: - -[source,json] ----- -[ - { - "user_info": { - "domain": "local", - "external_groups": [], - "groups": [], - "id": "jane", - "name": "Jane Doe", - "password_change_date": "2019-05-07T02:31:53.000Z", - "roles": [ - { - "origins": [ - { - "type": "user" - } - ], - "role": "admin" - } - ] - } - }, - { - "user_info": { - "domain": "ns_server", - "id": "Administrator", - "name": "Administrator", - "roles": [ - { - "role": "admin" - } - ] - } - } -] ----- - -[#sys-nodes] -== Monitor Nodes - -The `system:nodes` catalog shows the datastore topology information. -This is separate from the Query clustering view, in that Query clustering shows a map of the Query cluster, as provided by the cluster manager, while `system:nodes` shows a view of the nodes and services that make up the actual datastore, which may or may not include Query nodes. - -* The dichotomy is important in that Query nodes could be clustered by one entity (e.g. Zookeeper) and be connected to a clustered datastore (e.g. Couchbase) such that each does not have visibility of the other. -* Should {sqlpp} be extended to be able to concurrently connect to multiple datastores, each datastore will report its own topology, so that `system:nodes` offers a complete view of all the storage nodes, whatever those may be. -* The `system:nodes` keyspace provides a way to report services advertised by each node as well as services that are actually running. -This is datastore dependent. -* Query clustering is still reported by the `/admin` endpoints. - -To see the list of all current node information, use: - -[source,sqlpp] ----- -SELECT * FROM system:nodes; ----- - -This will result in a list similar to: - -[source,json] ----- -[ - { - "nodes": { - "name": "127.0.0.1:8091", - "ports": { - "cbas": 8095, - "cbasAdmin": 9110, - "cbasCc": 9111, - "cbasSSL": 18095, - "eventingAdminPort": 8096, - "eventingSSL": 18096, - "fts": 8094, - "ftsSSL": 18094, - "indexAdmin": 9100, - "indexHttp": 9102, - "indexHttps": 19102, - "indexScan": 9101, - "indexStreamCatchup": 9104, - "indexStreamInit": 9103, - "indexStreamMaint": 9105, - "kv": 11210, - "kvSSL": 11207, - "n1ql": 8093, - "n1qlSSL": 18093 - }, - "services": [ - "cbas", - "eventing", - "fts", - "index", - "kv", - "n1ql" - ] - } - } -] ----- - -[#sys-app-roles] -== Monitor Applicable Roles - -The `system:applicable_roles` catalog maintains a list of all applicable roles and grantee of each bucket. - -To see the list of all current applicable role information, use: - -[source,sqlpp] ----- -SELECT * FROM system:applicable_roles; ----- - -This will result in a list similar to: - -[source,json] ----- -[ - { - "applicable_roles": { - "grantee": "anil", - "role": "replication_admin" - } - }, - { - "applicable_roles": { - "bucket_name": "travel-sample", - "grantee": "anil", - "role": "select" - } - }, - { - "applicable_roles": { - "bucket_name": "*", - "grantee": "anil", - "role": "select" - } - } -] ----- - -For more examples, take a look at the blog: https://blog.couchbase.com/optimize-n1ql-performance-using-request-profiling/[Optimize {sqlpp} performance using request profiling^]. - -[#sys-dictionary] -== Monitor Statistics - -The `system:dictionary` catalog maintains a list of the on-disk optimizer statistics stored in the `_query` collection within the `_system` scope. - -If you have multiple query nodes, the data retrieved from this catalog will be the same, regardless of the node on which you run the query. - -To see the list of on-disk optimizer statistics, use: - -[source,sqlpp] ----- -SELECT * FROM system:dictionary; ----- - -This will result in a list similar to: - -[source,json] ----- -[ - { - "dictionary": { - "avgDocKeySize": 12, - "avgDocSize": 278, - "bucket": "travel-sample", - "distributionKeys": [ - "airportname", - "faa", - "city" - ], - "docCount": 1968, - "indexes": [ - { - "indexId": "bc3048e87bf84828", - "indexName": "def_inventory_airport_primary", - "indexStats": [ - { - "avgItemSize": 24, - "avgPageSize": 11760, - "numItems": 1968, - "numPages": 4, - "resRatio": 1 - } - ] - }, - // ... - ], - "keyspace": "airport", - "namespace": "default", - "scope": "inventory" - } - }, - // ... -] ----- - -This catalog contains an array of dictionaries, one for each keyspace for which optimizer statistics are available. -Each dictionary gives the following information: - -[options="header", cols="~a,~a,~a"] -|=== -|Name|Description|Schema - -|**avgDocKeySize** + -__required__ -|Average doc key size. -|Integer - -|**avgDocSize** + -__required__ -|Average doc size. -|Integer - -|**bucket** + -__required__ -|The bucket for which statistics are available. -|String - -|**keyspace** + -__required__ -|The keyspace for which statistics are available. -|String - -|**namespace** + -__required__ -|The namespace for which statistics are available. -|String - -|**scope** + -__required__ -|The scope for which statistics are available. -|String - -|**distributionKeys** + -__required__ -|Distribution keys for which histograms are available. -|String array - -|**docCount** + -__required__ -|Document count. -|Integer - -|**indexes** + -__required__ -|An array of indexes in this keyspace for which statistics are available. -|<> array - -|**node** + -__required__ -|The query node where this dictionary cache is resident. -|String -|=== - -[[indexes]] -**Indexes** - -[options="header", cols="~a,~a,~a"] -|=== -|Name|Description|Schema - -|**indexId** + -__required__ -|The index ID. -|String - -|**indexName** + -__required__ -|The index name. -|String - -|**indexStats** + -__required__ -|An array of statistics for each index, with one element for each index partition. -|<> array -|=== - -[[indexStats]] -**Index Statistics** - -[options="header", cols="~a,~a,~a"] -|=== -|Name|Description|Schema - -|**avgItemSize** + -__required__ -|Average item size. -|Integer - -|**avgPageSize** + -__required__ -|Average page size. -|Integer - -|**numItems** + -__required__ -|Number of items. -|Integer - -|**numPages** + -__required__ -|Number of pages. -|Integer - -|**resRatio** + -__required__ -|Resident ratio. -|Integer -|=== - -For further details, see xref:n1ql:n1ql-language-reference/updatestatistics.adoc[]. - -[#sys-dictionary-cache] -== Monitor Cached Statistics - -The `system:dictionary_cache` catalog maintains a list of the in-memory cached subset of the optimizer statistics. - -If you have multiple query nodes, the data retrieved from this node shows cached optimizer statistics from all nodes. -Individual nodes may have a different subset of cached information. - -To see the list of in-memory optimizer statistics, use: - -[source,sqlpp] ----- -SELECT * FROM system:dictionary_cache; ----- - -This will result in a list similar to: - -[source,json] ----- -[ - { - "dictionary_cache": { - "avgDocKeySize": 12, - "avgDocSize": 278, - "bucket": "travel-sample", - "distributionKeys": [ - "airportname", - "faa", - "city" - ], - "docCount": 1968, - "indexes": [ - { - "indexId": "bc3048e87bf84828", - "indexName": "def_inventory_airport_primary", - "indexStats": [ - { - "avgItemSize": 24, - "avgPageSize": 11760, - "numItems": 1968, - "numPages": 4, - "resRatio": 1 - } - ] - }, - // ... - ], - "keyspace": "airport", - "namespace": "default", - "node": "172.23.0.3:8091", - "scope": "inventory" - } - }, - // ... -] ----- - -This catalog contains an array of dictionary caches, one for each keyspace for which optimizer statistics are available. -Each dictionary cache gives the same information as the <> catalog. - -For further details, see xref:n1ql:n1ql-language-reference/updatestatistics.adoc[]. - -[#sys-functions] -== Monitor Functions - -The `system:functions` catalog maintains a list of all user-defined functions across all nodes. -To see the list of all user-defined functions, use: - -[source,sqlpp] ----- -SELECT * FROM system:functions; ----- - -This will result in a list similar to: - -[source,json] ----- -[ - { - "functions": { - "definition": { - "#language": "inline", - "expression": "(((`fahrenheit` - 32) * 5) / 9)", - "parameters": [ - "fahrenheit" - ], - "text": "((fahrenheit - 32) * 5/9)" - }, - "identity": { - "bucket": "travel-sample", - "name": "celsius", - "namespace": "default", - "scope": "inventory", - "type": "scope" - } - } - }, - { - "functions": { - "definition": { - "#language": "javascript", - "library": "geohash-js", - "name": "geohash-js", - "object": "calculateAdjacent", - "parameters": [ - "src", - "dir" - ] - }, - "identity": { - "name": "adjacent", - "namespace": "default", - "type": "global" - } - } - }, - // ... -] ----- - -This catalog contains the following attributes: - -[options="header", cols="~a,~a,~a"] -|=== -|Name|Description|Schema - -|**definition** + -__required__ -|The definition of the function. -|<> object - -|**identity** + -__required__ -|The identity of the function. -|<> object -|=== - -[[definition]] -**Definition** - -[options="header", cols="~a,~a,~a"] -|=== -|Name|Description|Schema - -|**#language** + -__required__ -|The language of the function. - -*Example*: `inline` -|String - -|**parameters** + -__required__ -|The parameters required by the function. -|String array - -|**expression** + -__optional__ -|For inline functions only: the expression defining the function. -|String - -|**text** + -__optional__ -|For inline functions: the verbatim text of the function. - -''' - -For {sqlpp} managed user-defined functions: the external code defining the function. -|String - -|**library** + -__optional__ -|For external functions only: the library containing the function. -|String - -|**name** + -__optional__ -|For external functions only: the relative name of the library. -|String - -|**object** + -__optional__ -|For external functions only: the object defining the function. -|String -|=== - -[[identity]] -**Identity** - -[options="header", cols="~a,~a,~a"] -|=== -|Name|Description|Schema - -|**name** + -__required__ -|The name of the function. -|String - -|**namespace** + -__required__ -|The namespace of the function. - -*Example*: `default` -|String - -|**type** + -__required__ -|The type of the function. - -*Example*: `global` -|String - -|**bucket** + -__optional__ -|For scoped functions only: the bucket containing the function. -|String - -|**scope** + -__optional__ -|For scoped functions only: the scope containing the function. -|String -|=== - -[#sys-functions-cache] -== Monitor Cached Functions - -The `system:functions_cache` catalog maintains a list of recently-used user-defined functions across all nodes. -The catalog also lists user-defined functions that have been called recently, but do not exist. -To see the list of recently-used user-defined functions, use: - -[source,sqlpp] ----- -SELECT * FROM system:functions_cache; ----- - -This will result in a list similar to: - -[source,json] ----- -[ - { - "functions_cache": { - "#language": "inline", - "avgServiceTime": "3.066847ms", - "expression": "(((`fahrenheit` - 32) * 5) / 9)", - "lastUse": "2022-03-09 00:17:59.60659793 +0000 UTC m=+35951.429537902", - "maxServiceTime": "3.066847ms", - "minServiceTime": "0s", - "name": "celsius", - "namespace": "default", - "node": "127.0.0.1:8091", - "parameters": [ - "fahrenheit" - ], - "scope": "inventory", - "text": "((fahrenheit - 32) * 5/9)", - "type": "scope", - "uses": 1 - } - }, - { - "functions_cache": { - "#language": "javascript", - "avgServiceTime": "56.892636ms", - "lastUse": "2022-03-09 00:15:46.289934029 +0000 UTC m=+35818.007560703", - "library": "geohash-js", - "maxServiceTime": "146.025426ms", - "minServiceTime": "0s", - "name": "geohash-js", - "namespace": "default", - "node": "127.0.0.1:8091", - "object": "calculateAdjacent", - "parameters": [ - "src", - "dir" - ], - "type": "global", - "uses": 4 - } - }, - { - "functions_cache": { - "avgServiceTime": "3.057421ms", - "lastUse": "2022-03-09 00:17:25.396840275 +0000 UTC m=+35917.199008929", - "maxServiceTime": "3.057421ms", - "minServiceTime": "0s", - "name": "notFound", - "namespace": "default", - "node": "127.0.0.1:8091", - "type": "global", - "undefined_function": true, - "uses": 1 - } - } -] ----- - -This catalog contains the following attributes: - -[options="header", cols="~a,~a,~a"] -|=== -|Name|Description|Schema - -|**#language** + -__required__ -|The language of the function. - -*Example*: `inline` -|String - -|**name** + -__required__ -|The name of the function. -|String - -|**namespace** + -__required__ -|The namespace of the function. - -*Example*: `default` -|String - -|**parameters** + -__required__ -|The parameters required by the function. -|String array - -|**type** + -__required__ -|The type of the function. - -*Example*: `global` -|String - -|**scope** + -__optional__ -|For scoped functions only: the scope containing the function. -|String - -|**expression** + -__optional__ -|For inline functions only: the expression defining the function. -|String - -|**text** + -__optional__ -|For inline functions: the verbatim text of the function. - -''' - -For {sqlpp} managed user-defined functions: the external code defining the function. -|String - -|**library** + -__optional__ -|For external functions only: the library containing the function. -|String - -|**object** + -__optional__ -|For external functions only: the object defining the function. -|String - -|**avgServiceTime** + -__required__ -|The mean service time for the function. -|String - -|**lastUse** + -__required__ -|The date and time when the function was last used. -|String - -|**maxServiceTime** + -__required__ -|The maximum service time for the function. -|String - -|**minServiceTime** + -__required__ -|The minimum service time for the function. -|String - -|**node** + -__required__ -|The query node where the function is cached. -|String - -|**undefined_function** + -__required__ -|Whether the function exists or is undefined. -|Boolean - -|**uses** + -__required__ -|The number of uses of the function. -|Number -|=== - -Each query node keeps its own cache of recently-used user-defined functions, so you may see the same function listed for multiple nodes. - -[#sys-tasks-cache] -== Monitor Cached Tasks - -The `system:tasks_cache` catalog maintains a list of recently-used scheduled tasks, such as index advisor sessions. -To see the list of recently-used scheduled tasks, use: - -[source,sqlpp] ----- -SELECT * FROM system:tasks_cache; ----- - -This will result in a list similar to: - -[source,json] ----- -[ - { - "tasks_cache": { - "class": "advisor", - "delay": "1h0m0s", - "id": "bcd9f8e4-b324-504c-a98b-ace90dba869f", - "name": "aa7f688a-bf29-438f-888f-eeaead87ca40", - "node": "10.143.192.101:8091", - "state": "scheduled", - "subClass": "analyze", - "submitTime": "2019-09-17 05:18:12.903122381 -0700 PDT m=+8460.550715992" - } - }, - { - "tasks_cache": { - "class": "advisor", - "delay": "5m0s", - "id": "254abec5-5782-543e-9ee0-d07da146b94e", - "name": "ca2cfe56-01fa-4563-8eb0-a753af76d865", - "node": "10.143.192.101:8091", - "results": [ - // ... - ], - "startTime": "2019-09-17 05:03:31.821597725 -0700 PDT m=+7579.469191487", - "state": "completed", - "stopTime": "2019-09-17 05:03:31.963133954 -0700 PDT m=+7579.610727539", - "subClass": "analyze", - "submitTime": "2019-09-17 04:58:31.821230131 -0700 PDT m=+7279.468823737" - } - } -] ----- - -This catalog contains the following attributes: - -[options="header", cols="~a,~a,~a"] -|=== -|Name|Description|Schema - -|**class** + -__required__ -|The class of the task. - -*Example*: ``advisor`` -|string - -|**delay** + -__required__ -|The scheduled duration of the task. -|string - -|**id** + -__required__ -|The internal ID of the task. -|string - -|**name** + -__required__ -|The name of the task. -|string - -|**node** + -__required__ -|The node where the task was started. -|string - -|**state** + -__required__ -|The state of the task. - -*Values*: `scheduled`, `cancelled`, `completed` -|string - -|**subClass** + -__required__ -|The subclass of the task. - -*Example*: `analyze` -|string - -|**submitTime** + -__required__ -|The date and time when the task was submitted. -|string - -|**results** + -__optional__ -|Not scheduled tasks: the results of the task. -|Any array - -|**startTime** + -__optional__ -|Not scheduled tasks: the date and time when the task started. -|string (date-time) - -|**stopTime** + -__optional__ -|Not scheduled tasks: the date and time when the task stopped. -|string (date-time) -|=== - -Refer to xref:n1ql:n1ql-language-reference/advisor.adoc[ADVISOR Function] for more information on index advisor sessions. - -[#sys-transactions] -== Monitor Transactions - -The `system:transactions` catalog maintains a list of active Couchbase transactions. -To see the list of active transactions, use: - -[source,sqlpp] ----- -SELECT * FROM system:transactions; ----- - -This will result in a list similar to: - -[source,json] ----- -[ - { - "transactions": { - "durabilityLevel": "majority", - "durabilityTimeout": "2.5s", - "expiryTime": "2021-04-21T12:53:48.598+01:00", - "id": "85aea637-2288-434b-b7c5-413ad8e7c175", - "isolationLevel": "READ COMMITED", - "lastUse": "2021-04-21T12:51:48.598+01:00", - "node": "127.0.0.1:8091", - "numAtrs": 1024, - "scanConsistency": "unbounded", - "status": 0, - "timeout": "2m0s", - "usedMemory": 960, - "uses": 1 - } - // ... - } -] ----- +== Query Profiling Summary -This catalog contains the following attributes: +The following table summarizes Query profiling behavior. -[options="header", cols="~a,~a,~a"] +[%header,cols=7*] |=== -|Name|Description|Schema - -|**durabilityLevel** + -__required__ -|Durability level for all mutations within a transaction. -|string - -|**durabilityTimeout** + -__required__ -|Durability timeout per mutation within the transaction. -|string (duration) - -|**expiryTime** + -__required__ -| -|string (date-time) - -|**id** + -__required__ -|The transaction ID. -|string - -|**isolationLevel** + -__required__ -|The isolation level of the transaction. -|string - -|**lastUse** + -__required__ -| -|string (date-time) - -|**node** + -__required__ -|The node where the transaction was started. -|string - -|**numAtrs** + -__required__ -|The total number of active transaction records. -|integer - -|**scanConsistency** + -__required__ -|The transactional scan consistency. -|string - -|**status** + -__required__ -| -|integer - -|**timeout** + -__required__ -|The transaction timeout duration. -|string (duration) - -|**usedMemory** + -__required__ -| -|integer - -|**uses** + -__required__ -| -|integer -|=== - -Refer to xref:n1ql:n1ql-language-reference/transactions.adoc[{sqlpp} Support for Couchbase Transactions] for more information. - -[#sys-sequences] -== Monitor Sequences - -The `system:sequences` catalog maintains a list of loaded sequences on any node: that is, sequences that have been accessed since the last restart. -To see the list of loaded sequences, use: - -[source,sqlpp] ----- -SELECT * FROM system:sequences; ----- - -This will result in a list similar to: +s| Profile is ... +3+s| Query returns ... +3+s| Catalog includes ... + +s| {blank} +s| Query + +workbench +s| +ifdef::flag-devex-tools[cbq] +s| +ifdef::flag-devex-rest-api[REST API] +s| Active + +Requests +s| Completed + +Requests +s| Prepared + +Statements -[source,json] ----- -[ - { - "sequences": { - "bucket": "travel-sample", - "cache": 50, - "cycle": false, - "increment": 1, - "max": 9223372036854776000, - "min": -9223372036854776000, - "name": "seq1", - "namespace": "default", - "namespace_id": "default", - "path": "`default`:`travel-sample`.`inventory`.`seq1`", - "scope_id": "inventory", - "value": { - "73428daec3c68d8632ae66b09b70f14d": null, - "~next_block": 0 - } - } - }, -// ... -] ----- - -This catalog contains the following attributes: - -[options="header", cols="~a,~a,~a"] -|=== -|Name|Description|Schema - -|**bucket** + -__required__ -|The bucket containing the sequence. -|string - -|**cache** + -__required__ -|The sequence's cache size. -|integer - -|**cycle** + -__required__ -|Whether the sequence is set to cycle. -|boolean - -|**increment** + -__required__ -|The sequence step value. -|integer - -|**min** + -__required__ -|The minimum value permitted for the sequence. -|integer - -|**max** + -__required__ -|The maximum value permitted for the sequence. -|integer - -|**name** + -__required__ -|The name of the sequence. -|string - -|**namespace** + -__required__ -|Namespace to which the sequence belongs. -|string - -|**namespace_id** + -__required__ -|ID of the namespace to which the sequence belongs. -|string - -|**path** + -__required__ -|The fully qualified sequence name. -|string - -|**scope_id** + -__required__ -|ID of the scope to which the sequence belongs. -|string - -|**value** + -__required__ -|The current value of the sequence on each Query node. -| <> object -|=== - -[[value]] -**Values** - -[options="header", cols="~a,~a,~a"] -|=== -|Name|Description|Schema - -|**** + -__required__ -|The name of this property is the UUID of a Query node. +ifdef::flag-devex-tools[] +| `off` +a| — +a| — +a| — +a| icon:check[fw] phases +a| icon:check[fw] phases +a| icon:check[fw] timings +endif::flag-devex-tools[] -The value of this property is the current value of the sequence on that node. -|Integer +| `phases` +a| — +a| +ifdef::flag-devex-tools[] +icon:check[fw] phases +endif::flag-devex-tools[] +a| +ifdef::flag-devex-rest-api[] +icon:check[fw] phases +endif::flag-devex-rest-api[] +a| icon:check[fw] phases +a| icon:check[fw] phases +a| icon:check[fw] timings -|**~next_block** + -__optional__ -|The starting vale of the next block of values that can be reserved for the sequence. -|Integer +| `timings` +a| — +a| [%hardbreaks] +ifdef::flag-devex-tools[] +icon:check[fw] phases +icon:check[fw] timings +endif::flag-devex-tools[] +a| [%hardbreaks] +ifdef::flag-devex-rest-api[] +icon:check[fw] phases +icon:check[fw] timings +endif::flag-devex-rest-api[] +a| [%hardbreaks] +icon:check[fw] phases +icon:check[fw] timings +a| [%hardbreaks] +icon:check[fw] phases +icon:check[fw] timings +a| icon:check[fw] timings |=== -For further details, see xref:n1ql:n1ql-language-reference/createsequence.adoc[]. - -[#sys-all-sequences] -== Monitor All Sequences - -The `system:all_sequences` catalog maintains a list of all defined sequences. -To see the list of all defined sequences, use: - -[source,sqlpp] ----- -SELECT * FROM system:all_sequences; ----- - -This will result in a list similar to: - -[source,json] ----- -[ - { - "sequences": { - "bucket": "travel-sample", - "cache": 50, - "cycle": false, - "increment": -1, - "max": 9223372036854776000, - "min": 0, - "name": "seq4", - "namespace": "default", - "namespace_id": "default", - "path": "`default`:`travel-sample`.`inventory`.`seq4`", - "scope_id": "inventory", - "value": { - "73428daec3c68d8632ae66b09b70f14d": 10, - "~next_block": -40 - } - } - }, - { - "sequences": { - "bucket": "travel-sample", - "cache": 50, - "cycle": true, - "increment": 5, - "max": 1000, - "min": 0, - "name": "seq3", - "namespace": "default", - "namespace_id": "default", - "path": "`default`:`travel-sample`.`inventory`.`seq3`", - "scope_id": "inventory", - "value": { - "73428daec3c68d8632ae66b09b70f14d": 5, - "~next_block": 255 - } - } - }, -// ... -] ----- - -This catalog gives the same information as the <> catalog. - -For further details, see xref:n1ql:n1ql-language-reference/createsequence.adoc[]. - == Related Links * Refer to xref:n1ql:n1ql-intro/sysinfo.adoc[Getting System Information] for more information on the system namespace.