diff --git a/CIRCLECI.md b/CIRCLECI.md index eaed1daa19..a2a2e800d1 100644 --- a/CIRCLECI.md +++ b/CIRCLECI.md @@ -75,9 +75,9 @@ arguments are invoked: | Parameter type | Name | Value | |:---------------|:-----|:------| | string | `workflow` | `generate` | -| string | `arangodb-3_10` | `{string in PR Template at 3.10}` | -| string | `arangodb-3_11` | `{string in PR Template at 3.11}` | -| string | `arangodb-3_12` | `{string in PR Template at 3.12}` | +| string | `arangodb-3_10` | [Upstream reference](#upstream-reference) for 3.10 | +| string | `arangodb-3_11` | [Upstream reference](#upstream-reference) for 3.11 | +| string | `arangodb-3_12` | [Upstream reference](#upstream-reference) for 3.12 | | string | `generators` | `examples` | | string | `deploy-url` | `deploy-preview-{PR_NUMBER}` | @@ -93,9 +93,9 @@ arguments are invoked: | Parameter type | Name | Value | |:---------------|:-----|:------| | string | `workflow` | `generate` | -| string | `arangodb-3_10` | `{string in PR Template at 3.10}` | -| string | `arangodb-3_11` | `{string in PR Template at 3.11}` | -| string | `arangodb-3_12` | `{string in PR Template at 3.12}` | +| string | `arangodb-3_10` | [Upstream reference](#upstream-reference) for 3.10 | +| string | `arangodb-3_11` | [Upstream reference](#upstream-reference) for 3.11 | +| string | `arangodb-3_12` | [Upstream reference](#upstream-reference) for 3.12 | | string | `generators` | `examples` | | string | `deploy-url` | `deploy-preview-{PR_NUMBER}` | | boolean | `commit-generated` | `true` | @@ -121,13 +121,38 @@ or for multiple versions. | Parameter type | Name | Value | |:---------------|:-----|:------| | string | `workflow` | `generate` | -| string | `arangodb-3_10` | `{string in PR Template at 3.10}` | -| string | `arangodb-3_11` | `{string in PR Template at 3.11}` | +| string | `arangodb-3_10` | [Upstream reference](#upstream-references) for 3.10 | +| string | `arangodb-3_11` | [Upstream reference](#upstream-references) for 3.11 | | string | `generators` | `examples` | | boolean | `commit-generated` | `true` | | string | `deploy-url` | `deploy-preview-{PR_NUMBER}` | | string | `override` | `http,^aql.*` | +### Upstream references + +Documentation pull requests specify upstream references like so: + +```markdown +- 3.10: +- 3.11: https://github.com/arangodb/arangodb/pull/12345 +- 3.12: arangodb/enterprise-preview:devel-nightly +``` + +The above example indicates that ArangoDB versions 3.11 and 3.12 contain changes +relevant to the docs PR, but 3.10 does not. Relevant changes are typically +behavior changes of _arangod_ that will be visible in documentation examples. + +For 3.11, a link to a PR in the `arangodb/arangodb` repository is given. It is +used by the GitHub integration to determine the feature branch to compile and +use for generating examples. Do not specify a link when manually triggering a +pipeline in CircleCI but the **branch name** (like `feature/new-aql-function`)! + +For 3.12, an ArangoDB Enterprise Edition image hosted on +[Docker Hub](https://hub.docker.com/) is specified. Using Docker images has the +advantage that the compilation of ArangoDB can be skipped, making the example +generation faster. Of course, this requires that an image containing relevant +changes to ArangoDB exists. + ## Release workflow for ArangoDB releases To run a release job for a new ArangoDB patch release (e.g. 3.11.4), diff --git a/site/content/3.10/about-arangodb/features/_index.md b/site/content/3.10/about-arangodb/features/_index.md index c782536143..6704ed1d28 100644 --- a/site/content/3.10/about-arangodb/features/_index.md +++ b/site/content/3.10/about-arangodb/features/_index.md @@ -63,6 +63,7 @@ See all [Enterprise Edition Features](enterprise-edition.md). {{% /comment %}} | Only regular cluster deployments | **OneShard** deployment option to store all collections of a database on a single cluster node, to combine the performance of a single server and ACID semantics with a fault-tolerant cluster setup | | ACID transactions for multi-document / multi-collection queries on single servers, for single document operations in clusters, and for multi-document queries in clusters for collections with a single shard | In addition, ACID transactions for multi-collection queries using the OneShard feature | +| Always read from leader shards in clusters | Optionally allow dirty reads to **read from followers** to scale reads | | TLS key and certificate rotation | In addition, **key rotation for JWT secrets** and **server name indication** (SNI) | | Built-in user management and authentication | Additional **LDAP authentication** option | | Only server logs | **Audit log** of server interactions | diff --git a/site/content/3.10/about-arangodb/features/enterprise-edition.md b/site/content/3.10/about-arangodb/features/enterprise-edition.md index 5e07530902..6370df0205 100644 --- a/site/content/3.10/about-arangodb/features/enterprise-edition.md +++ b/site/content/3.10/about-arangodb/features/enterprise-edition.md @@ -68,6 +68,10 @@ features outlined below. For additional information, see stored values, primary sort columns, and primary key columns in memory to improve the performance of Views and inverted indexes. +- [**Read from followers in clusters**](../../develop/http-api/documents.md#read-from-followers): + Allow dirty reads so that Coordinators can read from any shard replica and not + only from the leader, for scaling reads. + ## Querying - [**Pregel in Cluster**](../../data-science/pregel/_index.md#prerequisites): diff --git a/site/content/3.10/about-arangodb/features/highlights-by-version.md b/site/content/3.10/about-arangodb/features/highlights-by-version.md index 0269642d92..c62ff81bf6 100644 --- a/site/content/3.10/about-arangodb/features/highlights-by-version.md +++ b/site/content/3.10/about-arangodb/features/highlights-by-version.md @@ -65,6 +65,10 @@ aliases: Optimized data loading for AQL traversal queries if only a few document attributes are accessed. +- [**Read from followers in clusters**](../../develop/http-api/documents.md#read-from-followers): + Allow dirty reads so that Coordinators can read from any shard replica and not + only from the leader, for scaling reads. + Also see [What's New in 3.10](../../release-notes/version-3.10/whats-new-in-3-10.md). ## Version 3.9 diff --git a/site/content/3.10/develop/transactions/stream-transactions.md b/site/content/3.10/develop/transactions/stream-transactions.md index f5c5743997..521114e5cc 100644 --- a/site/content/3.10/develop/transactions/stream-transactions.md +++ b/site/content/3.10/develop/transactions/stream-transactions.md @@ -45,7 +45,7 @@ on the Coordinator to ensure that abandoned transactions cannot block the cluster from operating properly: - Maximum idle timeout of up to **120 seconds** between operations. -- Maximum transaction size of **128 MB** per DB-Server. +- Maximum transaction size of **128 MB** (per DB-Server in clusters). These limits are also enforced for Stream Transactions on single servers. @@ -103,7 +103,7 @@ Additionally, `options` can have the following optional attributes: waiting on collection locks. This option is only meaningful when using `exclusive` locks. If not specified, a default value is used. Setting `lockTimeout` to `0` makes ArangoDB not time out waiting for a lock. -- `maxTransactionSize`: Transaction size limit in bytes. +- `maxTransactionSize`: Transaction size limit in bytes. Can be at most 128 MiB. The method returns an object that lets you run supported operations as part of the transactions, get the status information, and commit or abort the transaction. diff --git a/site/content/3.10/release-notes/version-3.10/whats-new-in-3-10.md b/site/content/3.10/release-notes/version-3.10/whats-new-in-3-10.md index 2c29b72196..179d40431f 100644 --- a/site/content/3.10/release-notes/version-3.10/whats-new-in-3-10.md +++ b/site/content/3.10/release-notes/version-3.10/whats-new-in-3-10.md @@ -1048,7 +1048,12 @@ that have copies of the data. Therefore, the read throughput is higher. This feature is only available in the Enterprise Edition. -For more information, see [Read from followers](../../develop/http-api/documents.md#read-from-followers). +For more information, see [Read from followers](../../develop/http-api/documents.md#read-from-followers) +in the HTTP API documentation. + +The JavaScript API supports an `allowDirtyReads` option for +[AQL queries](../../aql/how-to-invoke-aql/with-arangosh.md#allowdirtyreads) and +[reading documents](../../develop/javascript-api/@arangodb/collection-object.md#collectiondocumentobject--options). ## Improved shard rebalancing diff --git a/site/content/3.11/about-arangodb/features/_index.md b/site/content/3.11/about-arangodb/features/_index.md index c782536143..6704ed1d28 100644 --- a/site/content/3.11/about-arangodb/features/_index.md +++ b/site/content/3.11/about-arangodb/features/_index.md @@ -63,6 +63,7 @@ See all [Enterprise Edition Features](enterprise-edition.md). {{% /comment %}} | Only regular cluster deployments | **OneShard** deployment option to store all collections of a database on a single cluster node, to combine the performance of a single server and ACID semantics with a fault-tolerant cluster setup | | ACID transactions for multi-document / multi-collection queries on single servers, for single document operations in clusters, and for multi-document queries in clusters for collections with a single shard | In addition, ACID transactions for multi-collection queries using the OneShard feature | +| Always read from leader shards in clusters | Optionally allow dirty reads to **read from followers** to scale reads | | TLS key and certificate rotation | In addition, **key rotation for JWT secrets** and **server name indication** (SNI) | | Built-in user management and authentication | Additional **LDAP authentication** option | | Only server logs | **Audit log** of server interactions | diff --git a/site/content/3.11/about-arangodb/features/enterprise-edition.md b/site/content/3.11/about-arangodb/features/enterprise-edition.md index 5e07530902..6370df0205 100644 --- a/site/content/3.11/about-arangodb/features/enterprise-edition.md +++ b/site/content/3.11/about-arangodb/features/enterprise-edition.md @@ -68,6 +68,10 @@ features outlined below. For additional information, see stored values, primary sort columns, and primary key columns in memory to improve the performance of Views and inverted indexes. +- [**Read from followers in clusters**](../../develop/http-api/documents.md#read-from-followers): + Allow dirty reads so that Coordinators can read from any shard replica and not + only from the leader, for scaling reads. + ## Querying - [**Pregel in Cluster**](../../data-science/pregel/_index.md#prerequisites): diff --git a/site/content/3.11/about-arangodb/features/highlights-by-version.md b/site/content/3.11/about-arangodb/features/highlights-by-version.md index 79ff0130cb..acf64d6438 100644 --- a/site/content/3.11/about-arangodb/features/highlights-by-version.md +++ b/site/content/3.11/about-arangodb/features/highlights-by-version.md @@ -91,6 +91,10 @@ Also see [What's New in 3.11](../../release-notes/version-3.11/whats-new-in-3-11 Optimized data loading for AQL traversal queries if only a few document attributes are accessed. +- [**Read from followers in clusters**](../../develop/http-api/documents.md#read-from-followers): + Allow dirty reads so that Coordinators can read from any shard replica and not + only from the leader, for scaling reads. + Also see [What's New in 3.10](../../release-notes/version-3.10/whats-new-in-3-10.md). ## Version 3.9 diff --git a/site/content/3.11/develop/transactions/stream-transactions.md b/site/content/3.11/develop/transactions/stream-transactions.md index f5c5743997..521114e5cc 100644 --- a/site/content/3.11/develop/transactions/stream-transactions.md +++ b/site/content/3.11/develop/transactions/stream-transactions.md @@ -45,7 +45,7 @@ on the Coordinator to ensure that abandoned transactions cannot block the cluster from operating properly: - Maximum idle timeout of up to **120 seconds** between operations. -- Maximum transaction size of **128 MB** per DB-Server. +- Maximum transaction size of **128 MB** (per DB-Server in clusters). These limits are also enforced for Stream Transactions on single servers. @@ -103,7 +103,7 @@ Additionally, `options` can have the following optional attributes: waiting on collection locks. This option is only meaningful when using `exclusive` locks. If not specified, a default value is used. Setting `lockTimeout` to `0` makes ArangoDB not time out waiting for a lock. -- `maxTransactionSize`: Transaction size limit in bytes. +- `maxTransactionSize`: Transaction size limit in bytes. Can be at most 128 MiB. The method returns an object that lets you run supported operations as part of the transactions, get the status information, and commit or abort the transaction. diff --git a/site/content/3.11/release-notes/version-3.10/api-changes-in-3-10.md b/site/content/3.11/release-notes/version-3.10/api-changes-in-3-10.md index b5b466f428..a7e690d0a5 100644 --- a/site/content/3.11/release-notes/version-3.10/api-changes-in-3-10.md +++ b/site/content/3.11/release-notes/version-3.10/api-changes-in-3-10.md @@ -126,6 +126,23 @@ if the request body was an empty array. Example: Now, a request like this succeeds and returns an empty array as response. +#### Limit to the number of databases in a deployment + +Introduced in: v3.10.10 + +The new `--database.max-databases` startup option can cap the number of databases +and creating databases using the `POST /_api/database` endpoint can thus now fail +for this reason if your deployment is at or above the configured maximum. Example: + +```json +{ + "code": 400, + "error": true, + "errorMessage": "unable to create additional database because it would exceed the configured maximum number of databases (2)", + "errorNum": 32 +} +``` + ### Endpoint return value changes - Since ArangoDB 3.8, there have been two APIs for retrieving the metrics in two @@ -807,6 +824,18 @@ The following metrics have been added: | `arangodb_file_descriptors_limit` | System limit for the number of open files for the arangod process. | | `arangodb_file_descriptors_current` | Number of file descriptors currently opened by the arangod process. | +--- + +Introduced in: v3.10.11 + +The following metrics have been added to improve the observability of in-memory +cache subsystem: + +- `rocksdb_cache_free_memory_tasks_total` +- `rocksdb_cache_free_memory_tasks_duration_total` +- `rocksdb_cache_migrate_tasks_total` +- `rocksdb_cache_migrate_tasks_duration_total` + #### Pregel API When loading the graph data into memory, a `"loading"` state is now returned by diff --git a/site/content/3.11/release-notes/version-3.10/whats-new-in-3-10.md b/site/content/3.11/release-notes/version-3.10/whats-new-in-3-10.md index 30b0c414e3..179d40431f 100644 --- a/site/content/3.11/release-notes/version-3.10/whats-new-in-3-10.md +++ b/site/content/3.11/release-notes/version-3.10/whats-new-in-3-10.md @@ -1048,7 +1048,12 @@ that have copies of the data. Therefore, the read throughput is higher. This feature is only available in the Enterprise Edition. -For more information, see [Read from followers](../../develop/http-api/documents.md#read-from-followers). +For more information, see [Read from followers](../../develop/http-api/documents.md#read-from-followers) +in the HTTP API documentation. + +The JavaScript API supports an `allowDirtyReads` option for +[AQL queries](../../aql/how-to-invoke-aql/with-arangosh.md#allowdirtyreads) and +[reading documents](../../develop/javascript-api/@arangodb/collection-object.md#collectiondocumentobject--options). ## Improved shard rebalancing @@ -1384,6 +1389,20 @@ attempt to create an additional database fails with error if other databases are dropped first. The default value for this option is unlimited, so an arbitrary amount of databases can be created. +### Configurable maximum for queued log entries + +Introduced in: v3.10.12 + +The new `--log.max-queued-entries` startup option lets you configure how many +log entries are queued in a background thread. + +Log entries are pushed on a queue for asynchronous writing unless you enable the +`--log.force-direct` startup option. If you use a slow log output (e.g. syslog), +the queue might grow and eventually overflow. + +You can configure the upper bound of the queue with this option. If the queue is +full, log entries are written synchronously until the queue has space again. + ## Miscellaneous changes ### Optimizer rules endpoint @@ -1574,6 +1593,15 @@ The following system metrics have been added: | `arangodb_file_descriptors_limit` | System limit for the number of open files for the arangod process. | | `arangodb_file_descriptors_current` | Number of file descriptors currently opened by the arangod process. | +### More instant Hot Backups + +Introduced in: v3.10.10, v3.11.3 + +Cluster deployments no longer wait for all in-progress transactions to get +committed when a user requests a Hot Backup. The waiting could cause deadlocks +and thus Hot Backups to fail, in particular in ArangoGraph. Now, Hot Backups are +created immediately and commits have to wait until the backup process is done. + ## Client tools ### arangobench diff --git a/site/content/3.12/about-arangodb/features/_index.md b/site/content/3.12/about-arangodb/features/_index.md index c782536143..6704ed1d28 100644 --- a/site/content/3.12/about-arangodb/features/_index.md +++ b/site/content/3.12/about-arangodb/features/_index.md @@ -63,6 +63,7 @@ See all [Enterprise Edition Features](enterprise-edition.md). {{% /comment %}} | Only regular cluster deployments | **OneShard** deployment option to store all collections of a database on a single cluster node, to combine the performance of a single server and ACID semantics with a fault-tolerant cluster setup | | ACID transactions for multi-document / multi-collection queries on single servers, for single document operations in clusters, and for multi-document queries in clusters for collections with a single shard | In addition, ACID transactions for multi-collection queries using the OneShard feature | +| Always read from leader shards in clusters | Optionally allow dirty reads to **read from followers** to scale reads | | TLS key and certificate rotation | In addition, **key rotation for JWT secrets** and **server name indication** (SNI) | | Built-in user management and authentication | Additional **LDAP authentication** option | | Only server logs | **Audit log** of server interactions | diff --git a/site/content/3.12/about-arangodb/features/enterprise-edition.md b/site/content/3.12/about-arangodb/features/enterprise-edition.md index 90ea02ca64..bfc2758c8f 100644 --- a/site/content/3.12/about-arangodb/features/enterprise-edition.md +++ b/site/content/3.12/about-arangodb/features/enterprise-edition.md @@ -72,6 +72,10 @@ features outlined below. For additional information, see Retrieve search results for the highest-ranking matches from Views faster by defining a list of sort expressions to optimize. +- [**Read from followers in clusters**](../../develop/http-api/documents.md#read-from-followers): + Allow dirty reads so that Coordinators can read from any shard replica and not + only from the leader, for scaling reads. + ## Querying - [**Pregel in Cluster**](../../data-science/pregel/_index.md#prerequisites): diff --git a/site/content/3.12/about-arangodb/features/highlights-by-version.md b/site/content/3.12/about-arangodb/features/highlights-by-version.md index 76de63b6db..e997c56cb7 100644 --- a/site/content/3.12/about-arangodb/features/highlights-by-version.md +++ b/site/content/3.12/about-arangodb/features/highlights-by-version.md @@ -105,6 +105,10 @@ Also see [What's New in 3.11](../../release-notes/version-3.11/whats-new-in-3-11 Optimized data loading for AQL traversal queries if only a few document attributes are accessed. +- [**Read from followers in clusters**](../../develop/http-api/documents.md#read-from-followers): + Allow dirty reads so that Coordinators can read from any shard replica and not + only from the leader, for scaling reads. + Also see [What's New in 3.10](../../release-notes/version-3.10/whats-new-in-3-10.md). ## Version 3.9 diff --git a/site/content/3.12/develop/transactions/stream-transactions.md b/site/content/3.12/develop/transactions/stream-transactions.md index f5c5743997..2fefb6768e 100644 --- a/site/content/3.12/develop/transactions/stream-transactions.md +++ b/site/content/3.12/develop/transactions/stream-transactions.md @@ -45,10 +45,13 @@ on the Coordinator to ensure that abandoned transactions cannot block the cluster from operating properly: - Maximum idle timeout of up to **120 seconds** between operations. -- Maximum transaction size of **128 MB** per DB-Server. +- Maximum transaction size with a default of **128 MiB** (per DB-Server in clusters). These limits are also enforced for Stream Transactions on single servers. +The maximum size for a single Stream Transaction can be adjusted with the +`--transaction.streaming-max-transaction-size` startup option. + The default maximum idle timeout is **60 seconds** between operations in a single Stream Transaction. The maximum value can be bumped up to at most 120 seconds by setting the `--transaction.streaming-idle-timeout` startup option. @@ -103,7 +106,8 @@ Additionally, `options` can have the following optional attributes: waiting on collection locks. This option is only meaningful when using `exclusive` locks. If not specified, a default value is used. Setting `lockTimeout` to `0` makes ArangoDB not time out waiting for a lock. -- `maxTransactionSize`: Transaction size limit in bytes. +- `maxTransactionSize`: Transaction size limit in bytes. Can be at most the + value of the `--transaction.streaming-max-transaction-size` startup option. The method returns an object that lets you run supported operations as part of the transactions, get the status information, and commit or abort the transaction. diff --git a/site/content/3.12/release-notes/version-3.10/api-changes-in-3-10.md b/site/content/3.12/release-notes/version-3.10/api-changes-in-3-10.md index b5b466f428..a7e690d0a5 100644 --- a/site/content/3.12/release-notes/version-3.10/api-changes-in-3-10.md +++ b/site/content/3.12/release-notes/version-3.10/api-changes-in-3-10.md @@ -126,6 +126,23 @@ if the request body was an empty array. Example: Now, a request like this succeeds and returns an empty array as response. +#### Limit to the number of databases in a deployment + +Introduced in: v3.10.10 + +The new `--database.max-databases` startup option can cap the number of databases +and creating databases using the `POST /_api/database` endpoint can thus now fail +for this reason if your deployment is at or above the configured maximum. Example: + +```json +{ + "code": 400, + "error": true, + "errorMessage": "unable to create additional database because it would exceed the configured maximum number of databases (2)", + "errorNum": 32 +} +``` + ### Endpoint return value changes - Since ArangoDB 3.8, there have been two APIs for retrieving the metrics in two @@ -807,6 +824,18 @@ The following metrics have been added: | `arangodb_file_descriptors_limit` | System limit for the number of open files for the arangod process. | | `arangodb_file_descriptors_current` | Number of file descriptors currently opened by the arangod process. | +--- + +Introduced in: v3.10.11 + +The following metrics have been added to improve the observability of in-memory +cache subsystem: + +- `rocksdb_cache_free_memory_tasks_total` +- `rocksdb_cache_free_memory_tasks_duration_total` +- `rocksdb_cache_migrate_tasks_total` +- `rocksdb_cache_migrate_tasks_duration_total` + #### Pregel API When loading the graph data into memory, a `"loading"` state is now returned by diff --git a/site/content/3.12/release-notes/version-3.10/whats-new-in-3-10.md b/site/content/3.12/release-notes/version-3.10/whats-new-in-3-10.md index 30b0c414e3..179d40431f 100644 --- a/site/content/3.12/release-notes/version-3.10/whats-new-in-3-10.md +++ b/site/content/3.12/release-notes/version-3.10/whats-new-in-3-10.md @@ -1048,7 +1048,12 @@ that have copies of the data. Therefore, the read throughput is higher. This feature is only available in the Enterprise Edition. -For more information, see [Read from followers](../../develop/http-api/documents.md#read-from-followers). +For more information, see [Read from followers](../../develop/http-api/documents.md#read-from-followers) +in the HTTP API documentation. + +The JavaScript API supports an `allowDirtyReads` option for +[AQL queries](../../aql/how-to-invoke-aql/with-arangosh.md#allowdirtyreads) and +[reading documents](../../develop/javascript-api/@arangodb/collection-object.md#collectiondocumentobject--options). ## Improved shard rebalancing @@ -1384,6 +1389,20 @@ attempt to create an additional database fails with error if other databases are dropped first. The default value for this option is unlimited, so an arbitrary amount of databases can be created. +### Configurable maximum for queued log entries + +Introduced in: v3.10.12 + +The new `--log.max-queued-entries` startup option lets you configure how many +log entries are queued in a background thread. + +Log entries are pushed on a queue for asynchronous writing unless you enable the +`--log.force-direct` startup option. If you use a slow log output (e.g. syslog), +the queue might grow and eventually overflow. + +You can configure the upper bound of the queue with this option. If the queue is +full, log entries are written synchronously until the queue has space again. + ## Miscellaneous changes ### Optimizer rules endpoint @@ -1574,6 +1593,15 @@ The following system metrics have been added: | `arangodb_file_descriptors_limit` | System limit for the number of open files for the arangod process. | | `arangodb_file_descriptors_current` | Number of file descriptors currently opened by the arangod process. | +### More instant Hot Backups + +Introduced in: v3.10.10, v3.11.3 + +Cluster deployments no longer wait for all in-progress transactions to get +committed when a user requests a Hot Backup. The waiting could cause deadlocks +and thus Hot Backups to fail, in particular in ArangoGraph. Now, Hot Backups are +created immediately and commits have to wait until the backup process is done. + ## Client tools ### arangobench diff --git a/site/content/3.12/release-notes/version-3.11/api-changes-in-3-11.md b/site/content/3.12/release-notes/version-3.11/api-changes-in-3-11.md index e18b17d9ed..3f22ad2132 100644 --- a/site/content/3.12/release-notes/version-3.11/api-changes-in-3-11.md +++ b/site/content/3.12/release-notes/version-3.11/api-changes-in-3-11.md @@ -262,6 +262,23 @@ Disabled: arangodb_agency_cache_callback_number{role="SINGLE"}0 ``` +#### Limit to the number of databases in a deployment + +Introduced in: v3.10.10, v3.11.2 + +The new `--database.max-databases` startup option can cap the number of databases +and creating databases using the `POST /_api/database` endpoint can thus now fail +for this reason if your deployment is at or above the configured maximum. Example: + +```json +{ + "code": 400, + "error": true, + "errorMessage": "unable to create additional database because it would exceed the configured maximum number of databases (2)", + "errorNum": 32 +} +``` + ### Endpoint return value changes Introduced in: v3.8.8, v3.9.4, v3.10.1 @@ -599,6 +616,52 @@ The following metric stores the peak value of the `rocksdb_cache_allocated` metr |:------|:------------| | `rocksdb_cache_peak_allocated` | Global peak memory allocation of ArangoDB in-memory caches. | +--- + +Introduced in: v3.11.2 + +The following metrics have been added about the LZ4 compression for values in +the in-memory edge cache: + +- `rocksdb_cache_edge_inserts_effective_entries_size_total` +- `rocksdb_cache_edge_inserts_uncompressed_entries_size_total` +- `rocksdb_cache_edge_compression_ratio` + +--- + +Introduced in: v3.10.11, v3.11.4 + +The following metrics have been added to improve the observability of in-memory +cache subsystem: + +- `rocksdb_cache_free_memory_tasks_total` +- `rocksdb_cache_free_memory_tasks_duration_total` +- `rocksdb_cache_migrate_tasks_total` +- `rocksdb_cache_migrate_tasks_duration_total` + +--- + +Introduced in: v3.11.4 + +The following metrics have been added to improve the observability of in-memory +edge cache: + +- `rocksdb_cache_edge_compressed_inserts_total` +- `rocksdb_cache_edge_empty_inserts_total` +- `rocksdb_cache_edge_inserts_total` + +--- + +Introduced in: v3.11.5 + +The following metrics have been added to monitor and detect temporary or +permanent connectivity issues as well as how many scheduler threads are in the +detached state: + +- `arangodb_network_connectivity_failures_coordinators` +- `arangodb_network_connectivity_failures_dbservers_total` +- `arangodb_scheduler_num_detached_threads` + #### Log level API Introduced in: v3.10.2 diff --git a/site/content/3.12/release-notes/version-3.11/whats-new-in-3-11.md b/site/content/3.12/release-notes/version-3.11/whats-new-in-3-11.md index 01dd8ba116..65eed12913 100644 --- a/site/content/3.12/release-notes/version-3.11/whats-new-in-3-11.md +++ b/site/content/3.12/release-notes/version-3.11/whats-new-in-3-11.md @@ -515,7 +515,7 @@ Query Statistics: ### New stage in query profiling output -Introduced in: v3.10.3, v3.11.0 +Introduced in: v3.10.3 The query profiling output has a new `instantiating executors` stage. The time spent in this stage is the time needed to create the query executors @@ -1036,26 +1036,19 @@ permanent connectivity issues: - `arangodb_network_connectivity_failures_dbservers_total`: Number of failed connectivity check requests sent to DB-Servers. -### Detached scheduler threads +### Configurable maximum for queued log entries -Introduced in: v3.11.5 - -A scheduler thread now has the capability to detach itself from the scheduler -if it observes the need to perform a potentially long running task, like waiting -for a lock. This allows a new scheduler thread to be started and prevents -scenarios where all threads are blocked waiting for a lock, which has previously -led to deadlock situations. +Introduced in: v3.10.12, v3.11.5 -Threads waiting for more than 1 second on a collection lock will detach -themselves. +The new `--log.max-queued-entries` startup option lets you configure how many +log entries are queued in a background thread. -The following startup option has been added: -- `--server.max-number-detached-threads`: The maximum number of detached scheduler - threads. +Log entries are pushed on a queue for asynchronous writing unless you enable the +`--log.force-direct` startup option. If you use a slow log output (e.g. syslog), +the queue might grow and eventually overflow. -The following metric has been added: -- `arangodb_scheduler_num_detached_threads`: The number of worker threads - currently started and detached from the scheduler. +You can configure the upper bound of the queue with this option. If the queue is +full, log entries are written synchronously until the queue has space again. ## Miscellaneous changes @@ -1242,6 +1235,15 @@ The following system metrics have been added: | `arangodb_file_descriptors_limit` | System limit for the number of open files for the arangod process. | | `arangodb_file_descriptors_current` | Number of file descriptors currently opened by the arangod process. | +### More instant Hot Backups + +Introduced in: v3.10.10, v3.11.3 + +Cluster deployments no longer wait for all in-progress transactions to get +committed when a user requests a Hot Backup. The waiting could cause deadlocks +and thus Hot Backups to fail, in particular in ArangoGraph. Now, Hot Backups are +created immediately and commits have to wait until the backup process is done. + ### In-memory edge cache startup options and metrics Introduced in: v3.11.4 @@ -1287,6 +1289,27 @@ cache subsystem: by the cache subsystem to migrate existing cache hash tables to a bigger or smaller table. +### Detached scheduler threads + +Introduced in: v3.11.5 + +A scheduler thread now has the capability to detach itself from the scheduler +if it observes the need to perform a potentially long running task, like waiting +for a lock. This allows a new scheduler thread to be started and prevents +scenarios where all threads are blocked waiting for a lock, which has previously +led to deadlock situations. + +Threads waiting for more than 1 second on a collection lock will detach +themselves. + +The following startup option has been added: +- `--server.max-number-detached-threads`: The maximum number of detached scheduler + threads. + +The following metric has been added: +- `arangodb_scheduler_num_detached_threads`: The number of worker threads + currently started and detached from the scheduler. + ## Client tools ### arangodump diff --git a/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md b/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md index 2ae11ed92a..8eae47a756 100644 --- a/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md +++ b/site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md @@ -86,6 +86,14 @@ for this reason if your deployment is at or above the configured maximum. Exampl } ``` +#### Adjustable Stream Transaction size + +The [Stream Transactions HTTP API](../../develop/http-api/transactions/stream-transactions.md) +may now allow larger transactions or be limited to smaller transactions because +the maximum transaction size can now be configured with the +`--transaction.streaming-max-transaction-size` startup option. +The default value remains 128 MiB. + ### Privilege changes @@ -143,13 +151,14 @@ produced no warnings. #### Metrics API The metrics endpoint includes the following new metrics about AQL queries, -ongoing dumps, and ArangoSearch execution threads: +ongoing dumps, ArangoSearch parallelism and used file descriptors: - `arangodb_aql_cursors_active` - `arangodb_dump_memory_usage` - `arangodb_dump_ongoing` - `arangodb_dump_threads_blocked_total` - `arangodb_search_execution_threads_demand` +- `arangodb_search_file_descriptors` --- diff --git a/site/content/3.12/release-notes/version-3.12/incompatible-changes-in-3-12.md b/site/content/3.12/release-notes/version-3.12/incompatible-changes-in-3-12.md index 7a5a8aba50..0250cd75fc 100644 --- a/site/content/3.12/release-notes/version-3.12/incompatible-changes-in-3-12.md +++ b/site/content/3.12/release-notes/version-3.12/incompatible-changes-in-3-12.md @@ -79,6 +79,14 @@ The second option is the recommended one, as it signals the intent more clearly, and makes the cache behave "as expected", i.e. use up to the configured memory limit and not just 56% of it. +#### Adjustable Stream Transaction size + +[Stream Transactions](../../develop/transactions/stream-transactions.md) may +now be limited to smaller transaction sizes because the maximum transaction size +can now be configured with the `--transaction.streaming-max-transaction-size` +startup option. The default value remains 128 MiB but configuring a lower limit +can cause previously working Stream Transactions to fail. + ## Client tools ### jslint feature in arangosh diff --git a/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md b/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md index 93a9a7fdad..a54f18987e 100644 --- a/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md +++ b/site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md @@ -112,6 +112,12 @@ indexes have been allowing to index and store the `_id` system attribute. ## Server options +### Adjustable Stream Transaction size + +The previously fixed limit of 128 MiB for [Stream Transactions](../../develop/transactions/stream-transactions.md) +can now be configured with the new `--transaction.streaming-max-transaction-size` +startup option. The default value remains 128 MiB. + ### LZ4 compression for values in the in-memory edge cache Introduced in: v3.11.2 @@ -270,6 +276,13 @@ of outgrowing the maximum number of file descriptors the ArangoDB process can open. Thus, these options should only be enabled on deployments with a limited number of collections/shards/indexes. +### ArangoSearch file descriptor metric + +The following metric has been added: + +- `arangodb_search_file_descriptors`: + Current count of opened file descriptors for an ArangoSearch index. + ### More instant Hot Backups Introduced in: v3.10.10, v3.11.3 @@ -383,7 +396,7 @@ with large shards. the network is slow or its capacity is maxed out. The data is decompressed on the client side and recompressed if you enable the `--compress-output` option. -#### Resource usage limits and metrics +#### Server-side resource usage limits and metrics The following `arangod` startup options can be used to limit the resource usage of parallel _arangodump_ invocations: