From 2a39b1aa4d2cd774708c77859253b8fad93db518 Mon Sep 17 00:00:00 2001 From: Jesse Seldess Date: Mon, 27 Nov 2017 10:09:21 -0500 Subject: [PATCH 1/3] Document cluster setting for reduce timeseries interval --- v1.2/operational-faqs.md | 33 +++++++++++++++++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-) diff --git a/v1.2/operational-faqs.md b/v1.2/operational-faqs.md index e68aff8dfd4..6b6d140bd05 100644 --- a/v1.2/operational-faqs.md +++ b/v1.2/operational-faqs.md @@ -2,6 +2,7 @@ title: Operational FAQs summary: Get answers to frequently asked questions about operating CockroachDB. toc: false +toc_not_nester: true ---
@@ -35,9 +36,37 @@ Like most databases, CockroachDB caches the most recently accessed data in memor ## Why is disk usage increasing despite lack of writes? -The timeseries data used to power the graphs in the admin UI is stored within the cluster and accumulates for 30 days before it starts getting truncated. As a result, for the first 30 days or so of a cluster's life you will see a steady increase in disk usage and the number of ranges in the cluster even if you aren't writing data to it yourself. +The timeseries data used to power the graphs in the admin UI is stored within the cluster and accumulates for 30 days before it starts getting truncated. As a result, for the first 30 days or so of a cluster's life, you will see a steady increase in disk usage and the number of ranges even if you aren't writing data to the cluster yourself. -As of the 1.0 release, there is no way to change the number of days before timeseries data gets truncated. As a workaround, however, you can start each node with the `COCKROACH_METRICS_SAMPLE_INTERVAL` environment variable set higher than its default of `10s` to store fewer data points. For example, you could set it to `1m` to only collect data every 1 minute, which would result in storing 6x less timeseries data than the default setting. +There are 2 ways to reduce the size of timeseries data on disk: + +- [Truncate timeseries data sooner](#truncate-timeseries-data-sooner) +- [Store fewer timeseries data points](#store-fewer-timeseries-data-points) + +### Truncate timeseries data sooner New in v1.2 + +As mentioned above, the cluster stores timeseries data for 30 days by default before it starts getting truncated. To truncate timeseries data sooner, change the `timeseries.resolution_10s.storage_duration` cluster setting to an [`INTERVAL`](interval.html) value less than `720h0m0s` (30 days). For example, to truncate timeseries data after 15 days, you would execute the following [`SET CLUSTER SETTING`](set-cluster-setting.html) command: + +~~~ sql +> SET CLUSTER SETTING timeseries.resolution_10s.storage_duration = '360h0m0s'; +~~~ + +~~~ sql +> SHOW CLUSTER SETTING timeseries.resolution_10s.storage_duration; +~~~ + +~~~ ++--------------------------------------------+ +| timeseries.resolution_10s.storage_duration | ++--------------------------------------------+ +| 360h | ++--------------------------------------------+ +(1 row) +~~~ + +### Store fewer timeseries data points + +The cluster collects timeseries data points every 10 seconds by default. To store fewer data points, you can start each node with the `COCKROACH_METRICS_SAMPLE_INTERVAL` environment variable set higher than this default. For example, set it to `1m` to only collect data every 1 minute, which would result in storing 6x less timeseries data than the default setting. ## Why does CockroachDB collect anonymized cluster usage details by default? From bf16bf02789a996d0ee8f1bc73326ffb1088b2b7 Mon Sep 17 00:00:00 2001 From: Jesse Seldess Date: Mon, 27 Nov 2017 10:09:59 -0500 Subject: [PATCH 2/3] Add note about timeseries data in new cluster and link to FAQ --- v1.2/admin-ui-overview-dashboard.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/v1.2/admin-ui-overview-dashboard.md b/v1.2/admin-ui-overview-dashboard.md index 30ed3df9592..bc649c57e08 100644 --- a/v1.2/admin-ui-overview-dashboard.md +++ b/v1.2/admin-ui-overview-dashboard.md @@ -40,6 +40,8 @@ Ranges are subsets of your data, which are replicated to ensure survivability. R For details about how to control the number and location of replicas, see [Configure Replication Zones](configure-replication-zones.html). +{{site.data.alerts.callout_info}}The timeseries data used to power the graphs in the admin UI is stored within the cluster and accumulates for 30 days before it starts getting truncated. As a result, for the first 30 days or so of a cluster's life, you will see a steady increase in disk usage and the number of ranges even if you aren't writing data to the cluster yourself. For more details, see this FAQ.{{site.data.alerts.end}} + ## Capacity CockroachDB Admin UI Capacity graph From 52ff5a5e5516ee9e4428f912135b6c6130d1665c Mon Sep 17 00:00:00 2001 From: Jesse Seldess Date: Tue, 28 Nov 2017 17:00:41 -0500 Subject: [PATCH 3/3] Remove mention of storing fewer data points --- v1.2/operational-faqs.md | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/v1.2/operational-faqs.md b/v1.2/operational-faqs.md index 6b6d140bd05..2887557ac3f 100644 --- a/v1.2/operational-faqs.md +++ b/v1.2/operational-faqs.md @@ -38,19 +38,14 @@ Like most databases, CockroachDB caches the most recently accessed data in memor The timeseries data used to power the graphs in the admin UI is stored within the cluster and accumulates for 30 days before it starts getting truncated. As a result, for the first 30 days or so of a cluster's life, you will see a steady increase in disk usage and the number of ranges even if you aren't writing data to the cluster yourself. -There are 2 ways to reduce the size of timeseries data on disk: - -- [Truncate timeseries data sooner](#truncate-timeseries-data-sooner) -- [Store fewer timeseries data points](#store-fewer-timeseries-data-points) - -### Truncate timeseries data sooner New in v1.2 - -As mentioned above, the cluster stores timeseries data for 30 days by default before it starts getting truncated. To truncate timeseries data sooner, change the `timeseries.resolution_10s.storage_duration` cluster setting to an [`INTERVAL`](interval.html) value less than `720h0m0s` (30 days). For example, to truncate timeseries data after 15 days, you would execute the following [`SET CLUSTER SETTING`](set-cluster-setting.html) command: +To truncate timeseries data sooner, you can change the `timeseries.resolution_10s.storage_duration` cluster setting to an [`INTERVAL`](interval.html) value less than `720h0m0s` (30 days). For example, to truncate timeseries data after 15 days, you would execute the following [`SET CLUSTER SETTING`](set-cluster-setting.html) command: +{% include copy-clipboard.html %} ~~~ sql > SET CLUSTER SETTING timeseries.resolution_10s.storage_duration = '360h0m0s'; ~~~ +{% include copy-clipboard.html %} ~~~ sql > SHOW CLUSTER SETTING timeseries.resolution_10s.storage_duration; ~~~ @@ -64,10 +59,6 @@ As mentioned above, the cluster stores timeseries data for 30 days by default be (1 row) ~~~ -### Store fewer timeseries data points - -The cluster collects timeseries data points every 10 seconds by default. To store fewer data points, you can start each node with the `COCKROACH_METRICS_SAMPLE_INTERVAL` environment variable set higher than this default. For example, set it to `1m` to only collect data every 1 minute, which would result in storing 6x less timeseries data than the default setting. - ## Why does CockroachDB collect anonymized cluster usage details by default? Collecting information about CockroachDB's real world usage helps us prioritize the development of product features. We choose our default as "opt-in" to strengthen the information we receive from our collection efforts, but we also make a careful effort to send only anonymous, aggregate usage statistics. See [Diagnostics Reporting](diagnostics-reporting.html) for a detailed look at what information is sent and how to opt-out.