From 0ba8f1da63ac7a4cddb0a619cecbfe3536b068f1 Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 13:27:19 +0300 Subject: [PATCH 01/14] IGNORE - doc updates --- .../data-types/timeseries/configuration.md | 36 +++++++++++-------- 1 file changed, 22 insertions(+), 14 deletions(-) diff --git a/content/develop/data-types/timeseries/configuration.md b/content/develop/data-types/timeseries/configuration.md index 10f7636ca5..8454894b24 100644 --- a/content/develop/data-types/timeseries/configuration.md +++ b/content/develop/data-types/timeseries/configuration.md @@ -44,17 +44,17 @@ $ redis-server --loadmodule ./redistimeseries.so [OPT VAL]... The following table summarizes which configuration parameters can be set at module load-time and run-time: -| Configuration Parameter | Load-time | Run-time | -| :------- | :----- | :----------- | -| [NUM_THREADS](#num_threads) (since RedisTimeSeries v1.6) | :white_check_mark: | :white_large_square: | -| [COMPACTION_POLICY](#compaction_policy) | :white_check_mark: | :white_large_square: | -| [RETENTION_POLICY](#retention_policy) | :white_check_mark: | :white_large_square: | -| [DUPLICATE_POLICY](#duplicate_policy) | :white_check_mark: | :white_large_square: | -| [ENCODING](#encoding) (since RedisTimeSeries v1.6) | :white_check_mark: | :white_large_square: | -| [CHUNK_SIZE_BYTES](#chunk_size_bytes) | :white_check_mark: | :white_large_square: | -| [OSS_GLOBAL_PASSWORD](#oss_global_password) (since RedisTimeSeries v1.8.4) | :white_check_mark: | :white_large_square: | +| Configuration Parameter | Load-time | Run-time | +| :------- | :----- | :----------- | +| [NUM_THREADS](#num_threads) (since RedisTimeSeries v1.6) | :white_check_mark: | :white_large_square: | +| [COMPACTION_POLICY](#compaction_policy) | :white_check_mark: | :white_large_square: | +| [RETENTION_POLICY](#retention_policy) | :white_check_mark: | :white_large_square: | +| [DUPLICATE_POLICY](#duplicate_policy) | :white_check_mark: | :white_large_square: | +| [ENCODING](#encoding) (since RedisTimeSeries v1.6) | :white_check_mark: | :white_large_square: | +| [CHUNK_SIZE_BYTES](#chunk_size_bytes) | :white_check_mark: | :white_large_square: | +| [OSS_GLOBAL_PASSWORD](#oss_global_password) (since RedisTimeSeries v1.8.4) | :white_check_mark: | :white_large_square: | | [IGNORE_MAX_TIME_DIFF](#ignore_max_time_diff-and-ignore_max_val_diff) (since RedisTimeSeries v1.12) | :white_check_mark: | :white_large_square: | -| [IGNORE_MAX_VAL_DIFF](#ignore_max_time_diff-and-ignore_max_val_diff) (since RedisTimeSeries v1.12) | :white_check_mark: | :white_large_square: | +| [IGNORE_MAX_VAL_DIFF](#ignore_max_time_diff-and-ignore_max_val_diff) (since RedisTimeSeries v1.12) | :white_check_mark: | :white_large_square: | ### NUM_THREADS @@ -72,7 +72,7 @@ $ redis-server --loadmodule ./redistimeseries.so NUM_THREADS 3 ### COMPACTION_POLICY -Default compaction rules for newly created key with [`TS.ADD`]({{< baseurl >}}/commands/ts.add/). +Default compaction rules for newly created key with [`TS.ADD`]({{< baseurl >}}/commands/ts.add/), [`TS.INCRBY`]({{< baseurl >}}/commands/ts.incrby/), and [`TS.DECRBY`]({{< baseurl >}}/commands/ts.decrby/). Note that `COMPACTION_POLICY` has no effect on keys created with [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/). To understand the motivation for this behavior, consider the following scenario: Suppose a `COMPACTION_POLICY` is defined, but then one wants to manually create an additional compaction rule (using [`TS.CREATERULE`]({{< baseurl >}}/commands/ts.createrule/)) which requires first creating an empty destination key (using [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/)). But now there is a problem: due to the `COMPACTION_POLICY`, automatic compactions would be undesirably created for that destination key. @@ -251,11 +251,19 @@ $ redis-server --loadmodule ./redistimeseries.so OSS_GLOBAL_PASSWORD password ### IGNORE_MAX_TIME_DIFF and IGNORE_MAX_VAL_DIFF +Default values for newly created keys. + +Many sensors report data periodically. Often, the difference between the measured value and the previous measured value is negligible and related to random noise or to measurement accuracy limitations. In such situations it may be preferable not to add the new measurement to the time series. + A new sample is considered a duplicate and is ignored if the following conditions are met: +1. The time series is not a compaction; +1. The time series' `DUPLICATE_POLICY` IS `LAST`; +1. The sample is added in-order (`timestamp ≥ max_timestamp`); 1. The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `IGNORE_MAX_TIME_DIFF`; -1. The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`; -1. The sample is added in-order (`timestamp ≥ max_timestamp`). +1. The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`. + +where `max_timestamp` is the timestamp of the sample with the largest timestamp in the time series, and `value_at_max_timestamp` is the value at `max_timestamp`. #### Defaults @@ -267,4 +275,4 @@ A new sample is considered a duplicate and is ignored if the following condition ``` $ redis-server --loadmodule ./redistimeseries.so IGNORE_MAX_TIME_DIFF 1 IGNORE_MAX_VALUE_DIFF 0.1 -``` \ No newline at end of file +``` From 95468d4a1a52e036b1172125500e4d9ff71b0a5d Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 13:34:54 +0300 Subject: [PATCH 02/14] Update index.md --- content/commands/ts.add/index.md | 24 +++++++----------------- 1 file changed, 7 insertions(+), 17 deletions(-) diff --git a/content/commands/ts.add/index.md b/content/commands/ts.add/index.md index 0e1159e288..a2a65aa27e 100644 --- a/content/commands/ts.add/index.md +++ b/content/commands/ts.add/index.md @@ -182,22 +182,12 @@ This argument has no effect when a new time series is created by this command. is the policy for handling duplicate samples. A new sample is considered a duplicate and is ignored if the following conditions are met: - - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `ignoreMaxTimeDiff`; - - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `ignoreMaxValDiff`; - - The sample is added in-order (`timestamp ≥ max_timestamp`). - -This can be expressed algorithmically as follows: - -``` -if ((series is not a compaction) && - (series' DUPLICATE_POLICY == LAST) && - (timestamp ≥ max_timestamp) && - (timestamp - max_timestamp ≤ ignoreMaxTimeDiff) && - abs(value - value_at_max_timestamp) ≤ ignoreMaxValDiff)) - - ignore sample -``` - + - The time series is not a compaction; + - The time series' `DUPLICATE_POLICY` IS `LAST`; + - The sample is added in-order (`timestamp ≥ max_timestamp`); + - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `IGNORE_MAX_TIME_DIFF`; + - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`. + where `max_timestamp` is the timestamp of the sample with the largest timestamp in the time series, and `value_at_max_timestamp` is the value at `max_timestamp`. When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. @@ -215,7 +205,7 @@ Use it only if you are creating a new time series. It is ignored if you are addi Notes: - You can use this command to create a new time series and add data to it in a single command. - `RETENTION`, `ENCODING`, `CHUNK_SIZE`, `DUPLICATE_POLICY`, and `LABELS` are used only when creating a new time series, and ignored when adding samples to an existing time series. + `RETENTION`, `ENCODING`, `CHUNK_SIZE`, `DUPLICATE_POLICY`, `IGNORE`, and `LABELS` are used only when creating a new time series, and ignored when adding samples to an existing time series. - Setting `RETENTION` and `LABELS` introduces additional time complexity. From 2d6ebbd851b83953c116ca46074567d3b1640abe Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 13:42:37 +0300 Subject: [PATCH 03/14] Update index.md --- content/commands/ts.incrby/index.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/content/commands/ts.incrby/index.md b/content/commands/ts.incrby/index.md index 14cd44bf7d..5c8ef4e5ab 100644 --- a/content/commands/ts.incrby/index.md +++ b/content/commands/ts.incrby/index.md @@ -143,9 +143,11 @@ Use it only if you are creating a new time series. It is ignored if you are addi is the policy for handling duplicate samples. A new sample is considered a duplicate and is ignored if the following conditions are met: - - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `ignoreMaxTimeDiff`; - - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `ignoreMaxValDiff`; - - The sample is added in-order (`timestamp ≥ max_timestamp`). + - The time series is not a compaction; + - The time series' `DUPLICATE_POLICY` IS `LAST`; + - The sample is added in-order (`timestamp ≥ max_timestamp`); + - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `IGNORE_MAX_TIME_DIFF`; + - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`. When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. @@ -160,8 +162,9 @@ Use it only if you are creating a new time series. It is ignored if you are addi Notes -- You can use this command to add data to a nonexisting time series in a single command. This is why `RETENTION`, `ENCODING`, `CHUNK_SIZE`, `DUPLICATE_POLICY`, and `LABELS` are optional arguments. -- When specified and the key doesn't exist, a new time series is created. Setting the `RETENTION` and `LABELS` introduces additional time complexity. +- You can use this command to create a new time series and add data to it in a single command. + `RETENTION`, `ENCODING`, `CHUNK_SIZE`, `DUPLICATE_POLICY`, `IGNORE`, and `LABELS` are used only when creating a new time series, and ignored when adding or modifying samples in an existing time series. +- Setting `RETENTION` and `LABELS` introduces additional time complexity. ## Return value From 89ca17932e99348404c416158fda20b5eb469bad Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 13:46:29 +0300 Subject: [PATCH 04/14] Update index.md --- content/commands/ts.decrby/index.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/content/commands/ts.decrby/index.md b/content/commands/ts.decrby/index.md index 99fadb7c9a..d3e4416c3a 100644 --- a/content/commands/ts.decrby/index.md +++ b/content/commands/ts.decrby/index.md @@ -142,9 +142,11 @@ Use it only if you are creating a new time series. It is ignored if you are addi is the policy for handling duplicate samples. A new sample is considered a duplicate and is ignored if the following conditions are met: - - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `ignoreMaxTimeDiff`; - - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `ignoreMaxValDiff`; - - The sample is added in-order (`timestamp ≥ max_timestamp`). + - The time series is not a compaction; + - The time series' `DUPLICATE_POLICY` IS `LAST`; + - The sample is added in-order (`timestamp ≥ max_timestamp`); + - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `IGNORE_MAX_TIME_DIFF`; + - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`. When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. @@ -159,9 +161,9 @@ Use it only if you are creating a new time series. It is ignored if you are addi Notes - - - You can use this command to add data to a nonexisting time series in a single command. This is why `RETENTION`, `ENCODING`, `CHUNK_SIZE`, `DUPLICATE_POLICY`, and `LABELS` are optional arguments. - - When specified and the key doesn't exist, a new time series is created. Setting the `RETENTION` and `LABELS` introduces additional time complexity. +- You can use this command to create a new time series and add a sample to it in a single command. + `RETENTION`, `ENCODING`, `CHUNK_SIZE`, `DUPLICATE_POLICY`, `IGNORE`, and `LABELS` are used only when creating a new time series, and ignored when adding or modifying samples in an existing time series. +- Setting `RETENTION` and `LABELS` introduces additional time complexity. ## Return value From fad35e785ef0c93732fc79354dd90cc9e50b2f10 Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 13:46:35 +0300 Subject: [PATCH 05/14] Update index.md --- content/commands/ts.incrby/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/commands/ts.incrby/index.md b/content/commands/ts.incrby/index.md index 5c8ef4e5ab..02d8508460 100644 --- a/content/commands/ts.incrby/index.md +++ b/content/commands/ts.incrby/index.md @@ -162,7 +162,7 @@ Use it only if you are creating a new time series. It is ignored if you are addi Notes -- You can use this command to create a new time series and add data to it in a single command. +- You can use this command to create a new time series and add a sample to it in a single command. `RETENTION`, `ENCODING`, `CHUNK_SIZE`, `DUPLICATE_POLICY`, `IGNORE`, and `LABELS` are used only when creating a new time series, and ignored when adding or modifying samples in an existing time series. - Setting `RETENTION` and `LABELS` introduces additional time complexity. From 1841a6815becd88604c30e82e5777b229a2c4e56 Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 13:47:43 +0300 Subject: [PATCH 06/14] Update index.md --- content/commands/ts.decrby/index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/commands/ts.decrby/index.md b/content/commands/ts.decrby/index.md index d3e4416c3a..3abc2c1212 100644 --- a/content/commands/ts.decrby/index.md +++ b/content/commands/ts.decrby/index.md @@ -148,6 +148,8 @@ is the policy for handling duplicate samples. A new sample is considered a dupli - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `IGNORE_MAX_TIME_DIFF`; - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`. +where `max_timestamp` is the timestamp of the sample with the largest timestamp in the time series, and `value_at_max_timestamp` is the value at `max_timestamp`. + When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. These parameters are used when creating a new time series to set the per-key parameters, and are ignored when called with an existing time series (the existing per-key configuration parameters is used). From 2c368153b273695468866458d2862720075ca671 Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 13:48:27 +0300 Subject: [PATCH 07/14] Update index.md --- content/commands/ts.incrby/index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/commands/ts.incrby/index.md b/content/commands/ts.incrby/index.md index 02d8508460..8e2d0e6eb9 100644 --- a/content/commands/ts.incrby/index.md +++ b/content/commands/ts.incrby/index.md @@ -149,6 +149,8 @@ is the policy for handling duplicate samples. A new sample is considered a dupli - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `IGNORE_MAX_TIME_DIFF`; - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`. +where `max_timestamp` is the timestamp of the sample with the largest timestamp in the time series, and `value_at_max_timestamp` is the value at `max_timestamp`. + When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. These parameters are used when creating a new time series to set the per-key parameters, and are ignored when called with an existing time series (the existing per-key configuration parameters is used). From d5556d7a4bea1ae6fc8bcba336f0fcb52a74490e Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 13:54:15 +0300 Subject: [PATCH 08/14] Update index.md --- content/commands/ts.madd/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/commands/ts.madd/index.md b/content/commands/ts.madd/index.md index c5f4532dc7..a1fa3d7a68 100644 --- a/content/commands/ts.madd/index.md +++ b/content/commands/ts.madd/index.md @@ -74,7 +74,7 @@ is numeric data value of the sample (double). The double number should follow }}), where each element is an [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) representing the timestamp of a upserted sample or an [] (when duplication policy is `BLOCK`, or when `timestamp` is older than the retention period compared to the maximum existing timestamp). For each element that is ignored (see the `IGNORE` option to `TS.ADD` for definitions), the element will have the value `max_timestamp`. +- [Array reply]({{< relref "/develop/reference/protocol-spec#arrays" >}}), where each element is an [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) representing the timestamp of a upserted sample or an [] (when duplication policy is `BLOCK`, or when `timestamp` is older than the retention period compared to the maximum existing timestamp). For each element that is ignored (see the `IGNORE` option in `TS.ADD` for definitions), the reply element value will be the largest timestamp in the time series. - [] (invalid arguments, wrong key type, etc.) ## Complexity From 76ec49f7572ad1e4c40d4265c8a4ee41870864a2 Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 13:59:39 +0300 Subject: [PATCH 09/14] Update index.md --- content/commands/ts.alter/index.md | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/content/commands/ts.alter/index.md b/content/commands/ts.alter/index.md index 4ed16cc175..66aabcfd05 100644 --- a/content/commands/ts.alter/index.md +++ b/content/commands/ts.alter/index.md @@ -65,9 +65,9 @@ stack_path: docs/data-types/timeseries summary: Update the retention, chunk size, duplicate policy, and labels of an existing time series syntax: "TS.ALTER key \n [RETENTION retentionPeriod] \n [CHUNK_SIZE size] \n [DUPLICATE_POLICY\ - \ policy] \n [IGNORE ignoreMaxTimediff ignoreMaxValDiff] [LABELS [label value ...]]\n" + \ policy] \n [IGNORE ignoreMaxTimediff ignoreMaxValDiff] \n [LABELS [label value ...]]\n" syntax_fmt: "TS.ALTER key [RETENTION\_retentionPeriod] [CHUNK_SIZE\_size]\n [DUPLICATE_POLICY\_\ - ]\n [IGNORE ignoreMaxTimediff ignoreMaxValDiff] [LABELS\ [label value ...]]" + ]\n [IGNORE ignoreMaxTimediff ignoreMaxValDiff] \n [LABELS\ [label value ...]]" syntax_str: "[RETENTION\_retentionPeriod] [CHUNK_SIZE\_size] [DUPLICATE_POLICY\_] [IGNORE ignoreMaxTimediff ignoreMaxValDiff] [LABELS\ [label value ...]]" title: TS.ALTER @@ -105,15 +105,7 @@ is policy for handling multiple samples with identical timestamps. See `DUPLICAT
IGNORE ignoreMaxTimediff ignoreMaxValDiff -is the policy for handling duplicate samples. A new sample is considered a duplicate and is ignored if the following conditions are met: - - - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `ignoreMaxTimeDiff`; - - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `ignoreMaxValDiff`; - - The sample is added in-order (`timestamp ≥ max_timestamp`). - -When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. - -See [`TS.ADD`]({{< baseurl >}}/commands/ts.add/) for more details. +is the policy for handling duplicate samples. See `IGNORE` in [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/).
LABELS [{label value}...] From e47f0969bc9cd13dab78c58192198106553d3041 Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 14:01:15 +0300 Subject: [PATCH 10/14] Update index.md --- content/commands/ts.create/index.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/content/commands/ts.create/index.md b/content/commands/ts.create/index.md index b432884286..ce8599ad76 100644 --- a/content/commands/ts.create/index.md +++ b/content/commands/ts.create/index.md @@ -155,13 +155,15 @@ is policy for handling insertion ([`TS.ADD`]({{< baseurl >}}/commands/ts.add/) a is the policy for handling duplicate samples. A new sample is considered a duplicate and is ignored if the following conditions are met: - - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `ignoreMaxTimeDiff`; - - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `ignoreMaxValDiff`; - - The sample is added in-order (`timestamp ≥ max_timestamp`). + - The time series is not a compaction; + - The time series' `DUPLICATE_POLICY` IS `LAST`; + - The sample is added in-order (`timestamp ≥ max_timestamp`); + - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `IGNORE_MAX_TIME_DIFF`; + - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`. + +where `max_timestamp` is the timestamp of the sample with the largest timestamp in the time series, and `value_at_max_timestamp` is the value at `max_timestamp`. When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. - -See [`TS.ADD`]({{< baseurl >}}/commands/ts.add/) for more details.
LABELS {label value}... From 65859b6d6dd2bdf2ff4a3312edfbcdec871381f9 Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 14:23:59 +0300 Subject: [PATCH 11/14] Update index.md --- content/commands/ts.decrby/index.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/content/commands/ts.decrby/index.md b/content/commands/ts.decrby/index.md index 3abc2c1212..a6d77e1146 100644 --- a/content/commands/ts.decrby/index.md +++ b/content/commands/ts.decrby/index.md @@ -92,6 +92,7 @@ is numeric value of the subtrahend (double). Notes - When specified key does not exist, a new time series is created. - You can use this command as a counter or gauge that automatically gets history as a time series. +- If a policy for handling duplicate samples (`IGNORE`) is defined for this time series - `TS.DECRBY` operations are affected as well (sample additions/modifications can be filtered). - Explicitly adding samples to a compacted time series (using [`TS.ADD`]({{< baseurl >}}/commands/ts.add/), [`TS.MADD`]({{< baseurl >}}/commands/ts.madd/), [`TS.INCRBY`]({{< baseurl >}}/commands/ts.incrby/), or `TS.DECRBY`) may result in inconsistencies between the raw and the compacted data. The compaction process may override such samples. @@ -152,7 +153,7 @@ where `max_timestamp` is the timestamp of the sample with the largest timestamp When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. -These parameters are used when creating a new time series to set the per-key parameters, and are ignored when called with an existing time series (the existing per-key configuration parameters is used). +These parameters are used when creating a new time series to set the per-key parameters, and are ignored when called with an existing time series (the existing per-key configuration parameters are used).
LABELS [{label value}...] @@ -172,7 +173,7 @@ Use it only if you are creating a new time series. It is ignored if you are addi Returns one of these replies: -- [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) - the timestamp of the upserted sample +- [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) - the timestamp of the upserted sample. If the sample is ignored (See `IGNORE` in [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/)), the reply will be the largest timestamp in the time series. - [] on error (invalid arguments, wrong key type, etc.), or when `timestamp` is not equal to or higher than the maximum existing timestamp ## See also From d8878657358d531cf2257ec6fa9c2b3a50db2da5 Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 14:24:06 +0300 Subject: [PATCH 12/14] Update index.md --- content/commands/ts.incrby/index.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/content/commands/ts.incrby/index.md b/content/commands/ts.incrby/index.md index 8e2d0e6eb9..22d13fe775 100644 --- a/content/commands/ts.incrby/index.md +++ b/content/commands/ts.incrby/index.md @@ -93,6 +93,7 @@ is numeric value of the addend (double). Notes - When specified key does not exist, a new time series is created. - You can use this command as a counter or gauge that automatically gets history as a time series. +- If a policy for handling duplicate samples (`IGNORE`) is defined for this time series - `TS.INCRBY` operations are affected as well (sample additions/modifications can be filtered). - Explicitly adding samples to a compacted time series (using [`TS.ADD`]({{< baseurl >}}/commands/ts.add/), [`TS.MADD`]({{< baseurl >}}/commands/ts.madd/), `TS.INCRBY`, or [`TS.DECRBY`]({{< baseurl >}}/commands/ts.decrby/)) may result in inconsistencies between the raw and the compacted data. The compaction process may override such samples. @@ -104,7 +105,7 @@ is Unix time (integer, in milliseconds) specifying the sample timestamp or `*` t Unix time is the number of milliseconds that have elapsed since 00:00:00 UTC on 1 January 1970, the Unix epoch, without adjustments made due to leap seconds. -`timestamp` must be equal to or higher than the maximum existing timestamp. When equal, the value of the sample with the maximum existing timestamp is increased. If it is higher, a new sample with a timestamp set to `timestamp` is created, and its value is set to the value of the sample with the maximum existing timestamp plus `addend`. +`timestamp` must be equal to or higher than the maximum existing timestamp. When equal, the value of the sample with the maximum existing timestamp is increased. If it is higher, a new sample with a timestamp set to `timestamp` is created, and its value is set to the value of the sample with the maximum existing timestamp plus `addend`. If the time series is empty, the value is set to `addend`. @@ -153,7 +154,7 @@ where `max_timestamp` is the timestamp of the sample with the largest timestamp When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. -These parameters are used when creating a new time series to set the per-key parameters, and are ignored when called with an existing time series (the existing per-key configuration parameters is used). +These parameters are used when creating a new time series to set the per-key parameters, and are ignored when called with an existing time series (the existing per-key configuration parameters are used).
LABELS [{label value}...] @@ -173,7 +174,7 @@ Use it only if you are creating a new time series. It is ignored if you are addi Returns one of these replies: -- [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) - the timestamp of the upserted sample +- [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) - the timestamp of the upserted sample. If the sample is ignored (See `IGNORE` in [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/)), the reply will be the largest timestamp in the time series. - [] on error (invalid arguments, wrong key type, etc.), or when `timestamp` is not equal to or higher than the maximum existing timestamp ## Examples From d0c796bef73a729e5e0c58c4fac95b757c1282cc Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 14:25:07 +0300 Subject: [PATCH 13/14] Update index.md --- content/commands/ts.add/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/commands/ts.add/index.md b/content/commands/ts.add/index.md index a2a65aa27e..4563d30a67 100644 --- a/content/commands/ts.add/index.md +++ b/content/commands/ts.add/index.md @@ -213,7 +213,7 @@ Use it only if you are creating a new time series. It is ignored if you are addi Returns one of these replies: -- [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) - the timestamp of the upserted sample. For an element that is ignored (see the `IGNORE` parameter above), the value will be `max_timestamp`. +- [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) - the timestamp of the upserted sample. If the sample is ignored (See `IGNORE` in [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/)), the reply will be the largest timestamp in the time series. - [] on error (invalid arguments, wrong key type, etc.), when duplication policy is `BLOCK`, or when `timestamp` is older than the retention period compared to the maximum existing timestamp ## Complexity From e5c07446e85577a5c8483fce9872483e4a992cc9 Mon Sep 17 00:00:00 2001 From: Lior Kogan Date: Tue, 11 Jun 2024 14:27:36 +0300 Subject: [PATCH 14/14] Update index.md --- content/commands/ts.madd/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/commands/ts.madd/index.md b/content/commands/ts.madd/index.md index a1fa3d7a68..292eac9f00 100644 --- a/content/commands/ts.madd/index.md +++ b/content/commands/ts.madd/index.md @@ -74,7 +74,7 @@ is numeric data value of the sample (double). The double number should follow }}), where each element is an [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) representing the timestamp of a upserted sample or an [] (when duplication policy is `BLOCK`, or when `timestamp` is older than the retention period compared to the maximum existing timestamp). For each element that is ignored (see the `IGNORE` option in `TS.ADD` for definitions), the reply element value will be the largest timestamp in the time series. +- [Array reply]({{< relref "/develop/reference/protocol-spec#arrays" >}}), where each element is an [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) representing the timestamp of a upserted sample or an [] (when duplication policy is `BLOCK`, or when `timestamp` is older than the retention period compared to the maximum existing timestamp). For each element that is ignored (see `IGNORE` in [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/)), the reply element value will be the largest timestamp in the time series. - [] (invalid arguments, wrong key type, etc.) ## Complexity