diff --git a/content/commands/ts.add/index.md b/content/commands/ts.add/index.md index 0e1159e288..4563d30a67 100644 --- a/content/commands/ts.add/index.md +++ b/content/commands/ts.add/index.md @@ -182,22 +182,12 @@ This argument has no effect when a new time series is created by this command. is the policy for handling duplicate samples. A new sample is considered a duplicate and is ignored if the following conditions are met: - - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `ignoreMaxTimeDiff`; - - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `ignoreMaxValDiff`; - - The sample is added in-order (`timestamp ≥ max_timestamp`). - -This can be expressed algorithmically as follows: - -``` -if ((series is not a compaction) && - (series' DUPLICATE_POLICY == LAST) && - (timestamp ≥ max_timestamp) && - (timestamp - max_timestamp ≤ ignoreMaxTimeDiff) && - abs(value - value_at_max_timestamp) ≤ ignoreMaxValDiff)) - - ignore sample -``` - + - The time series is not a compaction; + - The time series' `DUPLICATE_POLICY` IS `LAST`; + - The sample is added in-order (`timestamp ≥ max_timestamp`); + - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `IGNORE_MAX_TIME_DIFF`; + - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`. + where `max_timestamp` is the timestamp of the sample with the largest timestamp in the time series, and `value_at_max_timestamp` is the value at `max_timestamp`. When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. @@ -215,7 +205,7 @@ Use it only if you are creating a new time series. It is ignored if you are addi Notes: - You can use this command to create a new time series and add data to it in a single command. - `RETENTION`, `ENCODING`, `CHUNK_SIZE`, `DUPLICATE_POLICY`, and `LABELS` are used only when creating a new time series, and ignored when adding samples to an existing time series. + `RETENTION`, `ENCODING`, `CHUNK_SIZE`, `DUPLICATE_POLICY`, `IGNORE`, and `LABELS` are used only when creating a new time series, and ignored when adding samples to an existing time series. - Setting `RETENTION` and `LABELS` introduces additional time complexity. @@ -223,7 +213,7 @@ Use it only if you are creating a new time series. It is ignored if you are addi Returns one of these replies: -- [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) - the timestamp of the upserted sample. For an element that is ignored (see the `IGNORE` parameter above), the value will be `max_timestamp`. +- [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) - the timestamp of the upserted sample. If the sample is ignored (See `IGNORE` in [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/)), the reply will be the largest timestamp in the time series. - [] on error (invalid arguments, wrong key type, etc.), when duplication policy is `BLOCK`, or when `timestamp` is older than the retention period compared to the maximum existing timestamp ## Complexity diff --git a/content/commands/ts.alter/index.md b/content/commands/ts.alter/index.md index 4ed16cc175..66aabcfd05 100644 --- a/content/commands/ts.alter/index.md +++ b/content/commands/ts.alter/index.md @@ -65,9 +65,9 @@ stack_path: docs/data-types/timeseries summary: Update the retention, chunk size, duplicate policy, and labels of an existing time series syntax: "TS.ALTER key \n [RETENTION retentionPeriod] \n [CHUNK_SIZE size] \n [DUPLICATE_POLICY\ - \ policy] \n [IGNORE ignoreMaxTimediff ignoreMaxValDiff] [LABELS [label value ...]]\n" + \ policy] \n [IGNORE ignoreMaxTimediff ignoreMaxValDiff] \n [LABELS [label value ...]]\n" syntax_fmt: "TS.ALTER key [RETENTION\_retentionPeriod] [CHUNK_SIZE\_size]\n [DUPLICATE_POLICY\_\ - ]\n [IGNORE ignoreMaxTimediff ignoreMaxValDiff] [LABELS\ [label value ...]]" + ]\n [IGNORE ignoreMaxTimediff ignoreMaxValDiff] \n [LABELS\ [label value ...]]" syntax_str: "[RETENTION\_retentionPeriod] [CHUNK_SIZE\_size] [DUPLICATE_POLICY\_] [IGNORE ignoreMaxTimediff ignoreMaxValDiff] [LABELS\ [label value ...]]" title: TS.ALTER @@ -105,15 +105,7 @@ is policy for handling multiple samples with identical timestamps. See `DUPLICAT
IGNORE ignoreMaxTimediff ignoreMaxValDiff -is the policy for handling duplicate samples. A new sample is considered a duplicate and is ignored if the following conditions are met: - - - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `ignoreMaxTimeDiff`; - - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `ignoreMaxValDiff`; - - The sample is added in-order (`timestamp ≥ max_timestamp`). - -When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. - -See [`TS.ADD`]({{< baseurl >}}/commands/ts.add/) for more details. +is the policy for handling duplicate samples. See `IGNORE` in [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/).
LABELS [{label value}...] diff --git a/content/commands/ts.create/index.md b/content/commands/ts.create/index.md index b432884286..ce8599ad76 100644 --- a/content/commands/ts.create/index.md +++ b/content/commands/ts.create/index.md @@ -155,13 +155,15 @@ is policy for handling insertion ([`TS.ADD`]({{< baseurl >}}/commands/ts.add/) a is the policy for handling duplicate samples. A new sample is considered a duplicate and is ignored if the following conditions are met: - - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `ignoreMaxTimeDiff`; - - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `ignoreMaxValDiff`; - - The sample is added in-order (`timestamp ≥ max_timestamp`). + - The time series is not a compaction; + - The time series' `DUPLICATE_POLICY` IS `LAST`; + - The sample is added in-order (`timestamp ≥ max_timestamp`); + - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `IGNORE_MAX_TIME_DIFF`; + - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`. + +where `max_timestamp` is the timestamp of the sample with the largest timestamp in the time series, and `value_at_max_timestamp` is the value at `max_timestamp`. When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. - -See [`TS.ADD`]({{< baseurl >}}/commands/ts.add/) for more details.
LABELS {label value}... diff --git a/content/commands/ts.decrby/index.md b/content/commands/ts.decrby/index.md index 99fadb7c9a..a6d77e1146 100644 --- a/content/commands/ts.decrby/index.md +++ b/content/commands/ts.decrby/index.md @@ -92,6 +92,7 @@ is numeric value of the subtrahend (double). Notes - When specified key does not exist, a new time series is created. - You can use this command as a counter or gauge that automatically gets history as a time series. +- If a policy for handling duplicate samples (`IGNORE`) is defined for this time series - `TS.DECRBY` operations are affected as well (sample additions/modifications can be filtered). - Explicitly adding samples to a compacted time series (using [`TS.ADD`]({{< baseurl >}}/commands/ts.add/), [`TS.MADD`]({{< baseurl >}}/commands/ts.madd/), [`TS.INCRBY`]({{< baseurl >}}/commands/ts.incrby/), or `TS.DECRBY`) may result in inconsistencies between the raw and the compacted data. The compaction process may override such samples. @@ -142,13 +143,17 @@ Use it only if you are creating a new time series. It is ignored if you are addi is the policy for handling duplicate samples. A new sample is considered a duplicate and is ignored if the following conditions are met: - - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `ignoreMaxTimeDiff`; - - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `ignoreMaxValDiff`; - - The sample is added in-order (`timestamp ≥ max_timestamp`). + - The time series is not a compaction; + - The time series' `DUPLICATE_POLICY` IS `LAST`; + - The sample is added in-order (`timestamp ≥ max_timestamp`); + - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `IGNORE_MAX_TIME_DIFF`; + - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`. + +where `max_timestamp` is the timestamp of the sample with the largest timestamp in the time series, and `value_at_max_timestamp` is the value at `max_timestamp`. When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. -These parameters are used when creating a new time series to set the per-key parameters, and are ignored when called with an existing time series (the existing per-key configuration parameters is used). +These parameters are used when creating a new time series to set the per-key parameters, and are ignored when called with an existing time series (the existing per-key configuration parameters are used).
LABELS [{label value}...] @@ -159,16 +164,16 @@ Use it only if you are creating a new time series. It is ignored if you are addi
Notes - - - You can use this command to add data to a nonexisting time series in a single command. This is why `RETENTION`, `ENCODING`, `CHUNK_SIZE`, `DUPLICATE_POLICY`, and `LABELS` are optional arguments. - - When specified and the key doesn't exist, a new time series is created. Setting the `RETENTION` and `LABELS` introduces additional time complexity. +- You can use this command to create a new time series and add a sample to it in a single command. + `RETENTION`, `ENCODING`, `CHUNK_SIZE`, `DUPLICATE_POLICY`, `IGNORE`, and `LABELS` are used only when creating a new time series, and ignored when adding or modifying samples in an existing time series. +- Setting `RETENTION` and `LABELS` introduces additional time complexity. ## Return value Returns one of these replies: -- [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) - the timestamp of the upserted sample +- [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) - the timestamp of the upserted sample. If the sample is ignored (See `IGNORE` in [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/)), the reply will be the largest timestamp in the time series. - [] on error (invalid arguments, wrong key type, etc.), or when `timestamp` is not equal to or higher than the maximum existing timestamp ## See also diff --git a/content/commands/ts.incrby/index.md b/content/commands/ts.incrby/index.md index 14cd44bf7d..22d13fe775 100644 --- a/content/commands/ts.incrby/index.md +++ b/content/commands/ts.incrby/index.md @@ -93,6 +93,7 @@ is numeric value of the addend (double). Notes - When specified key does not exist, a new time series is created. - You can use this command as a counter or gauge that automatically gets history as a time series. +- If a policy for handling duplicate samples (`IGNORE`) is defined for this time series - `TS.INCRBY` operations are affected as well (sample additions/modifications can be filtered). - Explicitly adding samples to a compacted time series (using [`TS.ADD`]({{< baseurl >}}/commands/ts.add/), [`TS.MADD`]({{< baseurl >}}/commands/ts.madd/), `TS.INCRBY`, or [`TS.DECRBY`]({{< baseurl >}}/commands/ts.decrby/)) may result in inconsistencies between the raw and the compacted data. The compaction process may override such samples. @@ -104,7 +105,7 @@ is Unix time (integer, in milliseconds) specifying the sample timestamp or `*` t Unix time is the number of milliseconds that have elapsed since 00:00:00 UTC on 1 January 1970, the Unix epoch, without adjustments made due to leap seconds. -`timestamp` must be equal to or higher than the maximum existing timestamp. When equal, the value of the sample with the maximum existing timestamp is increased. If it is higher, a new sample with a timestamp set to `timestamp` is created, and its value is set to the value of the sample with the maximum existing timestamp plus `addend`. +`timestamp` must be equal to or higher than the maximum existing timestamp. When equal, the value of the sample with the maximum existing timestamp is increased. If it is higher, a new sample with a timestamp set to `timestamp` is created, and its value is set to the value of the sample with the maximum existing timestamp plus `addend`. If the time series is empty, the value is set to `addend`. @@ -143,13 +144,17 @@ Use it only if you are creating a new time series. It is ignored if you are addi is the policy for handling duplicate samples. A new sample is considered a duplicate and is ignored if the following conditions are met: - - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `ignoreMaxTimeDiff`; - - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `ignoreMaxValDiff`; - - The sample is added in-order (`timestamp ≥ max_timestamp`). + - The time series is not a compaction; + - The time series' `DUPLICATE_POLICY` IS `LAST`; + - The sample is added in-order (`timestamp ≥ max_timestamp`); + - The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `IGNORE_MAX_TIME_DIFF`; + - The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`. + +where `max_timestamp` is the timestamp of the sample with the largest timestamp in the time series, and `value_at_max_timestamp` is the value at `max_timestamp`. When not specified: set to the global [IGNORE_MAX_TIME_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff) and [IGNORE_MAX_VAL_DIFF]({{< baseurl >}}/develop/data-types/timeseries/configuration#ignore_max_time_diff-and-ignore_max_val_diff), which are, by default, both set to 0. -These parameters are used when creating a new time series to set the per-key parameters, and are ignored when called with an existing time series (the existing per-key configuration parameters is used). +These parameters are used when creating a new time series to set the per-key parameters, and are ignored when called with an existing time series (the existing per-key configuration parameters are used).
LABELS [{label value}...] @@ -160,15 +165,16 @@ Use it only if you are creating a new time series. It is ignored if you are addi
Notes -- You can use this command to add data to a nonexisting time series in a single command. This is why `RETENTION`, `ENCODING`, `CHUNK_SIZE`, `DUPLICATE_POLICY`, and `LABELS` are optional arguments. -- When specified and the key doesn't exist, a new time series is created. Setting the `RETENTION` and `LABELS` introduces additional time complexity. +- You can use this command to create a new time series and add a sample to it in a single command. + `RETENTION`, `ENCODING`, `CHUNK_SIZE`, `DUPLICATE_POLICY`, `IGNORE`, and `LABELS` are used only when creating a new time series, and ignored when adding or modifying samples in an existing time series. +- Setting `RETENTION` and `LABELS` introduces additional time complexity. ## Return value Returns one of these replies: -- [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) - the timestamp of the upserted sample +- [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) - the timestamp of the upserted sample. If the sample is ignored (See `IGNORE` in [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/)), the reply will be the largest timestamp in the time series. - [] on error (invalid arguments, wrong key type, etc.), or when `timestamp` is not equal to or higher than the maximum existing timestamp ## Examples diff --git a/content/commands/ts.madd/index.md b/content/commands/ts.madd/index.md index c5f4532dc7..292eac9f00 100644 --- a/content/commands/ts.madd/index.md +++ b/content/commands/ts.madd/index.md @@ -74,7 +74,7 @@ is numeric data value of the sample (double). The double number should follow }}), where each element is an [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) representing the timestamp of a upserted sample or an [] (when duplication policy is `BLOCK`, or when `timestamp` is older than the retention period compared to the maximum existing timestamp). For each element that is ignored (see the `IGNORE` option to `TS.ADD` for definitions), the element will have the value `max_timestamp`. +- [Array reply]({{< relref "/develop/reference/protocol-spec#arrays" >}}), where each element is an [Integer reply]({{< relref "/develop/reference/protocol-spec#integers" >}}) representing the timestamp of a upserted sample or an [] (when duplication policy is `BLOCK`, or when `timestamp` is older than the retention period compared to the maximum existing timestamp). For each element that is ignored (see `IGNORE` in [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/)), the reply element value will be the largest timestamp in the time series. - [] (invalid arguments, wrong key type, etc.) ## Complexity diff --git a/content/develop/data-types/timeseries/configuration.md b/content/develop/data-types/timeseries/configuration.md index 10f7636ca5..8454894b24 100644 --- a/content/develop/data-types/timeseries/configuration.md +++ b/content/develop/data-types/timeseries/configuration.md @@ -44,17 +44,17 @@ $ redis-server --loadmodule ./redistimeseries.so [OPT VAL]... The following table summarizes which configuration parameters can be set at module load-time and run-time: -| Configuration Parameter | Load-time | Run-time | -| :------- | :----- | :----------- | -| [NUM_THREADS](#num_threads) (since RedisTimeSeries v1.6) | :white_check_mark: | :white_large_square: | -| [COMPACTION_POLICY](#compaction_policy) | :white_check_mark: | :white_large_square: | -| [RETENTION_POLICY](#retention_policy) | :white_check_mark: | :white_large_square: | -| [DUPLICATE_POLICY](#duplicate_policy) | :white_check_mark: | :white_large_square: | -| [ENCODING](#encoding) (since RedisTimeSeries v1.6) | :white_check_mark: | :white_large_square: | -| [CHUNK_SIZE_BYTES](#chunk_size_bytes) | :white_check_mark: | :white_large_square: | -| [OSS_GLOBAL_PASSWORD](#oss_global_password) (since RedisTimeSeries v1.8.4) | :white_check_mark: | :white_large_square: | +| Configuration Parameter | Load-time | Run-time | +| :------- | :----- | :----------- | +| [NUM_THREADS](#num_threads) (since RedisTimeSeries v1.6) | :white_check_mark: | :white_large_square: | +| [COMPACTION_POLICY](#compaction_policy) | :white_check_mark: | :white_large_square: | +| [RETENTION_POLICY](#retention_policy) | :white_check_mark: | :white_large_square: | +| [DUPLICATE_POLICY](#duplicate_policy) | :white_check_mark: | :white_large_square: | +| [ENCODING](#encoding) (since RedisTimeSeries v1.6) | :white_check_mark: | :white_large_square: | +| [CHUNK_SIZE_BYTES](#chunk_size_bytes) | :white_check_mark: | :white_large_square: | +| [OSS_GLOBAL_PASSWORD](#oss_global_password) (since RedisTimeSeries v1.8.4) | :white_check_mark: | :white_large_square: | | [IGNORE_MAX_TIME_DIFF](#ignore_max_time_diff-and-ignore_max_val_diff) (since RedisTimeSeries v1.12) | :white_check_mark: | :white_large_square: | -| [IGNORE_MAX_VAL_DIFF](#ignore_max_time_diff-and-ignore_max_val_diff) (since RedisTimeSeries v1.12) | :white_check_mark: | :white_large_square: | +| [IGNORE_MAX_VAL_DIFF](#ignore_max_time_diff-and-ignore_max_val_diff) (since RedisTimeSeries v1.12) | :white_check_mark: | :white_large_square: | ### NUM_THREADS @@ -72,7 +72,7 @@ $ redis-server --loadmodule ./redistimeseries.so NUM_THREADS 3 ### COMPACTION_POLICY -Default compaction rules for newly created key with [`TS.ADD`]({{< baseurl >}}/commands/ts.add/). +Default compaction rules for newly created key with [`TS.ADD`]({{< baseurl >}}/commands/ts.add/), [`TS.INCRBY`]({{< baseurl >}}/commands/ts.incrby/), and [`TS.DECRBY`]({{< baseurl >}}/commands/ts.decrby/). Note that `COMPACTION_POLICY` has no effect on keys created with [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/). To understand the motivation for this behavior, consider the following scenario: Suppose a `COMPACTION_POLICY` is defined, but then one wants to manually create an additional compaction rule (using [`TS.CREATERULE`]({{< baseurl >}}/commands/ts.createrule/)) which requires first creating an empty destination key (using [`TS.CREATE`]({{< baseurl >}}/commands/ts.create/)). But now there is a problem: due to the `COMPACTION_POLICY`, automatic compactions would be undesirably created for that destination key. @@ -251,11 +251,19 @@ $ redis-server --loadmodule ./redistimeseries.so OSS_GLOBAL_PASSWORD password ### IGNORE_MAX_TIME_DIFF and IGNORE_MAX_VAL_DIFF +Default values for newly created keys. + +Many sensors report data periodically. Often, the difference between the measured value and the previous measured value is negligible and related to random noise or to measurement accuracy limitations. In such situations it may be preferable not to add the new measurement to the time series. + A new sample is considered a duplicate and is ignored if the following conditions are met: +1. The time series is not a compaction; +1. The time series' `DUPLICATE_POLICY` IS `LAST`; +1. The sample is added in-order (`timestamp ≥ max_timestamp`); 1. The difference of the current timestamp from the previous timestamp (`timestamp - max_timestamp`) is less than or equal to `IGNORE_MAX_TIME_DIFF`; -1. The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`; -1. The sample is added in-order (`timestamp ≥ max_timestamp`). +1. The absolute value difference of the current value from the value at the previous maximum timestamp (`abs(value - value_at_max_timestamp`) is less than or equal to `IGNORE_MAX_VAL_DIFF`. + +where `max_timestamp` is the timestamp of the sample with the largest timestamp in the time series, and `value_at_max_timestamp` is the value at `max_timestamp`. #### Defaults @@ -267,4 +275,4 @@ A new sample is considered a duplicate and is ignored if the following condition ``` $ redis-server --loadmodule ./redistimeseries.so IGNORE_MAX_TIME_DIFF 1 IGNORE_MAX_VALUE_DIFF 0.1 -``` \ No newline at end of file +```