docs(aggregators.final): Swap useful statements

[powersj]

Summary

The statements seem switched around currently and are causing confusion for users. Periodic is used to avoid gaps in the data, while timeout is used to only show the last value when a whole bunch come in, i.e. downsample.

https://community.influxdata.com/t/get-at-least-every-5-min-datapoint-by-mqtt-consumer/33911/8

Checklist

• No AI generated code was used in this PR

[srebhan]

I don't think the new wording is correct. Ontimeout all metrics received within the timeout are swallow and the aggregator only outputs the last metric if it is older than the timeout! So it "fills a gap" if no metric arrived within timeout.

For downsampling you want to get the data in a fixed interval (usually with a lower frequency than the input). This is exactly what the periodic strategy does, it outputs the last metric received at a fixed interval (the period). Here the "age" of the metric is irrelevant.

[Hipska]

See this post for reference how we came to this conclusion. In the use-case of the TS, the current wording is very confusing.

[powersj]

So it "fills a gap" if no metric arrived within timeout.

This sentence is what is causing confusion. If no metrics are received within a timeout, then nothing is returned. i.e. if no metrics, a gap is produced. The text literally says to use "timeout" to fill gaps, but that is not the behavior.

For example, given the following config, and if I send a single metric:

[[aggregators.final]]
  period = "10s"
  delay = "5s"
  output_strategy = "timeout"

2024-04-16T13:53:00Z D! [aggregators.final] Updated aggregation range [2024-04-16 07:53:00 -0600 MDT, 2024-04-16 07:53:10 -0600 MDT]
metric,topic=telegraf/test value=43 1713275576116715552
2024-04-16T13:53:02Z D! [outputs.file] Wrote batch of 1 metrics in 51.7µs
2024-04-16T13:53:02Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2024-04-16T13:53:10Z D! [aggregators.final] Updated aggregation range [2024-04-16 07:53:10 -0600 MDT, 2024-04-16 07:53:20 -0600 MDT]
2024-04-16T13:53:12Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2024-04-16T13:53:20Z D! [aggregators.final] Updated aggregation range [2024-04-16 07:53:20 -0600 MDT, 2024-04-16 07:53:30 -0600 MDT]
2024-04-16T13:53:22Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2024-04-16T13:53:30Z D! [aggregators.final] Updated aggregation range [2024-04-16 07:53:30 -0600 MDT, 2024-04-16 07:53:40 -0600 MDT]
2024-04-16T13:53:32Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics

[srebhan]

@powersj I agree that this can cause confusion, but the statement of this PR

  ##   timeout  -- output a metric if no new input arrived for `series_timeout`;
  ##               useful to down sample the input data
  ##   periodic -- output the last received metric every `period`;
  ##               useful for filling gaps in input data

is just wrong. You won't be able to downsample using timeout.

[powersj]

You won't be able to downsample using timeout.

Your definition above, is that downsampling is about periodic return of data, but I can also read downsample as taking many measurements and reducing them down to a single one. Reducing the rate at which data is transmited. If you receive 100 values during one period, and final returns only the last one, is that not a downsampling?

[srebhan]

@powersj the definition of downsampling references a "rate" so my take is that is equidistant. But no matter if this is true or not, the timeout strategy will only output data on timeout So if you really get data at a regular interval, the aggregator will never output anything! That's not downsampling, is it?

[powersj]

the definition of downsampling references a "rate" so my take is that is equidistant. But no matter if this is true or not,

Unfortunately, I will disagree and state that this absolutely definition does matter.

In terms of telegraf, users are generally not doing signal processing. It is probably better to reference a definition specific to metrics and data, for example from our own blog post on downsamplling.

[powersj]

I want to specifically call out the following quote:

Additionally, I’ve seen many users ask: “How can I downsample my data to retrieve the mean value and value count from my data at every aggregation interval?”

Replace "mean" with "final".

Given that question from a user, what would you tell them? Use final with periodic or final with timeout?

I would argue that you could use either, and the key difference is if you want data to fill in gaps via periodic or not.

[srebhan]

@powersj so assume a metric that really is coming at a constant rate of 1 sample per minute and use the timeout strategy. How would that ever downsample as the aggregator will never output any metric!?!?

Given that question from a user, what would you tell them? Use final with periodic or final with timeout?

You can ONLY use periodic to achieve that... The period is then the interval for the aggregation and the sampling period...

[DStrand1]

LGTM!

[telegraf-tiger]

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

👍 This pull request doesn't change the Telegraf binary size

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB	RPM	TAR GZ	ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_arm64.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz

[srebhan]

Thanks @powersj!