Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggregator plugins: Support historical data #1992

Closed
sparrc opened this issue Nov 4, 2016 · 5 comments
Closed

Aggregator plugins: Support historical data #1992

sparrc opened this issue Nov 4, 2016 · 5 comments
Labels
feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin

Comments

@sparrc
Copy link
Contributor

sparrc commented Nov 4, 2016

Currently aggregator plugins only support aggregating data with "current" timestamps.

We should try to support a single stream of historical data in the aggregator plugins, as this likely could be a common use-case when parsing a logfile.

see #1989

from that issue:

One "easy" workaround might be to create a single "historical window" which would have it's period started on the timestamp of the first "historical" metric that arrives. This would cover your use-case where you have a single stream of historical and ordered metrics arriving (ordered is the crucial point). I imagine that when dealing with historical data, this would be the most common use-case, especially when dealing with logfile parsing.

@sparrc sparrc added the feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin label Nov 4, 2016
@sparrc sparrc added this to the 1.3.0 milestone Nov 4, 2016
@sparrc sparrc modified the milestones: Future Milestone, 1.3.0 Nov 4, 2016
@ghost
Copy link

ghost commented Apr 6, 2017

Is there [[aggregator.mean]] plugin available in telegraf ? I am searching through but couldn't find if there is any aggregator plugin other than [[aggregator.minmax]].

@danielnelson
Copy link
Contributor

@ghost Not currently but there is one in the works for 1.3.0 #2167

@bolek2000
Copy link

we are testing log parsing with telegraf and plan to import older logfiles into the DB. It will be handy if we can use the aggregator for historical data too.

@kevin-cantwell
Copy link

Can this issue be closed now with the advent of #6049 ?

@ssoroka
Copy link
Contributor

ssoroka commented Feb 12, 2021

I think so. If the grace period doesn't meet your needs, you also have the ability to use the state variable in a custom starlark script to write any sort of aggregation rules you would like, at least as far as available memory will allow.

@ssoroka ssoroka closed this as completed Feb 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin
Projects
None yet
Development

No branches or pull requests

5 participants