Filter out messages by metric values (not tags)

[jasonkeller]

Feature Request

Proposal:

Enable filtering based on metric values

Current behavior:

Thus far I have only found input/output filtering based on tags, instead of metric values

Desired behavior:

I need to be able to drop entire messages when two metrics are both zero

Use case: [Why is this important (helps with prioritizing requests)]

I will be performing SNMP queries for all interface octet counters on around 1800 devices; unfortunately the Extreme switches are sending in tons of message lines with not only physical ports (many of them unused), but also VLANs which perennially have 0 octets on their counters. This information is useless and will only bloat the database. If I could automatically drop entire messages that have both octet counters at zero, it would save a significant (probably 40%) amount of disk space in Elasticsearch and reduce query and index load.

[danielnelson]

@taishan69 Can you attach a few lines of the data preferably in influx format?

[jasonkeller]

Easiest way to do that is the file output right?

…

On Wed, Mar 22, 2017 at 3:26 PM Daniel Nelson ***@***.***> wrote: @taishan69 <https://github.com/taishan69> Can you attach a few lines of the data preferably in influx format? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2563 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AUTuiNFSqQYgNwhMPVGWNELwsTmYehdWks5roYP_gaJpZM4MlMqu> .

[danielnelson]

Yeah

[jasonkeller]

{"fields":{**"bytes_recv":0**,**"bytes_sent":0**,"ifname":"VLAN` 00323 (v323-BLAH-BLAH)"},"name":"interface_counters","tags":{"agent_host":"10.55.55.55","host":"holytaco.eat.org"},"timestamp":1490223734}

[danielnelson]

We would accept a pull request for this feature. One of the requirements would be that it performs well, especially if not used, and can handle the different field types.

[jasonkeller]

@danielnelson I'd be unable to assist with that portion. Although I'd love to help, I'm not a programmer and I can't even find where to sync an upstream in Github right now (very burned out and ^%$& off at Github right now).

[danielnelson]

@taishan69 In order to find a workaround until this feature is implemented, it probably makes sense to open an issue over on the InfluxData Community site. You should attach the snmp plugin portion of your telegraf config and use something like snmpwalk to list snmp tables you are using. Perhaps someone can find a way to promote a snmp field to a tag and then filter using it.

[jasonkeller]

@danielnelson I've kind of worked around it by wiring a GELF output in telegraf to a GELF input in logstash, and using the following filter in logstash...but it's not as clean as I'd like it to be (filter in telegraf and output right to Elasticsearch).

input {
  gelf {
    port => 12201
    remap => true
    strip_leading_underscore => true
    type => "telegraf"
  }
}

filter {
  if [bytes_recv] == 0 and [bytes_sent] == 0 {
    drop { }
  }
}

Prior to optimizing the ES template and implementing this, I was hovering around 3.4GB per day. With template optimizations and this tweak, I'm set to clear under 450MB for the day (and I added more data to poll in between!).

I'll surf over to the community page and post this there to see if I can get some attention drawn to it, as it is likely going to be a common issue for highly-automated deployments like we're building here.

[jasonkeller]

@danielnelson https://community.influxdata.com/t/filter-by-metric-value/338

[nhaugo]

The demand for this seems low versus the complication it might add. Closing for now.