Allow SNMP Value OIDs to be parsed using regular expressions

[gadzet21]

What introduces the change:

• we add a new definition in xml files: <value_parse_index>

What do we gain:

• we optimize the list of returned data by the method: "Get SNMP Data"
• we can better match the list in data query with what we want to see
• fewer records in the host_snmp_cache table
• we will increase the speed of returned data:
  • get method: for slow devices or a slow connection, a large list of returned data, only a few of which we care about
  • walk method: for fast devices and links

Example:

<interface>
<name>Get SNMP Interfaces - CMTS Upstream</name>
<description>Queries a host for a list of interfaces for CMTS, only Upstreams</description>
<oid_index>.1.3.6.1.2.1.2.2.1.3</oid_index>
<value_index_parse>VALUE/REGEXP:129|205</value_index_parse>
<oid_index_parse>OID/REGEXP:.*\.([0-9]+)$</oid_index_parse>
<index_order>ifIndex</index_order>
<index_order_type>numeric</index_order_type>
<index_title_format>|chosen_order_field|</index_title_format>

<fields>
<ifIndex>
	<name>Index</name>
	<source>index</source>
	<direction>input</direction>
</ifIndex>
<ifType>
<name>Type</name>
	<method>walk</method>
	<source>value</source>
	<direction>input</direction>
	<oid>.1.3.6.1.2.1.2.2.1.3</oid>
</ifType>
<ifInOctets>
	<name>Bytes In</name>
	<method>walk</method>
	<source>value</source>
	<direction>output</direction>
	<oid>.1.3.6.1.2.1.2.2.1.10</oid>
</ifInOctets>
</fields>
</interface>

It will only show us a list of upstream interfaces.

You can also like this:

<oid_index>.1.3.6.1.2.1.2.2.1.8</oid_index>
<value_index_parse>VALUE/REGEXP:1</value_index_parse>

It will only show the interfaces in the up status.

The new definition works for the methods: get and walk.

CMTS devices can have many interfaces (e.g. cbr 130,000 interfaces), a user who wants to reduce the number of entries can use this definition.

In my cacti there are over 30 million records in the host_snmp_cache table, this change will reduce that amount.

[gadzet21]

It should be better, though I don't know why 2x changelogs came out

@TheWitness Thank you for the idea of "partition by hash", first I will see how much the above idea will reduce my records and there is still a problem with spine (FATAL: Spine Interrupted by Abort Signal) - not often, but it happens

[TheWitness]

There is some leak in spine that I have not tracked down yet. Like you said, it works most of the time.

[TheWitness]

Much cleaner now. Thanks for your contribution @gadzet21 !