SNMP Table Indices not added as Tags #1948

einhirn · 2016-10-25T13:26:06Z

Bug report

When using the "auto table" conf syntax, table indices are not added as tags, even though the source code implies they should.

Relevant telegraf.conf:

[[inputs.snmp]]
  agents = [ "<PDU Network management Card>" ]
  version = 2
  [[inputs.snmp.table]]
    oid = "PDU-MIB::lineVoltageTable"

System info:

Telegraf - version 1.0.1
Ubuntu 14.04.5 LTS
NET-SNMP version: 5.7.2

This probably occurs on any MIB table structure, but here's the MIB-File for this example: Raritan PDU-MIB

Steps to reproduce:

create Telegraf config for a table object in a MIB like above
run telegraf configtest.
See that the output doesn't contain the indices

Expected behavior:

The output should show tags for the table.

> lineVoltageTable,agent_host=PDU-Agent,host=telegrafhost,lineVoltageIndex=2 lineVoltage=230000i,lineVoltageLabel="L2-N" 1477400104000000000
> lineVoltageTable,agent_host=PDU-Agent,host=telegrafhost,lineVoltageIndex=3 lineVoltage=228000i,lineVoltageLabel="L3-N" 1477400104000000000
> lineVoltageTable,agent_host=PDU-Agent,host=telegrafhost,lineVoltageIndex=4 lineVoltage=0i,lineVoltageLabel="" 1477400104000000000
> lineVoltageTable,agent_host=PDU-Agent,host=telegrafhost,lineVoltageIndex=1 lineVoltage=230000i,lineVoltageLabel="L1-N" 1477400104000000000

Actual behavior:

The output doesn't show any tags for the table.

> lineVoltageTable,agent_host=PDU-Agent,host=telegrafhost lineVoltage=228000i,lineVoltageLabel="L3-N" 1477399745000000000
> lineVoltageTable,agent_host=PDU-Agent,host=telegrafhost lineVoltage=0i,lineVoltageLabel="" 1477399745000000000
> lineVoltageTable,agent_host=PDU-Agent,host=telegrafhost lineVoltage=230000i,lineVoltageLabel="L1-N" 1477399745000000000
> lineVoltageTable,agent_host=PDU-Agent,host=telegrafhost lineVoltage=230000i,lineVoltageLabel="L2-N" 1477399745000000000

Additional info:

Problem probably results from using "SNMPtranslate" to parse MIB. Commands extracted from source code are: snmp.go#L221

# snmptranslate -Td PDU-MIB::lineVoltageTable.1
PDU-MIB::lineVoltageEntry
lineVoltageEntry OBJECT-TYPE
  -- FROM       PDU-MIB
  MAX-ACCESS    not-accessible
  STATUS        deprecated
  INDEX         { lineVoltageIndex }
  DESCRIPTION   "An entry containing  values and threshold
                        settings for a particular line input.
                        This object is deprecated in favour of inletPoleTable"
::= { iso(1) org(3) dod(6) internet(1) private(4) enterprises(1) raritan(13742) pdu(4) board(1) lineVoltages(6) lineVoltageTable(2) 1 }

The INDEX field probably gets parsed fine to tagOIDs, but then snmptable is used to fetch the table header:

# snmptable -Ch -Cl -c public 127.0.0.1  PDU-MIB:lineVoltageTable
lineVoltageLabel lineVoltage

As you can see, the index attribute is not listed. This results in
a) lineVoltageIndex not being queried
b) lineVoltageIndex not being marked as tag.

My current workaround is to add a seperate "table.field" for the index like this:

  [[inputs.snmp.table]]
    oid = "PDU-MIB::lineVoltageTable"
  [[inputs.snmp.table.field]]
    oid = "PDU-MIB::lineVoltageIndex"
    is_tag=true

The text was updated successfully, but these errors were encountered:

sparrc · 2016-10-25T13:34:10Z

cc @phemmer

phemmer · 2016-10-25T14:14:23Z

This is happening because the MIB has that OID listed as not-accessible. Meaning we shouldn't try to retrieve it.
It might be possible to ignore that, and fetch it anyway, but I'm not sure if we should. I think your workaround might be the proper solution.

einhirn · 2016-10-25T14:51:29Z

Ah, I see, somehow I was led to believe that snmptables just ignored the index column in my environment.
I did some digging and found a thread in the net-snmp mailing list archive. It quotes "SMI-v2" (some SNMP/MIB standard obviously)

As per SMI-v2 a MAX-ACCESS for index objects is

Not-accessible:  If there are other read-only columnar object in the table. [...]
Read-only:  There is only this column in the table.

Someone seeming to have authority said following:

As long as there is at least one non-index column in the table,
then the indexes themselves should be "not-accessible".
End of Story.

There are many examples where this is not true - typically down to
historical reasons, or poor MIB design.   And it does also make things
easier for lazy client application programmers.  But there is *no* need
to define index objects as accessible (whether for notifications, read
or write access).
   The *only* time you need to do so is when there are no non-index
columns in the table (e.g. the udpTable).    In all other circumstances,
you can work with one of the non-index columns, and you get the
index values for free (as part of the OID).

That means that the approach the code (or my workaround - which shouldn't work, because the attribute is still 'not-accessible') takes is indeed "illegal" from a standardization point of view. So instead of relying on a Index Attribute the index should probably extracted from the OID?

phemmer · 2016-10-25T15:00:55Z

We do extract the index from the OID. It's just used internally within the plugin for identifying all the columns in each row. If you're proposing we should automatically add this index to the output tags, I would be opposed to that.
Basically the auto table mode is meant to work just like the output of the snmptable command (and if you notice, the lineVoltageIndex OID is missing there too). I think deviations from this should require explicit configuration.

einhirn · 2016-10-25T15:35:16Z

With 'snmptable' I can add the '-Ci' switch to show the index. In Telegraf I can't access this information, even if I wanted to. The workaround indeed only works if the snmp agent doesn't adhere to the definitions in the MIB and actually allows access to an attribute that has "MAX-ACCESS not-accessible"... With an other SNMP Agent I do not get any data when querying the index value - and telegraf aborts the query.

So while I can understand your point, I don't have a column to identify a row in every table I'd ever want to query. Think of a switch listing all the fancy error and packet counters for every port it has but no Identifier as to which port this data belongs to... Maybe all Switch MIBs are designed carefully enough to never be in that situation, but there are other SNMP agents out there...

So I'd like to add something like "-Ci" to the query to have the Index added to the table.

einhirn · 2016-10-25T15:46:56Z

Maybe all Switch MIBs are designed carefully enough

Err. No. IF-MIB just ignores the SMI-v2-Rule stated above and defines "ifIndex" as Read-Only.

phemmer · 2016-10-25T16:37:10Z

I'll look into a forceIndices option, or something of the sort. It might be doable without any negative side effects.

You said you had one agent which doesn't respond to queries for the not-accessible OID. What do you get when you try to use snmpget on the object? I'm guessing something like a No Such Object error?

einhirn · 2016-10-26T06:31:29Z

Thanks, that'd be great.
Yes, exactly, I get a No such object error:

# snmptranslate -On ENLOGIC-PDU-MIB::pduUnitStatusIndex.1
.1.3.6.1.4.1.38446.1.2.4.1.1.1

#snmpget -v2c -c pr-RZ-TUC 139.174.33.11 ENLOGIC-PDU-MIB::pduUnitStatusIndex.1
Error in packet
Reason: (noSuchName) There is no such variable name in this MIB.
Failed object: ENLOGIC-PDU-MIB::pduUnitStatusIndex.1


# snmptranslate -On ENLOGIC-PDU-MIB::pduUnitStatusName.1
.1.3.6.1.4.1.38446.1.2.4.1.2.1

# snmpget -v2c -c pr-RZ-TUC 139.174.33.11 ENLOGIC-PDU-MIB::pduUnitStatusName.1
ENLOGIC-PDU-MIB::pduUnitStatusName.1 = STRING: L1/1-2

flokli · 2016-12-07T12:31:36Z

@phemmer: I might be running into the same issue here.
I'd like to query UBNT-MIB::ubntStaTable, which lists associated wifi stations and produces something like the following:

snmpwalk -O teXf -v1 -c $community $agent_ip UBNT-MIB::ubntStaTable
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaMac[1][STRING: 44:d9:e7:48:5:da] = STRING: 44:d9:e7:48:5:da
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaName[1][STRING: 44:d9:e7:48:5:da] = STRING: sb-roof
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaSignal[1][STRING: 44:d9:e7:48:5:da] = INTEGER: -62
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaNoiseFloor[1][STRING: 44:d9:e7:48:5:da] = INTEGER: -87
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaDistance[1][STRING: 44:d9:e7:48:5:da] = INTEGER: 1126
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaCcq[1][STRING: 44:d9:e7:48:5:da] = INTEGER: 100
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaAmp[1][STRING: 44:d9:e7:48:5:da] = INTEGER: 0
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaAmq[1][STRING: 44:d9:e7:48:5:da] = INTEGER: 0
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaAmc[1][STRING: 44:d9:e7:48:5:da] = INTEGER: 0
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaLastIp[1][STRING: 44:d9:e7:48:5:da] = IpAddress: 192.168.1.21
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaTxRate[1][STRING: 44:d9:e7:48:5:da] = INTEGER: 114706
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaRxRate[1][STRING: 44:d9:e7:48:5:da] = INTEGER: 114706
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaTxBytes[1][STRING: 44:d9:e7:48:5:da] = Counter64: 18362972716
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaRxBytes[1][STRING: 44:d9:e7:48:5:da] = Counter64: 2714331356
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaConnTime[1][STRING: 44:d9:e7:48:5:da] = 1698800
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaLocalCINR[1][STRING: 44:d9:e7:48:5:da] = INTEGER: 24
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaTxCapacity[1][STRING: 44:d9:e7:48:5:da] = INTEGER: 94770
.iso.org.dod.internet.private.enterprises.ubnt.ubntMIB.ubntAirMAX.ubntStaTable.ubntStaEntry.ubntStaRxCapacity[1][STRING: 44:d9:e7:48:5:da] = INTEGER: 90090

As you can see, the mac address is used as an index here, and also given in the ubntStaMac field.
I wanted to use it as a tag, however it is only added as a field.

My config looks like this:

 [[inputs.snmp.table]]
    name = "stations"
    inherit_tags = ["hostname"]
    oid = "UBNT-MIB::ubntStaTable"
    [[inputs.snmp.table.field]]
      name = "mac"
      oid = "UBNT-MIB::ubntStaMac"
      is_tag = true

Is this the same issue, or something else?

phemmer · 2016-12-07T14:56:05Z

@flokli can you provide the output of snmpwalk -On? I've tried creating some sample data which mimmicks yours, and am unable to reproduce the issue. But converting that output format into raw numerical OIDs is a PITA.

However at a glance, this does not seem like the same issue.

flokli · 2016-12-07T15:23:30Z

@phemmer Sure, the output looks like this:

snmpwalk -On -v1 -c $community $agent_ip UBNT-MIB::ubntStaTable
.1.3.6.1.4.1.41112.1.4.7.1.1.1.68.217.231.72.5.218 = STRING: 44:d9:e7:48:5:da
.1.3.6.1.4.1.41112.1.4.7.1.2.1.68.217.231.72.5.218 = STRING: sb-roof
.1.3.6.1.4.1.41112.1.4.7.1.3.1.68.217.231.72.5.218 = INTEGER: -63
.1.3.6.1.4.1.41112.1.4.7.1.4.1.68.217.231.72.5.218 = INTEGER: -87
.1.3.6.1.4.1.41112.1.4.7.1.5.1.68.217.231.72.5.218 = INTEGER: 1126
.1.3.6.1.4.1.41112.1.4.7.1.6.1.68.217.231.72.5.218 = INTEGER: 100
.1.3.6.1.4.1.41112.1.4.7.1.7.1.68.217.231.72.5.218 = INTEGER: 0
.1.3.6.1.4.1.41112.1.4.7.1.8.1.68.217.231.72.5.218 = INTEGER: 0
.1.3.6.1.4.1.41112.1.4.7.1.9.1.68.217.231.72.5.218 = INTEGER: 0
.1.3.6.1.4.1.41112.1.4.7.1.10.1.68.217.231.72.5.218 = IpAddress: 192.168.1.21
.1.3.6.1.4.1.41112.1.4.7.1.11.1.68.217.231.72.5.218 = INTEGER: 114706
.1.3.6.1.4.1.41112.1.4.7.1.12.1.68.217.231.72.5.218 = INTEGER: 114706
.1.3.6.1.4.1.41112.1.4.7.1.13.1.68.217.231.72.5.218 = Counter64: 32857331908
.1.3.6.1.4.1.41112.1.4.7.1.14.1.68.217.231.72.5.218 = Counter64: 4360514026
.1.3.6.1.4.1.41112.1.4.7.1.15.1.68.217.231.72.5.218 = Timeticks: (2734800) 7:35:48.00
.1.3.6.1.4.1.41112.1.4.7.1.16.1.68.217.231.72.5.218 = INTEGER: 24
.1.3.6.1.4.1.41112.1.4.7.1.17.1.68.217.231.72.5.218 = INTEGER: 95940
.1.3.6.1.4.1.41112.1.4.7.1.18.1.68.217.231.72.5.218 = INTEGER: 88920

The MIB can be found here.

If it helps, I can also provide the output on a device with 2 simultaniously connected stations, just in case :-)

phemmer · 2016-12-07T16:03:49Z

Yeah, using that data I am unable to reproduce. Are you using the current telegraf version?

# snmpwalk -On -v1 -c public 127.0.0.1:1161 UBNT-MIB::ubntStaTable
.1.3.6.1.4.1.41112.1.4.7.1.1.1.68.217.231.72.5.218 = STRING: 44:d9:e7:48:5:da
.1.3.6.1.4.1.41112.1.4.7.1.2.1.68.217.231.72.5.218 = STRING: sb-roof
.1.3.6.1.4.1.41112.1.4.7.1.3.1.68.217.231.72.5.218 = INTEGER: -62
.1.3.6.1.4.1.41112.1.4.7.1.4.1.68.217.231.72.5.218 = INTEGER: -87
.1.3.6.1.4.1.41112.1.4.7.1.5.1.68.217.231.72.5.218 = INTEGER: 1126
.1.3.6.1.4.1.41112.1.4.7.1.6.1.68.217.231.72.5.218 = INTEGER: 100
.1.3.6.1.4.1.41112.1.4.7.1.7.1.68.217.231.72.5.218 = INTEGER: 0
.1.3.6.1.4.1.41112.1.4.7.1.8.1.68.217.231.72.5.218 = INTEGER: 0
.1.3.6.1.4.1.41112.1.4.7.1.9.1.68.217.231.72.5.218 = INTEGER: 0
.1.3.6.1.4.1.41112.1.4.7.1.11.1.68.217.231.72.5.218 = INTEGER: 114706
.1.3.6.1.4.1.41112.1.4.7.1.12.1.68.217.231.72.5.218 = INTEGER: 114706
.1.3.6.1.4.1.41112.1.4.7.1.15.1.68.217.231.72.5.218 = Timeticks: (1698800) 4:43:08.00
.1.3.6.1.4.1.41112.1.4.7.1.16.1.68.217.231.72.5.218 = INTEGER: 24
.1.3.6.1.4.1.41112.1.4.7.1.17.1.68.217.231.72.5.218 = INTEGER: 94770
.1.3.6.1.4.1.41112.1.4.7.1.18.1.68.217.231.72.5.218 = INTEGER: 90090
## I omitted records 10, 13, & 14 because my test server is snmpd, which can't configure them.

# telegraf --config teletest.conf --test
> stations,mac=44:d9:e7:48:05:da,agent_host=127.0.0.1,host=X ubntStaSignal=-62i,ubntStaNoiseFloor=-87i,ubntStaAmq=0i,ubntStaConnTime=1698800i,ubntStaName="sb-roof",ubntStaDistance=1126i,ubntStaCcq=100i,ubntStaAmp=0i,ubntStaAmc=0i,ubntStaTxRate=114706i,ubntStaRxRate=114706i 1481126294000000000

# cat teletest.conf
[[outputs.influxdb]]
 urls = ["http://localhost"]

[[inputs.snmp]]
 agents = ["127.0.0.1:1161"]
 version = 2
 [[inputs.snmp.table]]
    name = "stations"
    inherit_tags = ["hostname"]
    oid = "UBNT-MIB::ubntStaTable"
    [[inputs.snmp.table.field]]
      name = "mac"
      oid = "UBNT-MIB::ubntStaMac"
      is_tag = true

flokli · 2016-12-07T16:30:47Z

Argh, I had a UBNT-MIB::ubntStaMac.1 in the oid field. Using it without the .1 works perfectly. Sorry!

phemmer · 2017-02-05T06:40:26Z

@einhirn PR #2366 should address your request if you want to try it out.

WaterByWind · 2017-02-19T22:36:10Z

@phemmer - I tried PR #2366 with a more common IP-MIB::icmpStatsTable and IP-MIB::ipIfStatsTable and it seems to do the trick in both cases (thanks!). Many routers support these for more-granular metrics but present the same issue as noted by @einhirn

In both cases the primary index identifies the IP version as defined in the module compliance section:

            -- OBJECT        ipIfStatsIPVersion
            -- SYNTAX        InetVersion {ipv4(1), ipv6(2)}
            -- DESCRIPTION
            --     This MIB requires support for only IPv4 and IPv6
            --     versions.
            --
            -- OBJECT        icmpStatsIPVersion
            -- SYNTAX        InetVersion {ipv4(1), ipv6(2)}
            -- DESCRIPTION
            --     This MIB requires support for only IPv4 and IPv6
            --     versions.

The effort (if even possible) may not be justified, but would it be possible to use these names for the index value? IE: instead of using a tag index=1 use a tag index=ipv4

Using an snmptable -Ci query for the simpler IP-MIB::icmpStatsTable shows:

SNMP table: IP-MIB::icmpStatsTable

 index icmpStatsInMsgs icmpStatsInErrors icmpStatsOutMsgs icmpStatsOutErrors
  ipv4           57861                 9                0             534675
  ipv6               0                 0               25                  0

WaterByWind · 2017-02-19T23:07:54Z

While this PR does allow the metrics to be collected using an auto-table, it does present a quirk for multi-index items.

The above-noted IP-MIB::ipIfStatsTable is one example, and IP-MIB::icmpMsgStatsTable is another. In both cases a single tag is created rather than separate tags for each level of index. While this is consistent with snmptable -Ci it might be more useful here if the two indices were individually used to in turn create individual tags.

For instance, instead of using index=1.0 for the first item below, use something like index1=1,index2=0 (or index1=ipv4,index2=0)


SNMP table: IP-MIB::icmpMsgStatsTable

   index icmpMsgStatsInPkts icmpMsgStatsOutPkts
  ipv4.0                  8               57177
  ipv4.3                676               35200
  ipv4.8              57186                  12
 ipv4.11                  ?              442310
 ipv4.69                  ?                  63
ipv6.133                  ?                   9
ipv6.135                  ?                   4
ipv6.143                  ?                  12

sparrc changed the title ~~Table Indices not added as Tags~~ SNMP Table Indices not added as Tags Oct 25, 2016

sparrc added this to the Future Milestone milestone Oct 25, 2016

sparrc added the bug unexpected problem or unintended behavior label Oct 25, 2016

phemmer mentioned this issue Nov 14, 2016

Telegraf writing metrics twice, but one is wrong? #2041

Closed

phemmer mentioned this issue Jan 28, 2017

[SNMP Plugin] Use Part of OID as Field Name #2291

Closed

This was referenced Feb 5, 2017

Remove metric.Point from metric interface #2365

Closed

snmp: support table indexes as tags #2366

Merged

danielnelson closed this as completed in #2366 Mar 24, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SNMP Table Indices not added as Tags #1948

SNMP Table Indices not added as Tags #1948

einhirn commented Oct 25, 2016

sparrc commented Oct 25, 2016

phemmer commented Oct 25, 2016

einhirn commented Oct 25, 2016 •

edited

Loading

phemmer commented Oct 25, 2016

einhirn commented Oct 25, 2016 •

edited

Loading

einhirn commented Oct 25, 2016

phemmer commented Oct 25, 2016

einhirn commented Oct 26, 2016

flokli commented Dec 7, 2016

phemmer commented Dec 7, 2016

flokli commented Dec 7, 2016

phemmer commented Dec 7, 2016 •

edited

Loading

flokli commented Dec 7, 2016

phemmer commented Feb 5, 2017

WaterByWind commented Feb 19, 2017

WaterByWind commented Feb 19, 2017

SNMP Table Indices not added as Tags #1948

SNMP Table Indices not added as Tags #1948

Comments

einhirn commented Oct 25, 2016

Bug report

Relevant telegraf.conf:

System info:

Steps to reproduce:

Expected behavior:

Actual behavior:

Additional info:

sparrc commented Oct 25, 2016

phemmer commented Oct 25, 2016

einhirn commented Oct 25, 2016 • edited Loading

phemmer commented Oct 25, 2016

einhirn commented Oct 25, 2016 • edited Loading

einhirn commented Oct 25, 2016

phemmer commented Oct 25, 2016

einhirn commented Oct 26, 2016

flokli commented Dec 7, 2016

phemmer commented Dec 7, 2016

flokli commented Dec 7, 2016

phemmer commented Dec 7, 2016 • edited Loading

flokli commented Dec 7, 2016

phemmer commented Feb 5, 2017

WaterByWind commented Feb 19, 2017

WaterByWind commented Feb 19, 2017

einhirn commented Oct 25, 2016 •

edited

Loading

einhirn commented Oct 25, 2016 •

edited

Loading

phemmer commented Dec 7, 2016 •

edited

Loading