OID filtering - compare to string...

[elviosebastianelli]

It is possible to filter by OID comparison with a string?
Example: filter traps that have a particular node name in a varbind.

[etingof]

Do you need to filter by the OID or by its value or by its MIB name or by some combination of those?

[elviosebastianelli]

I need to filter first the OID (to select a trap for node down for example) and then compare other OID with an IP address, to forward only node down from this source address.

[etingof]

I think this exact work flow is yet to be implemented.

What you could do now is to filter/route messages by the OIDs or match the OIDs and override their values in command response (note to self: we need to extend that to notifications).

Perhaps the best bet would be to create a simple plugin which would do exactly what you need i.e. search by a specific OID, then search for the other and compare with a value. In case of no match, just return status.DROP to drop the message. Otherwise route it to the intended destination. May be this plugin could serve as an inspiration. ;-)

Let me know if you need any further help.

[elviosebastianelli]

Hi Ilya,
I'm trying to code a new plugin, as you suggested, but I have some mistakes in OID's check loop (all traps are rejected).
Can you help me looking (if you want) my code (attached)?

Plugin: filterplus.py

#
# This file is an extension of snmpfwd software.
#
# Copyright (c) 2018-2019, Elvio Sebastianelli <elvio.sebastianelli@gmail.com>
#
# SNMP Proxy Forwarder plugin module
#
import re
import sys
import shlex
from snmpfwd.plugins import status
from snmpfwd.error import SnmpfwdError
from snmpfwd.log import debug, info, error
from pysnmp.proto.api import v2c

hostProgs = 'snmpfwd-server', 'snmpfwd-client'

apiVersions = 0, 2

PLUGIN_NAME = 'filterplus'

# This map represents an "empty" value per SNMP type
nullifyMap = {
    v2c.ObjectIdentifier.tagSet: '0.0',
    v2c.Integer.tagSet: 0,
    v2c.Integer32.tagSet: 0,
    v2c.OctetString.tagSet: '',
    v2c.IpAddress.tagSet: '0.0.0.0',
    v2c.Counter32.tagSet: 0,
    v2c.Gauge32.tagSet: 0,
    v2c.Unsigned32.tagSet: 0,
    v2c.TimeTicks.tagSet: 0,
    v2c.Opaque.tagSet: '',
    v2c.Counter64.tagSet: 0
}

rewriteList = []

for moduleOption in moduleOptions:

    optionName, optionValue = moduleOption.split('=', 1)

    if optionName == 'config':

        try:
            configFile = optionValue

            for lineNo, line in enumerate(open(configFile).readlines()):
                line = line.strip()

                if not line or line.startswith('#'):
                    continue

                try:
                    firstOID, secondOID, hostname = shlex.split(line)

                except ValueError:
                    raise SnmpfwdError('%s: syntax error at %s:%s: %s' % (PLUGIN_NAME, configFile, lineNo + 1, sys.exc_info()[1]))

                debug('%s: for OIDs like "%s" and OID "%s" like "%s" sent to destination' % (PLUGIN_NAME, firstOID, secondOID, hostname))

                rewriteList.append((re.compile(firstOID), re.compile(secondOID), hostname))

        except Exception:
            raise SnmpfwdError('%s: config file load failure: %s' % (PLUGIN_NAME, sys.exc_info()[1]))

info('%s: plugin initialization complete' % PLUGIN_NAME)

def dump(obj):
  for attr in dir(obj):
    print("obj.%s = %r" % (attr, getattr(obj, attr)))

def processCommandResponse(pluginId, snmpEngine, pdu, trunkMsg, reqCtx):
    varBinds = []

    for oid, val in v2c.apiPDU.getVarBinds(pdu):
        check = 0
        for firstOID, secondOID, hostname in rewriteList:

            if firstOID.match(str(oid)):
                if str(secondOID) == hostname:
                    check = 1
            else:
                check = 0

    if check == 0:
        return status.DROP, pdu
    else:
        return status.NEXT, pdu


processNotificationRequest = processCommandResponse

PLUGIN CONF FILE: filterplus.conf

#
# SNMP Proxy Forwarder: filterplus plugin configuration
#

# filter by firstOID and compare secondOID with the string in third field (hostname/IP)

# firstOID       secondOID     hostname/IP
"^1\.3\.6\.1\.4\.1\.11\.2\.17\.19\.2\.0\.32$"	"^1\.3\.6\.1\.4\.1\.11\.2\.17\.19\.2\.2\.25$"	"192.168.1.111"

Thanks in advance

[etingof]

If I understand the logic you are looking for, the loop seems suspicious to me... How about this simplistic linear approach?

var_binds = v2c.apiPDU.getVarBinds(pdu)
if len(var_binds) < 2:
    return status.DROP
oid, val = var_binds[0]
if not firstOID.match(str(oid)):
    return status.DROP
oid, val = var_binds[1]
if not secondOID.match(str(oid)):
    return status.DROP
if hostname == str(val):
    return status.NEXT
return status.DROP

[elviosebastianelli]

Hi,
as you suggested, I've changed the checks in processCommandResponse method:

#
#
# SNMP Proxy Forwarder plugin module  (filterplus.py)
#
import re
import sys
import shlex
from snmpfwd.plugins import status
from snmpfwd.error import SnmpfwdError
from snmpfwd.log import debug, info, error
from pysnmp.proto.api import v2c

hostProgs = 'snmpfwd-server', 'snmpfwd-client'

apiVersions = 0, 2

PLUGIN_NAME = 'filterplus'

# This map represents an "empty" value per SNMP type
nullifyMap = {
    v2c.ObjectIdentifier.tagSet: '0.0',
    v2c.Integer.tagSet: 0,
    v2c.Integer32.tagSet: 0,
    v2c.OctetString.tagSet: '',
    v2c.IpAddress.tagSet: '0.0.0.0',
    v2c.Counter32.tagSet: 0,
    v2c.Gauge32.tagSet: 0,
    v2c.Unsigned32.tagSet: 0,
    v2c.TimeTicks.tagSet: 0,
    v2c.Opaque.tagSet: '',
    v2c.Counter64.tagSet: 0
}

rewriteList = []

for moduleOption in moduleOptions:

    optionName, optionValue = moduleOption.split('=', 1)

    if optionName == 'config':

        try:
            configFile = optionValue

            for lineNo, line in enumerate(open(configFile).readlines()):
                line = line.strip()

                if not line or line.startswith('#'):
                    continue

                try:
                    firstOID, secondOID, hostname = shlex.split(line)

                except ValueError:
                    raise SnmpfwdError('%s: syntax error at %s:%s: %s' % (PLUGIN_NAME, configFile, lineNo + 1, sys.exc_info()[1]))

                debug('%s: for OIDs like "%s" and OID "%s" like "%s" sent to destination' % (PLUGIN_NAME, firstOID, secondOID, hostname))

                rewriteList.append((re.compile(firstOID), re.compile(secondOID), hostname))

        except Exception:
            raise SnmpfwdError('%s: config file load failure: %s' % (PLUGIN_NAME, sys.exc_info()[1]))

info('%s: plugin initialization complete' % PLUGIN_NAME)

def dump(obj):
  for attr in dir(obj):
    print("obj.%s = %r" % (attr, getattr(obj, attr)))

def processCommandResponse(pluginId, snmpEngine, pdu, trunkMsg, reqCtx):
    var_binds = v2c.apiPDU.getVarBinds(pdu)
    if len(var_binds) < 2:
        return status.DROP
    oid, val = var_binds[0]
    if not firstOID.match(str(oid)):
        return status.DROP
    oid, val = var_binds[1]
    if not secondOID.match(str(oid)):
        return status.DROP
    if hostname == str(val):
        return status.NEXT
    return status.DROP


processNotificationRequest = processCommandResponse

but there is an error about str and attirbute match:

2019-01-16T11:45:56.00 snmpfwd-server: INFO TrunkingClient at 127.0.0.1:0, peer 127.0.0.1:30301: client is now connected 
2019-01-16T11:46:51.69 snmpfwd-server: ERROR poll error: Traceback (most recent call last):
;  File "/StorageData/netcool/test_snmpfwd/test/pysnmp/carrier/asyncore/dispatch.py", line 46, in runDispatcher
    use_poll=True, map=self.__sockMap, count=1)
;  File "/usr/lib/python2.7/asyncore.py", line 220, in loop
    poll_fun(timeout, map)
;  File "/usr/lib/python2.7/asyncore.py", line 201, in poll2
    readwrite(obj, flags)
;  File "/usr/lib/python2.7/asyncore.py", line 123, in readwrite
    obj.handle_error()
;  File "/usr/lib/python2.7/asyncore.py", line 108, in readwrite
    obj.handle_read_event()
;  File "/usr/lib/python2.7/asyncore.py", line 444, in handle_read_event
    self.handle_read()
;  File "/StorageData/netcool/test_snmpfwd/test/pysnmp/carrier/asyncore/dgram/base.py", line 163, in handle_read
    self._cbFun(self, transportAddress, incomingMessage)
;  File "/StorageData/netcool/test_snmpfwd/test/pysnmp/carrier/base.py", line 70, in _cbFun
    self, transportDomain, transportAddress, incomingMessage
;  File "/StorageData/netcool/test_snmpfwd/test/pysnmp/entity/engine.py", line 152, in __receiveMessageCbFun
    self, transportDomain, transportAddress, wholeMsg
;  File "/StorageData/netcool/test_snmpfwd/test/pysnmp/proto/rfc3412.py", line 433, in receiveMessage
    PDU, maxSizeResponseScopedPDU, stateReference)
;  File "./snmpfwd-server.py", line 226, in processPdu
    pluginId, snmpEngine, pdu, trunkReq, reqCtx
;  File "/StorageData/netcool/test_snmpfwd/test/snmpfwd/plugins/manager.py", line 106, in processNotificationRequest
    return plugin(pluginId, snmpEngine, pdu, snmpReqInfo, reqCtx)
;  File "./plugins/filterplus.py", line 78, in processCommandResponse
    if not firstOID.match(str(oid)):
;AttributeError: 'str' object has no attribute 'match'
caused by <type 'exceptions.AttributeError'>: 'str' object has no attribute 'match' 
(END) 

Any help?

Thanks a lot

[etingof]

At this line:

    if not firstOID.match(str(oid)):

you are using the value of the firstOID variable left from the exception of this code:

                firstOID, secondOID, hostname = shlex.split(line)

Which is obviously a str type rather than a compiled regexp object. Perhaps you should take these values from the rewriteList list.

[elviosebastianelli]

Hi Ilya,
I'm sorry, but I'm not expert in python and, trying to use rewriteList:

#
# This file is an extension of snmpfwd software.
#
# SNMP Proxy Forwarder plugin module
#
import re
import sys
import shlex
from snmpfwd.plugins import status
from snmpfwd.error import SnmpfwdError
from snmpfwd.log import debug, info, error
from pysnmp.proto.api import v2c

hostProgs = 'snmpfwd-server', 'snmpfwd-client'

apiVersions = 0, 2

PLUGIN_NAME = 'filterplus'

# This map represents an "empty" value per SNMP type
nullifyMap = {
    v2c.ObjectIdentifier.tagSet: '0.0',
    v2c.Integer.tagSet: 0,
    v2c.Integer32.tagSet: 0,
    v2c.OctetString.tagSet: '',
    v2c.IpAddress.tagSet: '0.0.0.0',
    v2c.Counter32.tagSet: 0,
    v2c.Gauge32.tagSet: 0,
    v2c.Unsigned32.tagSet: 0,
    v2c.TimeTicks.tagSet: 0,
    v2c.Opaque.tagSet: '',
    v2c.Counter64.tagSet: 0
}

rewriteList = []

for moduleOption in moduleOptions:

    optionName, optionValue = moduleOption.split('=', 1)

    if optionName == 'config':

        try:
            configFile = optionValue

            for lineNo, line in enumerate(open(configFile).readlines()):
                line = line.strip()

                if not line or line.startswith('#'):
                    continue

                try:
                    firstOID, secondOID, hostname = shlex.split(line)

                except ValueError:
                    raise SnmpfwdError('%s: syntax error at %s:%s: %s' % (PLUGIN_NAME, configFile, lineNo + 1, sys.exc_info()[1]))

                debug('%s: for OIDs like "%s" and OID "%s" like "%s" sent to destination' % (PLUGIN_NAME, firstOID, secondOID, hostname))

                rewriteList.append((re.compile(firstOID), re.compile(secondOID), hostname))

        except Exception:
            raise SnmpfwdError('%s: config file load failure: %s' % (PLUGIN_NAME, sys.exc_info()[1]))

info('%s: plugin initialization complete' % PLUGIN_NAME)

def dump(obj):
  for attr in dir(obj):
    print("obj.%s = %r" % (attr, getattr(obj, attr)))



def processCommandResponse(pluginId, snmpEngine, pdu, trunkMsg, reqCtx):
    for firstOID, secondOID, hostname in rewriteList:
        var_binds = v2c.apiPDU.getVarBinds(pdu)
        if len(var_binds) < 2:
            return status.DROP
        oid, val = var_binds[0]
        if not firstOID.match(str(oid)):
            return status.DROP
        oid, val = var_binds[1]
        if not secondOID.match(str(oid)):
            return status.DROP
        if hostname == str(val):
            return status.NEXT
        return status.DROP

processNotificationRequest = processCommandResponse

Now I have this error about int object not iterable:

n from (2, '127.0.0.1', 0) to (2, '127.0.0.1', 30301)... 
2019-01-21T11:13:12.83 snmpfwd-server: INFO TrunkingClient at 127.0.0.1:0, peer 127.0.0.1:30301: client is now connected 
2019-01-21T11:13:15.70 snmpfwd-server: ERROR poll error: Traceback (most recent call last):
;  File "/StorageData/netcool/test_snmpfwd/test/pysnmp/carrier/asyncore/dispatch.py", line 46, in runDispatcher
    use_poll=True, map=self.__sockMap, count=1)
;  File "/usr/lib/python2.7/asyncore.py", line 220, in loop
    poll_fun(timeout, map)
;  File "/usr/lib/python2.7/asyncore.py", line 201, in poll2
    readwrite(obj, flags)
;  File "/usr/lib/python2.7/asyncore.py", line 123, in readwrite
    obj.handle_error()
;  File "/usr/lib/python2.7/asyncore.py", line 108, in readwrite
    obj.handle_read_event()
;  File "/usr/lib/python2.7/asyncore.py", line 444, in handle_read_event
    self.handle_read()
;  File "/StorageData/netcool/test_snmpfwd/test/pysnmp/carrier/asyncore/dgram/base.py", line 163, in handle_read
    self._cbFun(self, transportAddress, incomingMessage)
;  File "/StorageData/netcool/test_snmpfwd/test/pysnmp/carrier/base.py", line 70, in _cbFun
    self, transportDomain, transportAddress, incomingMessage
;  File "/StorageData/netcool/test_snmpfwd/test/pysnmp/entity/engine.py", line 152, in __receiveMessageCbFun
    self, transportDomain, transportAddress, wholeMsg
;  File "/StorageData/netcool/test_snmpfwd/test/pysnmp/proto/rfc3412.py", line 433, in receiveMessage
    PDU, maxSizeResponseScopedPDU, stateReference)
;  File "./snmpfwd-server.py", line 226, in processPdu
    pluginId, snmpEngine, pdu, trunkReq, reqCtx
;TypeError: 'int' object is not iterable
caused by <type 'exceptions.TypeError'>: 'int' object is not iterable 

[etingof]

I think the plugin is expected to return a tuple of status, pdu such as this. Your code returns just one scalar integer which can't be iterated.

[elviosebastianelli]

Hi Ilya,
thanks for your help.
Now I have the snmpfwd up & running on the sunos 5.10 (thanks again) with my filterplus.py plugin without errors in start od service. Last version of filterplus.py have this processCommandResponse method:

def processCommandResponse(pluginId, snmpEngine, pdu, trunkMsg, reqCtx):
    for firstOID, secondOID, hostname in rewriteList:
        var_binds = v2c.apiPDU.getVarBinds(pdu)
        # if len(var_binds) < 2:
        #     return status.DROP, pdu
        oid, val = var_binds[0]
        dump(var_binds[0])
        if not firstOID.match(str(oid)):
            return status.DROP, pdu
        oid, val = var_binds[1]
        if not secondOID.match(str(oid)):
            return status.DROP, pdu
        if hostname == str(val):
            return status.NEXT, pdu
        return status.DROP, pdu

processNotificationRequest = processCommandResponse

Actually I'm trying to check the functionality that I want.
In order to filter trap by an OID (node down) and than by some hosts, I need to summarize a little bit.

My target is to receive only "node down" (firstOID in filterplus.conf) traps, but only from some hosts (hostname in filterplus.conf). I'm trying with and without regular expressions in conf, like this:

### #
# SNMP Proxy Forwarder: filterplus plugin configuration
#

# filter by firstOID and compare secondOID with the string in third field (hostname/IP)

# firstOID       secondOID     hostname/IP
"^1\.3\.6\.1\.4\.1\.11\.2\.17\.19\.2\.0\.32$"	"^1\.3\.6\.1\.4\.1\.11\.2\.17\.19\.2\.2\.25$"	"10.51.9.73"
# 1.3.6.1.4.1.11.2.17.19.2.0.32	1.3.6.1.4.1.11.2.17.19.2.2.25	"10.51.9.73" 

and the server configuration is this:

#
# SNMP forwarder: Agent part configuration
#

config-version: 2
program-name: snmpfwd-server

snmp-credentials-group {
  snmp-transport-domain: 1.3.6.1.6.1.1.100
  snmp-bind-address: 10.51.9.73:10161

  snmp-engine-id: 0x0102030405070809

  snmp-community-name: public
  snmp-security-name: public
  snmp-security-model: 2
  snmp-security-level: 1

  snmp-credentials-id: snmp-credentials
}

context-group {
  snmp-context-engine-id-pattern: .*?
  snmp-context-name-pattern: .*?

  snmp-context-id: any-context
}

content-group {
  snmp-pdu-type-pattern: .*?
  snmp-pdu-oid-prefix-pattern-list: .*?

  snmp-content-id: any-content
}

peers-group {
  snmp-transport-domain: 1.3.6.1.6.1.1.100
  snmp-bind-address-pattern-list: .*?
  snmp-peer-address-pattern-list: .*?

  snmp-peer-id: 100
}

plugin-modules-path-list: ./plugins ${plugin-dir}

plugin-group {
  plugin-module: filterplus
  plugin-options: config=${config-dir}/plugins/filterplus.conf

  plugin-id: filterplus-response-values
}

trunking-group {
  trunk-bind-address: 127.0.0.1
  trunk-peer-address: 127.0.0.1:30301
  trunk-ping-period: 60
  trunk-connection-mode: client

  trunk-id: trunk-1
}

routing-map {
  matching-snmp-context-id-list: any-context
  matching-snmp-content-id-list: any-content

  matching-snmp-credentials-id-list: snmp-credentials
  matching-snmp-peer-id-list: 100

  using-plugin-id-list: filterplus-response-values
  using-trunk-id-list: trunk-1
}

Now, if I send a trap to local server with snmp_trapsend, like this:
snmp_trapsend -p 10161 -a "1.3.6.1.4.1.11.2.17.19.2.0.32 STRING (generic text)" -e 1.3.6.1.4.1.42

I have a "ERROR no route configured....", as you can see:
2019-01-28T09:09:48.89 snmpfwd-server: ERROR no route configured callflow-id=7d218cb328 snmp-engine-id=0x0102030405070809 snmp-transport-domain=1.3.6.1.6.1.1.100 snmp-bind-address=10.51.9.73 snmp-bind-port=10161 snmp-security-model=1 snmp-security-level=1 snmp-security-name=public snmp-credentials-id=<nil> snmp-context-engine-id=0x0102030405070809 snmp-context-name=<nil> snmp-context-id=any-context snmp-pdu=SNMPv2TrapPDU#1.3.6.1.2.1.1.3.0:'1304004132',1.3.6.1.6.3.1.1.4.1.0:'1.3.6.1.4.1.42.0.1',1.3.6.1.6.3.18.1.3.0:'10.51.9.73',1.3.6.1.6.3.18.1.4.0:<nil>,1.3.6.1.6.3.1.1.4.3.0:'1.3.6.1.4.1.42',1.3.6.1.4.1.11.2.17.19.2.0.32:'generic text', snmp-content-id=any-content snmp-peer-address=10.51.9.73 snmp-peer-port=58545 snmp-peer-id=100

But if I send to local server a trap with network node manager (hp openview), like this:

node=10.51.9.73
nnmi=10.51.9.73
/opt/OV/bin/nnmsnmpnotify.ovpl -v2c -p 10161 -e .1.3.6.1.4.1.11.2.17.19.2.0.32 -a $node $nnmi .1.3.6.1.4.1.11.2.17.19.2.0.32 ...

I have no message in server log.

Any idea about it?

[etingof]

This snmp-credentials-id=<nil> looks suspicious!

May be your notification comes in a way that it does not match snmp-credentials entry? Perhaps snmp-security-model is different? Can it be that you are actually sending SNMPv1 TRAP?

[elviosebastianelli]

Yes, snmp_trapsend send only v1 traps, but I can see something in logs.
When I try to send v2c traps with the command:

/opt/OV/bin/nnmsnmpnotify.ovpl -v2c -p 10161 -e .1.3.6.1.4.1.11.2.17.19.2.0.32 -a $node $nnmi .1.3.6.1.4.1.11.2.17.19.2.0.32 (where $node and $nnmi are the host receiver)

nothing changes in log

[etingof]

If you intend to send SNMPv1 TRAPs, try to set snmp-security-model: 1 in the configuration so that the entry snmp-credentials would be chosen.