Threading example

[nrolans]

Hello

I am trying to use snimpy in a multi-threaded app and run into two issues (see stack traces). I have tried to isolate the problem in the example below. The approach is to load the MIBs in the main thread then create a Manager in each additional thread. I also tried creating the managers in the main thread but that did not help. Is this the right/supported approach? (Could you provide an example maybe?)

These errors do not happen systematically, I noticed at least one of them ~50% of the time with the example code.

Environment

• python v2.7.8
• snimpy v0.8.6
• pysnmp v4.2.5
• (4 cores machines)

Example

#!/usr/bin/env python

import threading
from snimpy.manager import load
import time
from snimpy.manager import Manager as M
from datetime import datetime
import collections
import pprint


def cpumem(config={}, switch={}):
    m = M(
        host=switch['hostname'],
        community=switch['community'],
        version=2,
        cache=config.get('cache', 8),
        timeout=config.get('timeout', 5),
        retries=config.get('retries', 1),
        none=True)

    oids = [
        ('cpmCPUTotal5secRev', int),
        ('cpmCPUTotal1minRev', int),
        ('cpmCPUTotal5minRev', int),
        ('cpmCPUMemoryUsed', int),
        ('cpmCPUMemoryFree', int),
    ]

    cpus = collections.defaultdict(dict)

    for oidtype in oids:
        oid = oidtype[0]
        data = m.__getattribute__(oid)
        for idx in data:
            cpus[idx][oid] = (oidtype[1](data[idx]), datetime.utcnow())
    pprint.pprint(cpus)


class ThPoller(threading.Thread):

    def __init__(self, switch):
        super(ThPoller, self).__init__()
        self.switch = switch

    def run(self):
        for i in range(2):
            cpumem({}, self.switch)
            time.sleep(0.5)

switches = [
    {
        'switch': {'hostname': 'switch1.example.org', 'community': 'abcdef'},
        'config': {},
    },
    {
        'switch': {'hostname': 'switch2.example.org', 'community': 'abcdef'},
        'config': {},
    },
]

load('./mibs/IF-MIB')
load('./mibs/CISCO-SMI.my')
load('./mibs/CISCO-TC.my')
load('./mibs/CISCO-PROCESS-MIB.my')
load('./mibs/RFC1213-MIB.my')

tp1 = ThPoller(switches[0]['switch'])
tp2 = ThPoller(switches[1]['switch'])
tp1.start()
tp2.start()

tp1.join()
tp2.join()

IndexError: pop from empty list

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/threading.py", line 813, in __bootstrap_inner
    self.run()
  File "./th.py", line 48, in run
    cpumem({}, self.switch)
  File "./th.py", line 35, in cpumem
    for idx in data:
  File "/usr/lib64/python2.7/site-packages/snimpy/manager.py", line 384, in __iter__
    for k, _ in self.iteritems():
  File "/usr/lib64/python2.7/site-packages/snimpy/manager.py", line 395, in iteritems
    for noid, result in self.session.walk(oid):
  File "/usr/lib64/python2.7/site-packages/snimpy/manager.py", line 126, in walk
    return self.getorwalk("walk", *args)
  File "/usr/lib64/python2.7/site-packages/snimpy/manager.py", line 111, in getorwalk
    value = getattr(self._session, op)(*args)
  File "/usr/lib64/python2.7/site-packages/snimpy/snmp.py", line 273, in walk
    return self._op(self._cmdgen.bulkCmd, *args)
  File "/usr/lib64/python2.7/site-packages/snimpy/snmp.py", line 227, in _op
    self._auth, self._transport, *oids)
  File "/usr/lib/python2.7/site-packages/pysnmp/entity/rfc3413/oneliner/cmdgen.py", line 633, in bulkCmd
    self.__asynCmdGen.snmpEngine.transportDispatcher.runDispatcher()
  File "/usr/lib/python2.7/site-packages/pysnmp/carrier/asynsock/dispatch.py", line 41, in runDispatcher
    raise PySnmpError('poll error: %s' % ';'.join(format_exception(*exc_info())))
PySnmpError: poll error: Traceback (most recent call last):
;  File "/usr/lib/python2.7/site-packages/pysnmp/carrier/asynsock/dispatch.py", line 37, in runDispatcher
    use_poll=True, map=self.__sockMap, count=1)
;  File "/usr/lib64/python2.7/asyncore.py", line 220, in loop
    poll_fun(timeout, map)
;  File "/usr/lib64/python2.7/asyncore.py", line 201, in poll2
    readwrite(obj, flags)
;  File "/usr/lib64/python2.7/asyncore.py", line 123, in readwrite
    obj.handle_error()
;  File "/usr/lib64/python2.7/asyncore.py", line 110, in readwrite
    obj.handle_write_event()
;  File "/usr/lib64/python2.7/asyncore.py", line 468, in handle_write_event
    self.handle_write()
;  File "/usr/lib/python2.7/site-packages/pysnmp/carrier/asynsock/dgram/base.py", line 60, in handle_write
    outgoingMessage, transportAddress = self.__outQueue.pop(0)
;IndexError: pop from empty list

CarrierError: Transport (1, 3, 6, 1, 6, 1, 1) already registered

This one typically happens when the program starts, as if a duplicate transport was being registered with pysnmp.

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/threading.py", line 813, in __bootstrap_inner
    self.run()
  File "./th.py", line 48, in run
    cpumem({}, self.switch)
  File "./th.py", line 35, in cpumem
    for idx in data:
  File "/usr/lib64/python2.7/site-packages/snimpy/manager.py", line 384, in __iter__
    for k, _ in self.iteritems():
  File "/usr/lib64/python2.7/site-packages/snimpy/manager.py", line 395, in iteritems
    for noid, result in self.session.walk(oid):
  File "/usr/lib64/python2.7/site-packages/snimpy/manager.py", line 126, in walk
    return self.getorwalk("walk", *args)
  File "/usr/lib64/python2.7/site-packages/snimpy/manager.py", line 111, in getorwalk
    value = getattr(self._session, op)(*args)
  File "/usr/lib64/python2.7/site-packages/snimpy/snmp.py", line 273, in walk
    return self._op(self._cmdgen.bulkCmd, *args)
  File "/usr/lib64/python2.7/site-packages/snimpy/snmp.py", line 227, in _op
    self._auth, self._transport, *oids)
  File "/usr/lib/python2.7/site-packages/pysnmp/entity/rfc3413/oneliner/cmdgen.py", line 630, in bulkCmd
    contextEngineId, contextName
  File "/usr/lib/python2.7/site-packages/pysnmp/entity/rfc3413/oneliner/cmdgen.py", line 380, in bulkCmd
    authData, transportTarget
  File "/usr/lib/python2.7/site-packages/pysnmp/entity/rfc3413/oneliner/cmdgen.py", line 96, in cfgCmdGen
    transport
  File "/usr/lib/python2.7/site-packages/pysnmp/entity/config.py", line 321, in addTransport
    transportDomain, transport
  File "/usr/lib/python2.7/site-packages/pysnmp/carrier/asynsock/dispatch.py", line 20, in registerTransport
    AbstractTransportDispatcher.registerTransport(self, tDomain, t)
  File "/usr/lib/python2.7/site-packages/pysnmp/carrier/base.py", line 97, in registerTransport
    'Transport %s already registered' % (tDomain,)
CarrierError: Transport (1, 3, 6, 1, 6, 1, 1) already registered

Thanks!

[vincentbernat]

Your code is correct. Could you try to apply 4b2db7f? I think it would solve the issue.

[nrolans]

That looks much better and the diff makes sense. Thanks!

[vincentbernat]

Ok, closing then.