UnicodeDecodeError on value conversion

[mrichar1]

I'm currently using snimpy (in a python2.6 virtualenv) to pull the cdp info from a variety of network devices.

Having set up a manager object:

host = "192.168.1.1"
comm = "public"

mibs = ("/usr/lib/net/MIBs/CISCO-SMI.my",
        "/usr/lib/net/MIBs/CISCO-TC.my",
        "/usr/lib/net/MIBs/CISCO-VTP-MIB.my",
        "/usr/lib/net/MIBs/CISCO-CDP-MIB.my")

for mib in mibs:
    snimpy.manager.load(mib)

session = snimpy.manager.Manager(host,
                                 community=comm,
                                 version=2,
                                 cache=True)

I then want to get cdpCacheDevicePort:

for index, port in session.cdpCacheDevicePort.iteritems():
    print repr((index, port))

Results:

((<Integer: 1>, <Integer: 1>), '26')
((<Integer: 2>, <Integer: 1>), '4')
((<Integer: 3>, <Integer: 1>), '3')
((<Integer: 75>, <Integer: 1>), '\x00`\xb9\xb7\xf6v')
((<Integer: 148>, <Integer: 1>), '\x00`\xb9\xb6\xc4~')

However, if I iterate over another related oid, and use it's index for that iteration to query the value of this field, I get an error when it reaches the 'hex format' values:

for index, chassis in session.cdpCacheDeviceId.iteritems():
    print repr((index, session.cdpCacheDevicePort[index]))

Results:

((<Integer: 1>, <Integer: 1>), <String: 26>)
((<Integer: 2>, <Integer: 1>), <String: 4>)
((<Integer: 3>, <Integer: 1>), <String: 3>)
Traceback (most recent call last):
  File "/tmp/snmpy.py", line 23, in <module>
    print repr((index, session.cdpCacheDevicePort[index]))
  File "/home/mrichar1/git/netmap/networkx/lib/python2.6/site-packages/snimpy/manager.py", line 347, in __getitem__
    return self._op("get", index)
  File "/home/mrichar1/git/netmap/networkx/lib/python2.6/site-packages/snimpy/manager.py", line 343, in _op
    return self.proxy.type(self.proxy, result)
  File "/home/mrichar1/git/netmap/networkx/lib/python2.6/site-packages/snimpy/basictypes.py", line 109, in __new__
    value = String._internal(entity, self)
  File "/home/mrichar1/git/netmap/networkx/lib/python2.6/site-packages/snimpy/basictypes.py", line 533, in _internal
    return cls._fromBytes(value._value, entity.fmt)
  File "/home/mrichar1/git/netmap/networkx/lib/python2.6/site-packages/snimpy/basictypes.py", line 442, in _fromBytes
    result += bb.decode("ascii")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xb9 in position 2: ordinal not in range(128)

It looks like in the iteration the raw value is returned - however when asking for a specific instance, some formatting is done (e.g '3' versus '<String: 3>'). Is this a bug? Or Is there a way to avoid this behaviour (without having to iterate over the oid to get a specific value) and always get the raw value?

Thanks!

[vincentbernat]

The MIB says that cdpCacheDevicePort is a DisplayString. Such a type mandates the use of ASCII charset, hence the error. The difference you get is a bug and you should get the same exception during your first try.

With the not-yet-released latest version of Snimpy, thanks to @Daemonae, it is possible to fix the type with something like this:

m.cdpCacheDevicePort.proxy.typeName = 'somethingelse'

You'll have to find what somethingelse could be. Unfortunately, it is not possible to use OCTET STRING here. SNMPv2-TC defines TAddress which is quite like an OCTET STRING.

The other way around is to fix the MIB as the type is not a DisplayString. Change the SYNTAX to OCTET STRING to fix this "problem". I have just pushed 0.8.4 with those changes so that you can test this workaround.

[mrichar1]

Hi,

Thanks for such a quick answer!

I've tried out the potential fixes - editing the MIB and changing DisplayString to OCTET STRING works as expected.

However, setting:

m.cdpCacheDevicePort.proxy.typeName = 'TAddress'

Still returns the same error. Is this related to the 'base type' still being DisplayString?

One other question - would you consider adding a way to return the 'raw' value with no coercion/casting? I know this is the wrong place to fix the problem, but given that vendors often play 'fast and loose' and don't obey the standards/match the MIBs, it could potentially be useful, even just for debugging these kinds of situations?

Many thanks once again for a quick and detailed response!

[vincentbernat]

Yes, the base type is still the one of display strings (ie types with a format), so the workaround won't work.

As for disabling type coercion, yes, we can try to do that when outputting values. This could be done in a context or in a special session (RawSession for example).

[vincentbernat]

The session stuff is the wrong abstraction. Let me think a bit. This could also be just a parameter for the manager.

[mrichar1]

That would work - I was mostly thinking of cases where you might want to try getting the coerced value, and if that failed, get the raw value - not sure how that could be implemented:

try:
    # Get the coerced/converted value as normal
    x = m.Foo[id]
except UnicodeDecodeError:
    # Something went wrong with conversion - get the raw value
    x = ???

I could imagine doing something like:

```python
m.getRaw(True)
x = m.Foo[id]
m.getRaw(False)

Or maybe a raw method on the manager?

m.raw.Foo[id]

Just some quick thinking around the idea, so take it all with a pinch of salt :-)

[mrichar1]

👍 many thanks!

[vincentbernat]

With the latest commit, you can pass loose=True when creating a manager.