-
Notifications
You must be signed in to change notification settings - Fork 24
SNMPv3 with DES and MD5 fails: invalid message authentication salt #40
Comments
Hi @tamiradari , thank you for the report. Could you fill me in on the following:
|
Hi Tiago,
Please find the details:
* Latest
* ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-darwin18]
* Added to the end
* Yes
NETSNMP::SecurityParameters.verify(stream#String, salt#String) at /Users/tamiradari/dev/Frontend/lib/netsnmp/security_parameters.rb:127
#<Class:NETSNMP::Message>.decode(stream#String, security_parameters#NETSNMP::SecurityParameters) at /Users/tamiradari/dev/Frontend/lib/netsnmp/message.rb:29
NETSNMP::V3Session.decode(stream#String, security_parameters#NETSNMP::SecurityParameters) at /Users/tamiradari/dev/Frontend/lib/netsnmp/v3_session.rb:78
NETSNMP::V3Session.send(pdu#NETSNMP::ScopedPDU) at /Users/tamiradari/dev/Frontend/lib/netsnmp/v3_session.rb:25
block in NETSNMP::Client.block in get(*oid_opts#Array) at /Users/tamiradari/dev/Frontend/lib/netsnmp/client.rb:55
NETSNMP::Client.handle_retries at /Users/tamiradari/dev/Frontend/lib/netsnmp/client.rb:155
NETSNMP::Client.get(*oid_opts#Array) at /Users/tamiradari/dev/Frontend/lib/netsnmp/client.rb:55
Thanks,
Tamir.
|
My first suspicion is that cleaning the auth bytes isn't being done as it's supposed to, for some reason. I'd breakpoint around this code, as there seems to be something fishier around the verification of the response bytestream and the auth param. The procedure followed here is described in this section nof the RFC. I'd also try doing the request with v2c, just to make sure that it's an authentication issue, and not some encoding incompatibility. |
Hi Tiago,
I am not familiar with the RFC details.
I compared the first packet send from the gem and another tool that works for me.
There is similarity but seems like extra data of 12 x ‘00’ got in the middle.
Hope this can give you a way to deeper understanding.
netsnmp gem:
30 4e 02 01 03 30 11 02 04 46 ff de dd 02 03 00
ff e3 04 01 04 02 01 03 04 20 30 1e 04 00 02 01
00 02 01 00 04 04 73 77 72 6f 04 0c 00 00 00 00
00 00 00 00 00 00 00 00 04 00 30 14 04 00 04 00
a0 0e 02 04 14 1f d7 a9 02 01 00 02 01 00 30 00
Sending 64 bytes to UDP (http://www.net-snmp.org/ snmp-get):
30 3E 02 01 03 30 11 02 04 72 C3 42 43 02 03 00
FF E3 04 01 04 02 01 03 04 10 30 0E 04 00 02 01
00 02 01 00 04 00 04 00 04 00 30 14 04 00 04 00
A0 0E 02 04 70 3C C0 1A 02 01 00 02 01 00 30 00
Thanks,
Tamir.
|
The first packet is the probe, and that doesn't seem to be the issue AFAICT, i.e. the problem is when receiving its response and verifying the signature. This is where the exchange happens. If you set up a breakpoint here, you'll be able to see the returned bytestream to the probe PDU, and check whether the "auth_param" can be found and replaced, before being verified. netsnmp supports hexdumping a bytestream directly, provided you install the "hexdump" gem. I wanted years ago to add proper debug output of PDU packets, but sadly never finished that work. |
Hello, I have the same kind of issue with my Cisco ASA and Cisco NX-OS using snmp v3. They are working fine with snmpwalk and snmpget but not with the gem. netsnmp gem: snmpget: I used byebug like you said in the comment above, the variables are: It seems there is an issue with the probe, but I don't know if it's an issue with the gem or with the snmp implementation on those devices. You can test this with a Cisco ASA or NX-OS on gns3 (this is how we test it). Also if you prefer we can expose one of our device. |
Hi @jinxka , Thank you for the investigation. If the issue is the probe PDU, let's try finding out whats going on here. First, are you having the same issue using the des/md5 combo? Now, let's dissect this:
Looking at the diff, I can spot a few things:
I can't really tell how this came to be. However, I'd try to "reverse-engineer" how the probe PDU is being assembled by the netsnmp cli tool (if I remember correctly, there's a debug level at which you can get the initial PDU and the random ids used to generate the salt/scoped_pdu, after which you could hammer these values into I'd also troubleshoot first whether using v2c would solve the trick, if that's possible. |
If my comments don't help you troubleshoot it, we can go that route, if that's ok from a security perspective. I'd need the full |
I tried with MD5/DES but it's not working (NETSNMP::Error (invalid encrypted PDU received)). Here is the full debug of the working snmpget with SHA/AES:
To test the gem, I put byebug at: ruby-netsnmp/lib/netsnmp/message.rb Line 79 in 9d47f0b
and ruby-netsnmp/lib/netsnmp/message.rb Line 27 in 9d47f0b Before the encoded packet is sent, I force it's value to the one sent by snmpget. For the first transaction, I changed the 63 bytes sent by snmpget to Following this I checked the bytes packet received and it's identical to the one received by snmpget minus the last 2 digit (65 in above result). I don't know the impact of the missing part. I used the same process for the second transaction and this time I got more differences (you can compare with the above snmpget result, second packet received) :
After this, there are no third transaction like in snmpget because the error NETSNMP::Error (invalid priv salt received) is raised. I'll try to decode the above and send you an access to one of our Cisco (there are no security issues as there are empty and only used for snmp testing) |
Perfect, thank you for the dumps. Will see if I can do some "replay" there in the next week or so. |
@jinxka I've just pushed some debug log improvements into the |
Hello, Sorry for the late answer, here is the the dumps using branch 'issue-40' with NETSNMP_DEBUG=2
|
@jinxka can you try again. sorry, there was a bug in the log layer... |
Can we use your email address to send you the info so you can connect to one of our cisco device? |
I've fixed this one. But it's alright, you can send (do send me the ruby script ready to execute, plz). |
|
thx for the output @jinxka . I'll try to work with it, and if I can't, I'll welcome that email with access to the node. the first thing I notice is that you're actually having troubles with the decryption, not hmac verification. So this might be something different to what @tamiradari experienced. |
@jinxka I found the issue! If you look at the "received message" section you pasted, you'll notice that the response PDU is already there, which shouldn't happen if the packet were encrypted. Analysing the v3 message content further, you'll see that the 3rd field of the header, which accounts for message flags, comes with "0401", and the least significant byte accounts for security level, which is 1, which is "auth_no_priv". So, your node is sending the response packet authenticated but not encrypted. The bug happens because I'm using the info of the sender (you) only to determine the security level, and this seems to be what's causing the issue. I'll fix it as soon as I find some time, and then I'll ask you to try again. |
(From #40) although the client can set auth_priv level for communicating with an agent, this does not mean that the agent will comply with the same security level. This means that, the processing of the message will have to be chosen by the advertised message flag.
(From #40) although the client can set auth_priv level for communicating with an agent, this does not mean that the agent will comply with the same security level. This means that, the processing of the message will have to be chosen by the advertised message flag.
@jinxka @tamiradari can you test against this branch? Still have some CI issues to fix, but it should be good fo you to give it a try. |
We have some issues in the lab, so I don’t have access to this router.
I will test it when it will be solved.
Thanks,
Tamir.
|
@HoneyryderChuck , I tried with your new changes and got this:
It's looking better, but there seems to be an issue with the empty priv_param. As to why it does not send it, I need to look into it more. We have some trouble to open the access on our cisco, but it will work eventually :). |
(From #40) although the client can set auth_priv level for communicating with an agent, this does not mean that the agent will comply with the same security level. This means that, the processing of the message will have to be chosen by the advertised message flag.
@jinxka can you pull and try again? I just realized there was a bug in the fix for this issue. It should work now. FYI the issue is not the empty |
@HoneyryderChuck I tried again after pulling and unfortunately I got the same error:
|
(From #40) although the client can set auth_priv level for communicating with an agent, this does not mean that the agent will comply with the same security level. This means that, the processing of the message will have to be chosen by the advertised message flag.
@jinxka just committed a potential fix. However, maybe to spare you from constantly hitting against the wall, I'll ask you to troubleshoot it yourself. In order to do so, I'll ask you to open the project on your side and inspect what's happening. The commit/line with the fix is: 340da59#diff-5fc6c95fdba643193b6d2e0e980c33d716231a660b5dea6b2b76d759098b08bdR38 That I think that I committed the proper fix this time, but then again, I also did 3 messages ago. Sorry for the back and forth. |
(From #40) although the client can set auth_priv level for communicating with an agent, this does not mean that the agent will comply with the same security level. This means that, the processing of the message will have to be chosen by the advertised message flag.
Thank you very much @HoneyryderChuck for your time. I confirm you that we are able to give you an access one of your test environment tomorow morning. |
@HoneyryderChuck I send you the SNMP credentials to acces the Cisco. |
@fwininger I'm getting a timeout.
|
@HoneyryderChuck We are working on it and coming back to you ASAP. |
The ICMP is blocked by our internet provider. The snmpget command I have send you should work now :) |
@fwininger @jinxka it's done. Although my fixes from yesterday fixed the exception issue, there was still a last remaining handshake problem that I have solved with my last commit (for details, please read the commit message). I'll be releasing a new version in the coming days. Thank you for the collaboration. |
Thank you very much 🥇 |
These parameters are used:
client = NETSNMP::Client.new({host: 'a.b.c.d', port: 161, version: '3', username: 'user', security_level: 3, auth_protocol: 'md5', auth_password: 'Password1', priv_protocol: 'des', priv_password: 'Password1'})
client.get(oid: "1.3.6.1.2.1.1.5.0")
Error: invalid message authentication salt
Testing in Windows and Ubuntu.
The same parameters were validated and work on Ubuntu with a command-line tool.
The text was updated successfully, but these errors were encountered: