Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Adds script to extract SMB Enum Services #987

Closed
wants to merge 50 commits into
from

Conversation

Projects
None yet
2 participants
Contributor

rewanth1997 commented Aug 25, 2017

There is an issue with smb-enum-services I created.

While you execute the script, you can observe the debug statements added, focus on the debug statements in line 3502 in msrpc.lua. Since we are not sure of the offset, I'm iterating over all the values and you can see the output as numbers like 11, 0 and nil. These are the error status codes as represented in here.

11 refers to ERROR_BAD_FORMAT
0 refers to no error
nil doesn't exist at all

I read the documentation as mentioned in https://msdn.microsoft.com/en-us/library/windows/desktop/ms682637(v=vs.85).aspx.

Any kind suggestions are appreciated, Thanks !

Check this out @dmiller-nmap @cldrn .

Contributor

rewanth1997 commented Aug 25, 2017

The above comment explains that I got error as ERROR_BAD_FORMAT, even after sending the correct request(I think so). So, here is something new I found.

The last 2 commits tries adding new function, enumservicestatusexw to retrieve the list of services. Reasons to add this service.

  1. Makes debugging easier, since I can compare the packets being sent between Nmap and the packets captured using Wireshark while running psservices.exe file.
  2. Retrieves more data than required.

The issue with this service is,
As mentioned in the above comment, since we are not sure of the offset, I'm iterating over all the values and you can see the error status code as 12 instead of 11 as above. As per the Microsoft error codes list in here,

12 (0xC) refers to ERROR_INVALID_ACCESS
The access code is invalid.

From these series of errors and debugging I think there is some issue with the crafting the request packets before being sent.

Any kind of help is highly appreciated !!

Contributor

rewanth1997 commented Aug 26, 2017 edited

You can ignore the above 2 comments for now(I think so).

The final code as of now, captures the entire buffer and unmarshalls pcbBytesNeeded, lpServicesReturned, lpResumeHandle, ReturnValue, pcbBytesAcquired successfully. The next issue I'm having is unmarshalling the LPENUM_SERVICE_STATUS data type as mentioned in here.

I tried unmarshalling the captured lpservices but the SERVICE_STATUS of all services are present in the beginning of the hexdump while lpDisplayName and lpServiceName of all services are at the ending of the buffer. I tried by unmarshalling it the lpservices with msrpctypes.unmarshall_int8_array as mentioned in here, but no use.

Please help me, in unmarshalling the LPENUM_SERVICE_STATUS structure to retrieve all the parameters in that enum for all the services using the correct offset. @dmiller-nmap @cldrn

Contributor

rewanth1997 commented Aug 26, 2017

The final code is ready, it will be pushed once I finish the documentation part.

Contributor

rewanth1997 commented Aug 26, 2017

Please review the code asap, @dmiller-nmap @bonsaiviking

Very exciting to see this working! Some polish items and some issues to deal with, and then you can commit.

nselib/msrpc.lua
+-- starting position of actual data.
+--@return startpos Returns the strating position of the string.
+--@return string Returns the string of unmarshalled data.
+function unmarshall_str(arguments, startpos, endpos, decoder, offset)
@dmiller-nmap

dmiller-nmap Aug 28, 2017

Instead of requiring the caller to provide endpos, decoder, and offset, write it as a simpler and easier to use function and move it to msrpctypes.lua:

--- Unmarshalls a null-terminated Unicode string based upon a 32-bit offset (LPTSTR)
-- @param data The data being processed
-- @param pos  The current position within the data
-- @return The new position
-- @return The string with null removed
function unmarshall_lptstr(data, pos)

This function will unmarshall the offset, then start from that offset looking at every 2 bytes for a "\0\0" which is the null terminator. When that is found, extract the string from the offset to the terminator and return it. The returned position will be just after the 4-byte offset; the string itself is not part of that calculation.

Example use:

pos, serviceName = unmarshall_lptstr(arguments, pos)
pos, displayName = unmarshall_lptstr(arguments, pos)
serviceName = unicode.utf16to8(serviceName)
displayName = unicode.utf16to8(displayName)
nselib/msrpc.lua
+ pos, serviceStatus = msrpctypes.unmarshall_SERVICE_STATUS(arguments, pos)
+
+ prevOffset, serviceName = unmarshall_str(arguments, serviceNameOffset, prevOffset, unicode.utf16to8)
+ prevOffset, displayName = unmarshall_str(arguments, displayNameOffset, prevOffset, unicode.utf16to8)
@dmiller-nmap

dmiller-nmap Aug 28, 2017

This is assuming that the display name will always be packed right after the service name. We can't guarantee that this will always be the case, since it's not documented anywhere. This is an opportunity to make the unmarshall_str function more general and easier to use: change it so that it unmarshalls a null-terminated unicode string from anywhere in the data based on offset. I'll describe better up at unmarshall_str.

nselib/msrpc.lua
+
+ stdnse.debug3("Arguments = %s", arguments)
+ stdnse.debug3("Length of arguments = %d", arguments:len())
+ stdnse.debug3("Hex format of arguments = %s", stdnse.tohex(arguments))
@dmiller-nmap

dmiller-nmap Aug 28, 2017

Remove a few of these debug statements now that it's working. Printing the arguments directly and in hex is particularly unnecessary now. Also, may want to check that result["arguments"] actually exists here; otherwise this will result in a script crash.

+
+ -- [out,ref] [range(0,0x40000)] uint32 *pcbBytesNeeded,
+ pos, result["pcbBytesNeeded"] = msrpctypes.unmarshall_int32(arguments, pos)
+
@dmiller-nmap

dmiller-nmap Aug 28, 2017

Unpack the rest of the return values here, too. Most especially you need to get the return value of the function call, since this could indicate that there's no point in continuing. Here are the possibilities based on error number:

  • ERROR_SUCCESS - Probably not going to happen because we sent 0 bytes of buffer, but it could mean there are no services available.
  • ERROR_INSUFFICIENT_BUFFER or ERROR_MORE_DATA - This is what we expect: we need to send a bigger buffer and/or there is more data after unpacking the current buffer.
  • ERROR_ACCESS_DENIED or anything else - stop processing and return an error; no point in making further calls. This will be the most common state for modern Windows without domain admin credentials.
nselib/msrpc.lua
+
+ ------- Functional calls here are made to retrieve the data -------------------------
+
+ local MAX_BUFFER_SIZE = 0x400
@dmiller-nmap

dmiller-nmap Aug 28, 2017

The max buffer size is defined for WinXP and Server 2003 as 64K according to MSDN. So this should be 0xfa00 instead.

nselib/msrpc.lua
+
+ -- If larger value is assigned to result["pcbBytesNeeded"], errored response
+ -- will be returned.
+ if result["pcbBytesNeeded"] < MAX_BUFFER_SIZE then
@dmiller-nmap

dmiller-nmap Aug 28, 2017

This and the check on lines 3631 to 3633 can be reduced to using math.min(result.pcbBytesNeeded, MAX_BUFFER_SIZE) directly in the call to enumservicestatusparams below.

nselib/msrpc.lua
+
+ stdnse.debug3("Arguments = %s", arguments)
+ stdnse.debug3("Length of arguments = %d", length)
+ stdnse.debug3("Hex format of arguments = %s", stdnse.tohex(arguments))
@dmiller-nmap

dmiller-nmap Aug 28, 2017

more debug statements to reduce.

scripts/smb-enum-services.nse
@@ -0,0 +1,1144 @@
+local msrpc = require "msrpc"
+local smb = require "smb"
+local smbauth = require "smbauth"
@dmiller-nmap

dmiller-nmap Aug 28, 2017

No need to include smbauth here; that's done by smb.lua.

scripts/smb-enum-services.nse
+References:
+* https://msdn.microsoft.com/en-us/library/windows/desktop/ms682637(v=vs.85).aspx
+* https://msdn.microsoft.com/en-us/library/windows/desktop/ms682651(v=vs.85).aspx
+* https://github.com/samba-team/samba/blob/d8a5565ae647352d11d622bd4e73ff4568678a7c/librpc/idl/svcctl.idl
@dmiller-nmap

dmiller-nmap Aug 28, 2017

These are useful references for msrpc.lua, but not particularly helpful for users. You could link to the docs for sc.exe or the Wikipedia page for Windows service instead, and move these links to msrpc.lua or msrpctypes.lua

scripts/smb-enum-services.nse
+-- nmap --script smb-enum-services.nse --script-args smbusername=<username>,smbpass=<password> -p445 <host>
+--
+-- The following lines displays the normal and xml results when this script
+-- was run against Windows 2003 R2 x64 Enterprise Server.
@dmiller-nmap

dmiller-nmap Aug 28, 2017

These explanatory lines won't make sense in the NSEdoc layout of the website, and the results are similar on Windows 10. I would leave them out.

scripts/smb-enum-services.nse
+-- | check_point: 0
+-- | wait_hint: 0
+-- | state:
+-- |
@dmiller-nmap

dmiller-nmap Aug 28, 2017

state isn't showing up, which is because it's being mixed up with a different enum. The constants in msrpctypes.lua for svcctl_State are for the dwServiceState input parameter, and are unrelated to the actual values defined for dwCurrentState in the SERVICE_STATE struct. I don't see that these are used anywhere else, either, so just change the contents of the svcctl_State table in msrpctypes.lua to match the values in that reference, which ought to fix it.

@rewanth1997

rewanth1997 Aug 28, 2017

Contributor

Should I change the table being passed in here, https://github.com/nmap/nmap/blob/master/nselib/msrpctypes.lua#L4470 ? Or should I create a duplicate of msrpctypes.unmarshall_SERVICE_STATUS function and change this line, https://github.com/nmap/nmap/blob/master/nselib/msrpctypes.lua#L4515 ?

scripts/smb-enum-services.nse
+-- | serviceName: ClipSrv
+-- | serviceStatus:
+-- | check_point: 0
+-- | wait_hint: 0
@dmiller-nmap

dmiller-nmap Aug 28, 2017

check_point, wait_hint, win32_exit_code, and service_exit_code are not really helpful or interesting items, so in the interest of reducing the size of the output somewhat, let's leave them out. Loop over the return table of svcctl_enumservicesstatusw and create a stdnse.output_table for each one, filling it out with the values we care about. This will ensure that the keys are in the same order for each service, which is important in reading and comparing results.

scripts/smb-enum-services.nse
+ -- SERVICE_ACTIVE - 0x00000001
+ -- SERVICE_INACTIVE - 0x00000002
+ -- SERVICE_STATE_ALL - 0x00000003 (default)
+ local dwservicestate = 0x00000003
@dmiller-nmap

dmiller-nmap Aug 28, 2017

Let's make the default SERVICE_ACTIVE. This will reduce output somewhat and also makes it more interesting to look at, since the set of running services can vary more than the set of available services depending on system configuration. Later, we can consider making this a script-arg.

scripts/smb-enum-services.nse
+
+ smb.stop(smbstate)
+
+ return result
@dmiller-nmap

dmiller-nmap Aug 28, 2017

Here is the recommended format of the result table, with each of the tables being the result of a call to stdnse.output_table() to preserve the ordering of the keys:

  ALG:
    display_name: Application Layer Gateway Service
    state:
      SERVICE_STATE_RUNNING
    type:
      SERVICE_TYPE_WIN32_OWN_PROCESS
    controls_accepted:
      SERVICE_CONTROL_NETBINDADD

Of course I've just truncated the lists for illustration; you would still show all the values for controls, type, and state. Maybe later once we've committed it we can work on better display formats. But this is at least not as large as the current format.

Some changes to unmarshall_lptstr

nselib/msrpctypes.lua
+-- @return The string with null removed
+function unmarshall_lptstr(arguments, startpos)
+
+ local offset = 5
@dmiller-nmap

dmiller-nmap Aug 28, 2017

Why is the offset 5? This function should handle reading the offset, extracting the string, and returning the new position and the string. In pseudocode:

function unmarshall_lptstr(args, pos)
  pos, offset = unmarshall_int32(args, pos)
  endpos = offset
  while endpos < #args
    if args:sub(endpos, endpos+2) == "\0\0"
      break
    end
  end
  return pos, args:sub(offset, endpos)
@rewanth1997

rewanth1997 Aug 28, 2017

Contributor

Offset 5 must be added to line 4545 because while receive the response from the server the first 4 bytes are allocated for displaying the size of the packet and so we have to start unmarshalling from the 5th position.

While we extract the starting position of the strings from the binary response, we get the starting position wrt to 1, since we were starting unmarshalling from 5th position, 5 has to be added to the unpack function.

If you think this will be a problem, we can change this by passing a parameter as offset like

function unmarshall_lptstr(arguments, startpos, offset)
offset = offset or 0
......
end

Does this look nice?

nselib/msrpctypes.lua
+
+ while s ~= "\0\0" do
+ s = string.unpack("<c2", arguments, startpos + offset)
+ str = str .. s
@dmiller-nmap

dmiller-nmap Aug 28, 2017

instead of using string.unpack, you can use string.sub to grab a 2-byte substring. Then just continue calculating the current position and return string.sub(arguments, startpos, curpos). This avoids repeated string building via concatenation, which can be very bad for performance due to repeated reallocation of str.

@rewanth1997

rewanth1997 Aug 29, 2017

Contributor

Your idea of using string.sub degrades the performance because when we know the end position there is no point in iterating all chars finding for NULL byte. I think its better to pass endpos argument to unmarshall_lptstr to improve the performance.

Your views?

@rewanth1997

rewanth1997 Aug 29, 2017

Contributor

With the above changes you suggested I made a PR #991 but for the sake of this script I added a local function in msrpc.lua as optimized_unmarshall_lptstr() which accepts end position of the string as a parameter and this is a bit faster than the function in #991 .

Contributor

rewanth1997 commented Sep 2, 2017

@dmiller-nmap , Please review the final code.

@nmap-bot nmap-bot closed this in 8e717e1 Sep 7, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment