New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add script to decode F5 BIG-IP cookies. #892

Closed
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
3 participants
@sethjackson

sethjackson commented May 26, 2017

This just adds a script to decode any unencrypted BIG-IP cookies in the response.

See this support article for information on the encoding scheme: https://support.f5.com/csp/article/K6917

@sethjackson

This comment has been minimized.

Show comment
Hide comment
@sethjackson

sethjackson commented Jul 31, 2017

Ping.

@cldrn

Thanks for the script and sorry for the late reply. The script looks good. I think we just need some small changes and we will be ready to commit.

Show outdated Hide outdated scripts/f5-cookie-decode.nse
local host = split[1]
local port = split[2]
local packed = string.pack("<I", tonumber(host))

This comment has been minimized.

@cldrn

cldrn Aug 9, 2017

Member

Please specify size because default size differs across platforms.

@cldrn

cldrn Aug 9, 2017

Member

Please specify size because default size differs across platforms.

Show outdated Hide outdated scripts/f5-cookie-decode.nse
end
if next(decoded) then
return decoded

This comment has been minimized.

@cldrn

cldrn Aug 9, 2017

Member

Can you make the script return a stdnse.output_table() to generate XML output automatically?

@cldrn

cldrn Aug 9, 2017

Member

Can you make the script return a stdnse.output_table() to generate XML output automatically?

Show outdated Hide outdated scripts/f5-cookie-decode.nse
@@ -0,0 +1,70 @@
-- See here: https://support.f5.com/csp/article/K6917
description = [[
Decodes any unencrypted F5 BIG-IP cookies in the HTTP response.

This comment has been minimized.

@cldrn

cldrn Aug 9, 2017

Member

Can you add a little more documentation about this? Maybe just add a reference link.

@cldrn

cldrn Aug 9, 2017

Member

Can you add a little more documentation about this? Maybe just add a reference link.

This comment has been minimized.

@sethjackson

sethjackson Aug 9, 2017

This would be the reference link no?

https://support.f5.com/csp/article/K6917

Not sure what else to add here or should that go in the description?

@sethjackson

sethjackson Aug 9, 2017

This would be the reference link no?

https://support.f5.com/csp/article/K6917

Not sure what else to add here or should that go in the description?

@sethjackson

This comment has been minimized.

Show comment
Hide comment
@sethjackson

sethjackson Aug 9, 2017

No problem. I've updated this. Let me know what you think.

Note that I added the

local host = tonumber(split[1])
...
if host then
  ...
end

to handle the case where the cookie is encrypted since the script does not work for encrypted cookies.

sethjackson commented Aug 9, 2017

No problem. I've updated this. Let me know what you think.

Note that I added the

local host = tonumber(split[1])
...
if host then
  ...
end

to handle the case where the cookie is encrypted since the script does not work for encrypted cookies.

@sethjackson

This comment has been minimized.

Show comment
Hide comment
@sethjackson

sethjackson Aug 9, 2017

Also note that it seems:

if next(decoded) then
  return decoded
end

Doesn't work with stdnse.output_table() so if there are no results there is some extra output
in the scan result.

Like so:

PORT    STATE SERVICE
443/tcp open  https
|_f5-cookie-decode: 

Nmap done: 1 IP address (1 host up) scanned in 0.83 seconds

Is there some way to avoid that?

sethjackson commented Aug 9, 2017

Also note that it seems:

if next(decoded) then
  return decoded
end

Doesn't work with stdnse.output_table() so if there are no results there is some extra output
in the scan result.

Like so:

PORT    STATE SERVICE
443/tcp open  https
|_f5-cookie-decode: 

Nmap done: 1 IP address (1 host up) scanned in 0.83 seconds

Is there some way to avoid that?

@nnposter

A few more ideas for your script.

Show outdated Hide outdated scripts/f5-cookie-decode.nse
local split = stdnse.strsplit("%.", cookie.value)
local host = tonumber(split[1])
local port = split[2]

This comment has been minimized.

@nnposter

nnposter Aug 12, 2017

This is shadowing function parameters host and port from above. While not technically incorrect, it muddies the code.

@nnposter

nnposter Aug 12, 2017

This is shadowing function parameters host and port from above. While not technically incorrect, it muddies the code.

Show outdated Hide outdated scripts/f5-cookie-decode.nse
local host = tonumber(split[1])
local port = split[2]
if host then

This comment has been minimized.

@nnposter

nnposter Aug 12, 2017

Strictly speaking, you are not really checking anywhere that the cookie truly has the expected value. All you know at this point is that the cookie consists of a valid number, integer or float, positive or negative, which is optionally followed by a dot and an arbitrary string. So the code tries to process values like -99E+99 or 0.!@$%, resulting in run-time complaints.

You might want to consider constructs like:

local chost,  cport = cookie.value:match("^(%d+)%.(%d+)%.")
if chost and tonumber(chost) < 0x100000000 and tonumber(cport) < 0x10000 then
@nnposter

nnposter Aug 12, 2017

Strictly speaking, you are not really checking anywhere that the cookie truly has the expected value. All you know at this point is that the cookie consists of a valid number, integer or float, positive or negative, which is optionally followed by a dot and an arbitrary string. So the code tries to process values like -99E+99 or 0.!@$%, resulting in run-time complaints.

You might want to consider constructs like:

local chost,  cport = cookie.value:match("^(%d+)%.(%d+)%.")
if chost and tonumber(chost) < 0x100000000 and tonumber(cport) < 0x10000 then
Show outdated Hide outdated scripts/f5-cookie-decode.nse
table.insert(values, utf8.codepoint(c))
end
local ip_address = stdnse.strjoin(".", values)

This comment has been minimized.

@nnposter

nnposter Aug 12, 2017

This can be collapsed into a one-liner like this one:

chost = table.concat({("BBBB"):unpack(("<I4"):pack(chost))}, ".", 1, 4)
@nnposter

nnposter Aug 12, 2017

This can be collapsed into a one-liner like this one:

chost = table.concat({("BBBB"):unpack(("<I4"):pack(chost))}, ".", 1, 4)
Show outdated Hide outdated scripts/f5-cookie-decode.nse
end
local ip_address = stdnse.strjoin(".", values)
port = tonumber(stdnse.tohex(string.pack("<H", port)), 16)

This comment has been minimized.

@nnposter

nnposter Aug 12, 2017

Similarly you could use pack - unpack to make it very clear what is going on here:

cport = (">I2"):unpack(("<I2"):pack(cport))
@nnposter

nnposter Aug 12, 2017

Similarly you could use pack - unpack to make it very clear what is going on here:

cport = (">I2"):unpack(("<I2"):pack(cport))
Show outdated Hide outdated scripts/f5-cookie-decode.nse
local ip_address = stdnse.strjoin(".", values)
port = tonumber(stdnse.tohex(string.pack("<H", port)), 16)
table.insert(decoded, string.format("%s:%s:%s", cookie.name, ip_address, port))

This comment has been minimized.

@nnposter

nnposter Aug 12, 2017

If you are ambitious, it would be nice if the XML output was truly structured. For cookie BIGipServeragdncdcqd05_pool=1193829386.24866.0000 this could be something like:

<cookie>
  <pool>agdncdcqd05_pool</pool>
  <address>
    <addr>10.100.40.71</addr>
    <type>ipv4</type>
  </address>
  <port>8801</port>
</cookie>

(Adding the IPv4 designation, which might look overbearing right now, also gives you the opportunity to add decoding for IPv6 cookies in the future without breaking backward compatibility of the output .)

You can still keep your current compact format for the regular textual output by maintaining two tables, one for the XML output and one for the textual, and then return the findings as follows:

  if #output > 0 then
    return output, stdnse.format_output(true, text_output)
  end

If you do decide to add the XML output then do not forget to add the corresponding XML output example to the documentation section above.

@nnposter

nnposter Aug 12, 2017

If you are ambitious, it would be nice if the XML output was truly structured. For cookie BIGipServeragdncdcqd05_pool=1193829386.24866.0000 this could be something like:

<cookie>
  <pool>agdncdcqd05_pool</pool>
  <address>
    <addr>10.100.40.71</addr>
    <type>ipv4</type>
  </address>
  <port>8801</port>
</cookie>

(Adding the IPv4 designation, which might look overbearing right now, also gives you the opportunity to add decoding for IPv6 cookies in the future without breaking backward compatibility of the output .)

You can still keep your current compact format for the regular textual output by maintaining two tables, one for the XML output and one for the textual, and then return the findings as follows:

  if #output > 0 then
    return output, stdnse.format_output(true, text_output)
  end

If you do decide to add the XML output then do not forget to add the corresponding XML output example to the documentation section above.

This comment has been minimized.

@nnposter

nnposter Aug 14, 2017

Just to make it clear, nmap does not give you a way how to produce a rich XML like the ideal I have outlined. This is what you could get:

<table key="cookies">
  <table>
    <elem key="pool">agdncdcqd05_pool</elem>
    <table key="address">
      <elem key="addr">10.100.40.71</elem>
      <elem key="type">ipv4</elem>
    </table>
    <elem key="port">8801</elem>
  </table>
  <table>
    ...another cookie...
  </table>
</table>

See https://nmap.org/book/nse-api.html#nse-structured-output

@nnposter

nnposter Aug 14, 2017

Just to make it clear, nmap does not give you a way how to produce a rich XML like the ideal I have outlined. This is what you could get:

<table key="cookies">
  <table>
    <elem key="pool">agdncdcqd05_pool</elem>
    <table key="address">
      <elem key="addr">10.100.40.71</elem>
      <elem key="type">ipv4</elem>
    </table>
    <elem key="port">8801</elem>
  </table>
  <table>
    ...another cookie...
  </table>
</table>

See https://nmap.org/book/nse-api.html#nse-structured-output

Show outdated Hide outdated scripts/f5-cookie-decode.nse
end
end
return decoded

This comment has been minimized.

@nnposter

nnposter Aug 12, 2017

As you said, it does not really make sense to return zero-length findings. You should be able to use #decoded for that test.

@nnposter

nnposter Aug 12, 2017

As you said, it does not really make sense to return zero-length findings. You should be able to use #decoded for that test.

Show outdated Hide outdated scripts/f5-cookie-decode.nse
local http = require "http"
local shortport = require "shortport"
local stdnse = require "stdnse"
local string = require "string"

This comment has been minimized.

@nnposter

nnposter Aug 12, 2017

Do not forget to import table.

@nnposter

nnposter Aug 12, 2017

Do not forget to import table.

@sethjackson

This comment has been minimized.

Show comment
Hide comment
@sethjackson

sethjackson Aug 14, 2017

Thanks! I've updated this. Is this sufficient so far?

I did not work on the structured output yet.

sethjackson commented Aug 14, 2017

Thanks! I've updated this. Is this sufficient so far?

I did not work on the structured output yet.

@sethjackson

This comment has been minimized.

Show comment
Hide comment
@sethjackson

sethjackson Aug 14, 2017

Added the structured output.

Unfortunately I have one more issue.
I'm getting an extra blank line inserted somehow?

| f5-cookie-decode:
|
|     pool: BIGipServer<pool_name>
|       address:
|         host: 10.1.1.100
|         port: 8080
|_       type: ipv4

When it should be like this:

| f5-cookie-decode:
|     pool: BIGipServer<pool_name>
|     address:
|       host: 10.1.1.100
|       port: 8080
|_      type: ipv4

sethjackson commented Aug 14, 2017

Added the structured output.

Unfortunately I have one more issue.
I'm getting an extra blank line inserted somehow?

| f5-cookie-decode:
|
|     pool: BIGipServer<pool_name>
|       address:
|         host: 10.1.1.100
|         port: 8080
|_       type: ipv4

When it should be like this:

| f5-cookie-decode:
|     pool: BIGipServer<pool_name>
|     address:
|       host: 10.1.1.100
|       port: 8080
|_      type: ipv4
@nnposter

This comment has been minimized.

Show comment
Hide comment
@nnposter

nnposter Aug 14, 2017

Two tweaks:

First, the description should be a little more, well, descriptive. You know what information F5 cookies carry but a casual user might not. It would be nice if somebody just browsing the scripts could get a basic idea what the script is about (without following up the provided link). It does not have to be extensive.

As an example, OpenVAS has this to say about the cookie:

The remote host appears to be a F5 BigIP load balancer which encodes within a cookie the IP address of the actual web server it is acting on behalf of. Additionally, information after 'BIGipServer' is configured by the user and may be the logical name of the device. These values may disclose sensitive information, such as internal IP addresses and names.

And Metasploit:

This module identifies F5 BigIP load balancers and leaks backend information (pool name, backend's IP address and port, routed domain) through cookies inserted by the BigIP system.

The second tweak is that the script should be renamed to be more consistent with other scripts, by prefixing it with "http-". It could be "http-bigip-cookie" or "http-f5-bigip-cookie".

nnposter commented Aug 14, 2017

Two tweaks:

First, the description should be a little more, well, descriptive. You know what information F5 cookies carry but a casual user might not. It would be nice if somebody just browsing the scripts could get a basic idea what the script is about (without following up the provided link). It does not have to be extensive.

As an example, OpenVAS has this to say about the cookie:

The remote host appears to be a F5 BigIP load balancer which encodes within a cookie the IP address of the actual web server it is acting on behalf of. Additionally, information after 'BIGipServer' is configured by the user and may be the logical name of the device. These values may disclose sensitive information, such as internal IP addresses and names.

And Metasploit:

This module identifies F5 BigIP load balancers and leaks backend information (pool name, backend's IP address and port, routed domain) through cookies inserted by the BigIP system.

The second tweak is that the script should be renamed to be more consistent with other scripts, by prefixing it with "http-". It could be "http-bigip-cookie" or "http-f5-bigip-cookie".

@sethjackson

This comment has been minimized.

Show comment
Hide comment
@sethjackson

sethjackson Aug 14, 2017

Ok. I'll update the script name and fix the description.
Any idea about that extra blank line in the output?

sethjackson commented Aug 14, 2017

Ok. I'll update the script name and fix the description.
Any idea about that extra blank line in the output?

@nnposter

This comment has been minimized.

Show comment
Hide comment
@nnposter

nnposter Aug 14, 2017

I'm getting an extra blank line inserted somehow?

That is the "unnamed" table representing each cookie. There are two ways how to deal with it:

  • Either maintain a parallel (normal) textual output. The value of this is that this output can be much more compact. (I already gave you an example how to return both results.)
  • Or do not build a cookie list but an associative array of the pools:
| f5-cookie-decode:
|   <pool_name>
|       port: 8080
|       address:
|         host: 10.1.1.100
|_       type: ipv4

nnposter commented Aug 14, 2017

I'm getting an extra blank line inserted somehow?

That is the "unnamed" table representing each cookie. There are two ways how to deal with it:

  • Either maintain a parallel (normal) textual output. The value of this is that this output can be much more compact. (I already gave you an example how to return both results.)
  • Or do not build a cookie list but an associative array of the pools:
| f5-cookie-decode:
|   <pool_name>
|       port: 8080
|       address:
|         host: 10.1.1.100
|_       type: ipv4
@nnposter

This comment has been minimized.

Show comment
Hide comment
@nnposter

nnposter Aug 14, 2017

BTW, you do not have to always use stdnse.output_table(). The reason for using it is if you are building an associative array then it remembers the order in which the individual keys were inserted. Since your output is just a list then there is no need for it (because it is already naturally ordered).

It could be as simple as:

local result = {
  pool = cookie.name:sub(12),
  address = {host = host, type = "ipv4"},
  port = port}
table.insert(output, result)

nnposter commented Aug 14, 2017

BTW, you do not have to always use stdnse.output_table(). The reason for using it is if you are building an associative array then it remembers the order in which the individual keys were inserted. Since your output is just a list then there is no need for it (because it is already naturally ordered).

It could be as simple as:

local result = {
  pool = cookie.name:sub(12),
  address = {host = host, type = "ipv4"},
  port = port}
table.insert(output, result)
@sethjackson

This comment has been minimized.

Show comment
Hide comment
@sethjackson

sethjackson Aug 14, 2017

Ah ok. I updated the script to avoid most of those calls now. :)

sethjackson commented Aug 14, 2017

Ah ok. I updated the script to avoid most of those calls now. :)

@nnposter

This comment has been minimized.

Show comment
Hide comment
@nnposter

nnposter Aug 14, 2017

Port is not really part of the IP address so it should be moved one level up.

Please take a look at other scripts (e.g. ssl-enum-ciphers) how to include @xmloutput in the documentation.

@cldrn Paulino, any more thoughts before merging?

nnposter commented Aug 14, 2017

Port is not really part of the IP address so it should be moved one level up.

Please take a look at other scripts (e.g. ssl-enum-ciphers) how to include @xmloutput in the documentation.

@cldrn Paulino, any more thoughts before merging?

@sethjackson

This comment has been minimized.

Show comment
Hide comment
@sethjackson

sethjackson Aug 14, 2017

Please take a look at other scripts (e.g. ssl-enum-ciphers) how to include @xmloutput in the documentation.

👍 Thanks! I will add that soon.

I've fixed the port part.

sethjackson commented Aug 14, 2017

Please take a look at other scripts (e.g. ssl-enum-ciphers) how to include @xmloutput in the documentation.

👍 Thanks! I will add that soon.

I've fixed the port part.

@sethjackson

This comment has been minimized.

Show comment
Hide comment
@sethjackson

sethjackson Aug 15, 2017

I added the @xmloutput documentation.
Also removed string require.

sethjackson commented Aug 15, 2017

I added the @xmloutput documentation.
Also removed string require.

@nnposter

This comment has been minimized.

Show comment
Hide comment
@nnposter

nnposter Aug 16, 2017

Looks good to me. Let's give it a few days in case @cldrn has more feedback.

One triviality:
Please replace pairs(response.cookies) with ipairs(...) because you are iterating over a list, not an associative array.

One false-negative case:
The script currently does not work correctly if the targeted path is a redirect. The reason is that by default http.get follows redirects so the returned response is not from the original request but from the destination of the redirect, which might reside outside of the original pool (and therefore you will not get the desired cookies). The remedy is to disable redirects by setting option redirect_ok to false:

local response = http.get(host, port, path, {redirect_ok=false})

nnposter commented Aug 16, 2017

Looks good to me. Let's give it a few days in case @cldrn has more feedback.

One triviality:
Please replace pairs(response.cookies) with ipairs(...) because you are iterating over a list, not an associative array.

One false-negative case:
The script currently does not work correctly if the targeted path is a redirect. The reason is that by default http.get follows redirects so the returned response is not from the original request but from the destination of the redirect, which might reside outside of the original pool (and therefore you will not get the desired cookies). The remedy is to disable redirects by setting option redirect_ok to false:

local response = http.get(host, port, path, {redirect_ok=false})
@sethjackson

This comment has been minimized.

Show comment
Hide comment
@sethjackson

sethjackson Aug 16, 2017

Ok thanks!. I've added those changes.

sethjackson commented Aug 16, 2017

Ok thanks!. I've added those changes.

@cldrn

This comment has been minimized.

Show comment
Hide comment
@cldrn

cldrn Aug 16, 2017

Member

Great work. I don't have any more requests except for including the reference URL in the description so it gets displayed on NSE documentation as well. (Instead of a comment inside the file)

Thanks for the effort @sethjackson!

Member

cldrn commented Aug 16, 2017

Great work. I don't have any more requests except for including the reference URL in the description so it gets displayed on NSE documentation as well. (Instead of a comment inside the file)

Thanks for the effort @sethjackson!

@sethjackson

This comment has been minimized.

Show comment
Hide comment
@sethjackson

sethjackson Aug 16, 2017

Thanks! I've added the URL to the description instead.

sethjackson commented Aug 16, 2017

Thanks! I've added the URL to the description instead.

@nmap-bot nmap-bot closed this in b2fb0b2 Aug 17, 2017

@sethjackson sethjackson deleted the sethjackson:f5-cookie-decode branch Aug 17, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment