Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Processing of malformed HTTP header names #573

Closed
nnposter opened this Issue Oct 23, 2016 · 0 comments

Comments

Projects
None yet
1 participant

nnposter commented Oct 23, 2016 edited

Local function parse_header() in library http.lua performs processing of HTTP response headers. It currently returns with an error (and no other results) if a header name is malformed, resulting in a completely failed response. This can be caused by an invalid character in the name or just superfluous whitespace between the name and the colon. This rigidity is causing problems with testing not-so-compliant targets.

A real-world example on which NSE scripts fail:

HTTP/1.1 200 OK
Server: Netgear
Content-Type: text/html
Pragma: no-cache
Last Modified: Fri, 16 July 2001 01:01:01 GMT
Connection: close

In contrast, browsers and other tools tend to handle this situation more gracefully:

  • Firefox ignores the particular header, continuing with the next header.
  • Internet Explorer tries to salvage the name if possible. As an example, it will treat line aaa bbb: ccc as header aaa bbb, despite the illegal space in the middle.
  • Wireshark will treat the remainder of the response as the body.

I would like to propose a change in parse_header() to mimic the Firefox behavior. Namely, a header with a malformed name will be skipped, i.e. it will not show up in the header table, but its original textual representation is still preserved in rawheader.

A corresponding patch is pretty straightforward. For the sake of clarity the patch presented here ignores whitespace changes (option -b). A real patch would include proper indentation.

--- ../../svn/nmap/nselib/http.lua  2016-09-27 12:26:44.928032942 -0600
+++ nselib/http.lua 2016-10-27 07:22:15.613209959 -0600
@@ -661,10 +661,15 @@
   while pos <= #header do
     -- Get the field name.
     e, name = get_token(header, pos)
-    if not name or e > #header or string.sub(header, e, e) ~= ":" then
-      return nil, string.format("Can't get header field name at %q", string.sub(header, pos, pos + 30))
+    -- Do not bail out if the header is malformed. Consume the header line
+    -- anyway, getting to the next header, but do not create a new entry in
+    -- the "header" table.
+    if e then
+      if header:sub(e, e) ~= ":" then
+        name = nil
     end
     pos = e + 1
+    end

     -- Skip initial space.
     pos = skip_lws(header, pos)
@@ -680,6 +685,7 @@
       pos = skip_lws(header, pos)
     end

+    if name then
     -- Set it in our table.
     name = string.lower(name)
     local value = table.concat(words, " ")
@@ -699,6 +705,7 @@
         -- Ignore any cookie parsing error
       end
     end
+    end

     -- Next field, or end of string. (If not it's an error.)
     s, e = string.find(header, "^\r?\n", pos)

Please provide feedback on whether this change would cause any issues.

@nmap-bot nmap-bot closed this in 2be31d5 Oct 31, 2016

suraj51k added a commit to suraj51k/nmap that referenced this issue Jan 31, 2017

Allows processing of HTTP responses with malformed headers. Such head…
…er lines are still captured in the rawheader list but skipped otherwise. Closes #573.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment