Missing HTTP response header in auditlog prevents mlogc from parsing correctly in Nginx.

[cpintado]

I installed Nginx + modsecurity + mlogc.
When I try to parse the audit logs via cron, using mlogc-batch-load, I get the following error:

[Mon Nov 09 16:51:07 2015] [2] [31066/1dfb8a8] Invalid entry (failed to match regex): nginx 10.10.0.1 - - [09/Nov/2015:15:37:00 +0000] \"GET /1QsIkLNV.nl HTTP/1.1\" - 168 \"-\" \"-\" AcEwAcAcAcjcAcAcAcAcQcAc \"-\" /20151109/20151109-1537/20151109-153700-AcEwAcAcAcjcAcAcAcAcQcAc 0 3605 md5:0e5bb92e1a063ffcfbffd71b1cf785c0

I suspect the problem is that mlogc-batch-load fails to parse the HTTP response header. In Apache's audit files, the section F (HTTP response headers) has this format:

--ac634d14-F--
HTTP/1.1 403 Forbidden
Content-Length: 301
Connection: close
Content-Type: text/html; charset=iso-8859-1

while in Nginx has this format:

--57dfa307-F--
HTTP/1.1
Content-Type: text/html
Content-Length: 168
Connection: keep-alive

mlogc-batch-load parses the string after "HTTP/1.1" and stores it in the variable response_status. In the logs generated by Nginx, it doesn't found that string and the variable is passed to mlogc with the value "-", causing the parse error.

In file modsecurity/apache2/msc_logging.c we have this relevant piece of code:

/**
 * Produce an audit log entry.
 */
void sec_audit_logger(modsec_rec *msr) {


        /* There are no response headers (or the status line) in HTTP 0.9 */
        if (msr->response_headers_sent) {
            if (msr->status_line != NULL) {
                text = apr_psprintf(msr->mp, "%s %s\n", msr->response_protocol,
                        msr->status_line);
            } else {
                text = apr_psprintf(msr->mp, "%s %u\n", msr->response_protocol,
                        msr->response_status);
            }
            sec_auditlog_write(msr, text, strlen(text));

After some debugging, I notice that msr->status_line has value "", which satisfies the condition of "if (msr->status_line != NULL)" and then the status line gets printed as an empty string.

I think a workaround would be applying this patch.

diff -Naur modsecurity/apache2/msc_logging.c modsecurity-patched/apache2/msc_logging.c
--- modsecurity/apache2/msc_logging.c   2015-11-18 19:44:18.103869115 +0000
+++ modsecurity-patched/apache2/msc_logging.c   2015-11-18 20:03:26.826943022 +0000
@@ -853,7 +853,7 @@

         /* There are no response headers (or the status line) in HTTP 0.9 */
         if (msr->response_headers_sent) {
-            if (msr->status_line != NULL) {
+            if (msr->status_line != NULL && msr->status_line[0] != '\0' ) {
                 text = apr_psprintf(msr->mp, "%s %s\n", msr->response_protocol,
                         msr->status_line);
             } else {

I'm willing to do more tests or writing a patch if you'd like.

This happened to me in nginx 1.8.0 and ModSecurity 2.9.0.

[zimmerle]

Hi @charlymps,

We are current working on libModSecurity v3. The version 3 is a rewritten. The version 2.x (where this patch is applicable) will be available and maintained, thus, I would love to have your patch.

[csanders-git]

I traced this problem too for an independent issue since the problem is also present in all nginx auditlogs (native format)

at ../apache2/msc_logging.c:1897

1897 if (msr->response_headers_sent) {
(gdb) print msr->response_headers_sent
$2 = 1
(gdb) s
1898 if (msr->status_line != NULL) {
(gdb) print msr->status_line
$3 = 0x7cd4a9 ""
(gdb) s
1899 text = apr_psprintf(msr->mp, "%s %s\n", msr->response_protocol,
(gdb)

It looks like the patch specified will work to fix this issue

[zimmerle]

Fxied by #1171