Parsing issues when logs contain pipes

[adepasquale]

I have some BRO_HTTP logs containing pipes in some of the tab-separated fields, e.g. an Havij-made SQL injection attempt with double-pipe string concatenation like the following:

1430161421.432863    Z1b8x42wevxlEVgEbz      10.0.0.2   32314   10.0.0.1   80      1       GET     www.example.com /index.asp?id=convert(int,chr(114)||chr(51)||chr(100)||chr(109)||chr(48)||chr(118)||chr(51)||chr(95)||chr(104)||chr(118)||chr(106)||chr(95)||chr(105)||chr(110)||chr(106)||chr(101)||chr(99)||chr(116)||chr(105)||chr(111)||chr(110))--      -       Mozilla/4.0       0       1239  200      OK      -       -       -       (empty) -       -       -       -       -       KKBOqc6QgqXwbOaIkz      text/html

When the BRO_HTTP file is processed by syslog-ng, every tab gets replaced by pipes:

source s_bro_http { file("/nsm/bro/logs/current/http_eth1.log" flags(no-parse) program_override("bro_http")); };
rewrite r_pipes { subst("\t", "|", value("MESSAGE") flags(global)); };
parser p_db { db-parser(file("/opt/elsa/node/conf/patterndb.xml")); };
template t_db_parsed { template("$R_UNIXTIME\t$HOST\t$PROGRAM\t${.classifier.class}\t$MSGONLY\t${i0}\t${i1}\t${i2}\t${i3}\t${i4}\t${i5}\t${s0}\t${s1}\t${s2}\t${s3}\t${s4}\t${s5}\n"); };
destination d_elsa { program("perl /opt/elsa/node/elsa.pl -c /etc/elsa_node.conf" template(t_db_parsed)); };

log {
    source(s_bro_http);
    rewrite(r_pipes);
    parser(p_db);
    destination(d_elsa);
};

As the Security Onion PatternDB configuration /opt/elsa/node/conf/patterndb.xml splits BRO_HTTP fields based on pipes, ELSA cannot show the above sample log correctly.

To my knowledge, this bug could be fixed by:

• escaping pre-existing pipes before the r_pipes rewrite directive; or
• changing the Security Onion parsers to use tabs instead of pipes.

@mcholste @dougburks what do you think?

[dougburks]

Please see Issue 447:
Security-Onion-Solutions/security-onion#447

[adepasquale]

Thanks @dougburks, I think this issue can be closed.