New xbit features

[YoichSec]

1. Added two additional ways of tracking xbits that are essentially one-sided reverse operations (event src IP to xbit dst IP, event dst IP to xbit src IP).
   - I use this for things like detecting callbacks after a potentially malicious download where the cnc server is not necessarily the source of the download.
2. Expanded the definition of an xbit to store source and destination port. Expanded matching logic to enable matching by IP and port.
   - I needed this to refine the correlation of events for a particular application on one or more hosts.
3. Updated the Sagan-peek tool to show the added xbit fields
   - Edited this neat (undocumented?) tool to look into xbits in the ipc directory
4. Added three new ways of setting an xbit to work with Configure is not finding the mysql client lib even though it is installed. #2 above
   - Simply adding all port pairs is quite wasteful because of unimportant ephemeral ports
   - By default, set adds the default port for src and destination
   - Three other set operations add the relevant source port, destination port, or both

[YoichSec]

The addition of these features would require a corresponding update to documentation. I'd be happy to work with the dev team at some point to get it done.

[beave]

Very cool! I'll review ASAP. Oh, well aware of the documentation issue to :)

[beave]

Quick question. I'm probably missing something here:

I noticed this adds xbit options like "set_srcport" & "set_dstport". Why not just add the ports as part of "set". For example, When an xbit "set" is issued, it stores both the source and destination IP address. Could we just store the source ip, source port, destination ip and destination port right there? That data could be reference from later with "isset", etc.

Maybe you could give me an example rule so I can make sure I full have my head wrapped around this :)

Other than that, this looks good.

Thank you.

[YoichSec]

Yup, so that's what I did initially. The problem with that method is that the number of xbits being set increases rapidly and unnecessarily.
Take an existing recon xbit for example. If an attacker portscans a target, you'd probably get one recon xbit set. Now with "set" adding port information, depending on what kind of logs you're getting, you might get thousands of xbits set by the same rule - one for each source-destination port pair.
I wanted to avoid that by default, keep the behavior of existing rules more or less the same, and use additional set options to add port information as desired.
Also because of ephemeral ports, you'd almost always want to ignore one of the ports unless you're trying to track sessions, hence the options to specify src, dst, or both. Make sense?

[beave]

That makes complete sense. Awesome sauce :) Let me look over a couple more things and I think we're good.

[beave]

This looks great. It's gonna be fun to play and test!