New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bro's http.log needs to be per-interface #205

Closed
GoogleCodeExporter opened this Issue Mar 24, 2015 · 11 comments

Comments

Projects
None yet
1 participant
@GoogleCodeExporter
Email conversation with Seth Hall:

"> My next step was going to be replacing httpry with Bro's http.log. But I 
realized last night that there will be a problem in Sguil when right-clicking 
an HTTP transaction and requesting the full pcap. Since we have one http.log 
which could be covering multiple interfaces, Sguil won't know which interface 
to request the pcap from.  I don't have access to a box right now...does 
http.log contain the interface that saw the transaction?  If so, perhaps I can 
tweak Sguil's http-agent to grep for the desired interface. If not, any other 
ideas?

I was about to reply and say that I didn't know what to do, but then I had an 
idea. :)  You can actually use the logging framework to split any of the logs 
like this fairly arbitrarily.  I attached a file named security-onion.bro which 
will do this for you with the http.log and conn.log (turning them into 
http_eth0.log, http_eth1.log, conn_eth0.log, conn_eth1.log, etc.) if the user 
has deployed as a cluster.  If they deploy in standalone mode they will just 
have http.log and conn.log

I recommend dropping the security-onion.bro script into 
<prefix>/share/bro/site/ and adding "@load security-onion" to local.bro."


event bro_init()
        {
        if ( reading_live_traffic() )
                {
                Log::remove_default_filter(HTTP::LOG);
                Log::add_filter(HTTP::LOG, [$name = "http-interfaces",
                                            $path_func(id: Log::ID, path: string, rec: HTTP::Info) = 
                                                { 
                                                local peer = get_event_peer()$descr;
                                                if ( peer in Cluster::nodes && Cluster::nodes[peer]?$interface )
                                                        return cat("http_", Cluster::nodes[peer]$interface);
                                                else
                                                        return "http";
                                                }
                                            ]);

                Log::remove_default_filter(Conn::LOG);
                Log::add_filter(Conn::LOG, [$name = "conn-interfaces",
                                            $path_func(id: Log::ID, path: string, rec: Conn::Info) = 
                                                { 
                                                local peer = get_event_peer()$descr;
                                                if ( peer in Cluster::nodes && Cluster::nodes[peer]?$interface )
                                                        return cat("conn_", Cluster::nodes[peer]$interface);
                                                else
                                                        return "conn";
                                                }
                                            ]);
                }
        }

Original issue reported on code.google.com by doug.bu...@gmail.com on 24 Jan 2012 at 1:49

@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

[deleted comment]
[deleted comment]
@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

Original comment by doug.bu...@gmail.com on 9 May 2012 at 10:43

  • Changed title: Bro's http.log needs to be per-interface
  • Added labels: ****
  • Removed labels: ****

Original comment by doug.bu...@gmail.com on 9 May 2012 at 10:43

  • Changed title: Bro's http.log needs to be per-interface
  • Added labels: ****
  • Removed labels: ****
@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

[deleted comment]
[deleted comment]
@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

[deleted comment]
[deleted comment]
@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

[deleted comment]
[deleted comment]
@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

/usr/bin/fpm -s dir -t deb -n securityonion-bro-security-onion -v 20120511 
/usr/local/share/bro/site/security-onion.bro 
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
Created /home/build/20120511/securityonion-bro-security-onion_20120511_i386.deb

Original comment by doug.bu...@gmail.com on 9 May 2012 at 12:50

  • Added labels: ****
  • Removed labels: ****
/usr/bin/fpm -s dir -t deb -n securityonion-bro-security-onion -v 20120511 
/usr/local/share/bro/site/security-onion.bro 
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
Created /home/build/20120511/securityonion-bro-security-onion_20120511_i386.deb

Original comment by doug.bu...@gmail.com on 9 May 2012 at 12:50

  • Added labels: ****
  • Removed labels: ****
@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

[deleted comment]
[deleted comment]
@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

Added the following to security-onion-upgrade.sh:


sed -i 's| |=|g' $CONF
source $CONF
if [ "$VERSION" = "20120508" ]; then
        NEW="20120511"
        echo "**********************************************"   | $LOGGER
        echo "* Upgrading from $VERSION to $NEW."               | $LOGGER
        echo "**********************************************"   | $LOGGER
        DIR="/nsm/backup/$NEW"
        mkdir -p $DIR                                           | $LOGGER
        cd $DIR

    if pgrep httpry>/dev/null; then
        echo "* Stopping old httpry processes."     | $LOGGER
        nsm_sensor_ps-stop --only-httpry
        nsm_sensor_ps-stop --only-httpry-agent
        pkill -f httpry
        echo ""
    fi

        for FILE in securityonion-nsmnow-admin-scripts_20120511_i386.deb securityonion-http-agent_20120511_i386.deb securityonion-bro-security-onion_20120511_i386.deb; do
                echo -n "* Downloading $FILE..."                | $LOGGER
                wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
                if [ $? -eq 1 ]; then
                        echo "FAIL"     | $LOGGER
                        exit 1
                else
                        echo "OK"       | $LOGGER
                fi
        done

        echo -n "* Installing downloaded packages..." | $LOGGER
        dpkg -i *.deb                                           >> $LOG
        if [ $? -eq 1 ]; then
                echo "FAIL"     | $LOGGER
                exit 1
        else
                echo "OK"       | $LOGGER
        fi
    echo ""

    echo "* Updating Bro."         | $LOGGER
    echo "@load security-onion" >> /usr/local/share/bro/site/local.bro
    broctl install      | $LOGGER
    if pgrep -f broctl>/dev/null; then
        broctl restart  | $LOGGER
        sleep 5
        echo ""
    fi

    grep -v "^#" /etc/nsm/sensortab |awk '{print $1}' |while read SENSOR; do
        mv /etc/nsm/$SENSOR/httpry_agent.exclude /etc/nsm/$SENSOR/http_agent.exclude
        mv /etc/nsm/$SENSOR/httpry_agent.conf /etc/nsm/$SENSOR/http_agent.conf
        echo "# LOG_FORMAT" >> /etc/nsm/$SENSOR/http_agent.conf
        echo "# httpry or suricata or bro" >> /etc/nsm/$SENSOR/http_agent.conf
        echo "set LOG_FORMAT bro" >> /etc/nsm/$SENSOR/http_agent.conf
        nsm_sensor_ps-start --sensor-name=$SENSOR --only-http-agent | $LOGGER
    done

        sed -i "s|VERSION=$VERSION|VERSION=$NEW|g" $CONF        | $LOGGER
        echo "* Upgrade to $NEW complete."                      | $LOGGER
        echo 
fi

Original comment by doug.bu...@gmail.com on 10 May 2012 at 12:19

  • Added labels: ****
  • Removed labels: ****
Added the following to security-onion-upgrade.sh:


sed -i 's| |=|g' $CONF
source $CONF
if [ "$VERSION" = "20120508" ]; then
        NEW="20120511"
        echo "**********************************************"   | $LOGGER
        echo "* Upgrading from $VERSION to $NEW."               | $LOGGER
        echo "**********************************************"   | $LOGGER
        DIR="/nsm/backup/$NEW"
        mkdir -p $DIR                                           | $LOGGER
        cd $DIR

    if pgrep httpry>/dev/null; then
        echo "* Stopping old httpry processes."     | $LOGGER
        nsm_sensor_ps-stop --only-httpry
        nsm_sensor_ps-stop --only-httpry-agent
        pkill -f httpry
        echo ""
    fi

        for FILE in securityonion-nsmnow-admin-scripts_20120511_i386.deb securityonion-http-agent_20120511_i386.deb securityonion-bro-security-onion_20120511_i386.deb; do
                echo -n "* Downloading $FILE..."                | $LOGGER
                wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
                if [ $? -eq 1 ]; then
                        echo "FAIL"     | $LOGGER
                        exit 1
                else
                        echo "OK"       | $LOGGER
                fi
        done

        echo -n "* Installing downloaded packages..." | $LOGGER
        dpkg -i *.deb                                           >> $LOG
        if [ $? -eq 1 ]; then
                echo "FAIL"     | $LOGGER
                exit 1
        else
                echo "OK"       | $LOGGER
        fi
    echo ""

    echo "* Updating Bro."         | $LOGGER
    echo "@load security-onion" >> /usr/local/share/bro/site/local.bro
    broctl install      | $LOGGER
    if pgrep -f broctl>/dev/null; then
        broctl restart  | $LOGGER
        sleep 5
        echo ""
    fi

    grep -v "^#" /etc/nsm/sensortab |awk '{print $1}' |while read SENSOR; do
        mv /etc/nsm/$SENSOR/httpry_agent.exclude /etc/nsm/$SENSOR/http_agent.exclude
        mv /etc/nsm/$SENSOR/httpry_agent.conf /etc/nsm/$SENSOR/http_agent.conf
        echo "# LOG_FORMAT" >> /etc/nsm/$SENSOR/http_agent.conf
        echo "# httpry or suricata or bro" >> /etc/nsm/$SENSOR/http_agent.conf
        echo "set LOG_FORMAT bro" >> /etc/nsm/$SENSOR/http_agent.conf
        nsm_sensor_ps-start --sensor-name=$SENSOR --only-http-agent | $LOGGER
    done

        sed -i "s|VERSION=$VERSION|VERSION=$NEW|g" $CONF        | $LOGGER
        echo "* Upgrade to $NEW complete."                      | $LOGGER
        echo 
fi

Original comment by doug.bu...@gmail.com on 10 May 2012 at 12:19

  • Added labels: ****
  • Removed labels: ****
@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

Tested by:
Scott Runnels
Tom De Vries
David Zawdie

Original comment by doug.bu...@gmail.com on 10 May 2012 at 12:22

  • Added labels: ****
  • Removed labels: ****
Tested by:
Scott Runnels
Tom De Vries
David Zawdie

Original comment by doug.bu...@gmail.com on 10 May 2012 at 12:22

  • Added labels: ****
  • Removed labels: ****
@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

Published:
http://securityonion.blogspot.com/2012/05/security-onion-20120511-now-available.
html

Original comment by doug.bu...@gmail.com on 10 May 2012 at 12:22

  • Added labels: ****
  • Removed labels: ****
Published:
http://securityonion.blogspot.com/2012/05/security-onion-20120511-now-available.
html

Original comment by doug.bu...@gmail.com on 10 May 2012 at 12:22

  • Added labels: ****
  • Removed labels: ****
@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

Original comment by doug.bu...@gmail.com on 10 May 2012 at 12:23

  • Changed state: Verified
  • Added labels: ****
  • Removed labels: ****

Original comment by doug.bu...@gmail.com on 10 May 2012 at 12:23

  • Changed state: Verified
  • Added labels: ****
  • Removed labels: ****
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment