snorby-db-fix is causing problems with large/busy snorby databases #201

Closed
GoogleCodeExporter opened this Issue Mar 24, 2015 · 4 comments

Comments

Projects
None yet
1 participant
@GoogleCodeExporter
Disabling until we can come up with a better fix.

Original issue reported on code.google.com by doug.bu...@gmail.com on 20 Jan 2012 at 6:38

@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

Updated /etc/cron.d/snorby-db-fix:

# Disabled 20120119 due to performance issues with larger databases
# */5 * * * * root  /usr/local/bin/snorby-db-fix >> 
/var/log/nsm/snorby-db-fix.log

Original comment by doug.bu...@gmail.com on 20 Jan 2012 at 6:39

  • Added labels: ****
  • Removed labels: ****
Updated /etc/cron.d/snorby-db-fix:

# Disabled 20120119 due to performance issues with larger databases
# */5 * * * * root  /usr/local/bin/snorby-db-fix >> 
/var/log/nsm/snorby-db-fix.log

Original comment by doug.bu...@gmail.com on 20 Jan 2012 at 6:39

  • Added labels: ****
  • Removed labels: ****
@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

Packaged:
/usr/local/lib/ruby/gems/1.9.1/gems/fpm-0.3.11/bin/fpm -s dir -t deb -n 
securityonion-snorby-db-fix -v 20120119 /usr/local/bin/snorby-db-fix 
/etc/cron.d/snorby-db-fix

Original comment by doug.bu...@gmail.com on 20 Jan 2012 at 6:39

  • Added labels: ****
  • Removed labels: ****
Packaged:
/usr/local/lib/ruby/gems/1.9.1/gems/fpm-0.3.11/bin/fpm -s dir -t deb -n 
securityonion-snorby-db-fix -v 20120119 /usr/local/bin/snorby-db-fix 
/etc/cron.d/snorby-db-fix

Original comment by doug.bu...@gmail.com on 20 Jan 2012 at 6:39

  • Added labels: ****
  • Removed labels: ****
@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

Added the following to security-onion-upgrade.sh:


sed -i 's| |=|g' $CONF
source $CONF
if [ "$VERSION" = "20120116" ]; then
        NEW="20120119"
        echo "**********************************************"   | $LOGGER
        echo "* Upgrading from $VERSION to $NEW."               | $LOGGER
        echo "**********************************************"   | $LOGGER
        DIR="/nsm/backup/$NEW"
        mkdir -p $DIR                                           | $LOGGER
        cd $DIR

    echo "* Disabling Snorby DB fix cronjob due to performance issues with large/busy DBs" | $LOG
GER
    FILE=securityonion-snorby-db-fix_"$NEW"_i386.deb
    wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
    dpkg -i $FILE                       | $LOGGER

    echo "* Installing new Setup script"            | $LOGGER
    FILE=securityonion-setup_"$NEW"_i386.deb
    wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
    dpkg -i $FILE                       | $LOGGER

    echo "* Installing new Suricata"                        | $LOGGER
        FILE=securityonion-suricata_"$NEW"-1_i386.deb
        wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
        dpkg -i $FILE                                           | $LOGGER

        echo "* Installing new Suricata config"                 | $LOGGER
        FILE=securityonion-suricata-config_"$NEW"_i386.deb
        wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
        dpkg -i $FILE                                           | $LOGGER

        echo "* Installing new Suricata-specific rules"         | $LOGGER
        FILE=securityonion-suricata-rules_"$NEW"_i386.deb
        wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
        dpkg -i $FILE                                           | $LOGGER

    apt-get -y remove securityonion-pulledpork-update
        echo "* Installing new PulledPork config"            | $LOGGER
        FILE=securityonion-pulledpork_"$NEW"_i386.deb
        wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
        dpkg -i $FILE                                           | $LOGGER

    cp /etc/crontab .
    sed -i '/pulledpork_update.sh/d' /etc/crontab
    service cron restart                    | $LOGGER

        echo "* Copying suricata.yaml to sensor directories"    | $LOGGER
        grep -v "^#" /etc/nsm/sensortab |awk '{print $1}' |while read SENSOR
        do
                mkdir -p $DIR/"$SENSOR"/
                cp /etc/nsm/"$SENSOR"/suricata.yaml $DIR/"$SENSOR"/
                cp /etc/suricata/suricata.yaml /etc/nsm/"$SENSOR"/
        sed -i "s|threshold-file: threshold.conf|threshold-file: /etc/nsm/$SENSOR/threshold.c
onf|g" /etc/nsm/"$SENSOR"/suricata.yaml
        done

    FILE=/etc/pulledpork/pulledpork.conf
    if [ -f $FILE ]
    then
        sed -i 's|local_rules=/etc/nsm/rules/local.rules,/etc/nsm/rules/decoder-events.rules,
/etc/nsm/rules/stream-events.rules|local_rules=/etc/nsm/rules/local.rules,/etc/n
sm/rules/decoder-even
ts.rules,/etc/nsm/rules/stream-events.rules,/etc/nsm/rules/http-events.rules,/et
c/nsm/rules/smtp-even
ts.rules|g' $FILE
        sed -i 's|rule_url=http://rules.emergingthreats.net|rule_url=https://rules.emergingth
reatspro.com|g' $FILE
        sed -i 's|distro=FreeBSD-8.0|distro=Ubuntu-10-4|g' $FILE
        /usr/local/bin/pulledpork_update.sh
    fi

        sed -i "s|VERSION=$VERSION|VERSION=$NEW|g" $CONF        | $LOGGER
        echo "* Upgrade to $NEW complete."                      | $LOGGER
        echo 
fi

Original comment by doug.bu...@gmail.com on 22 Jan 2012 at 5:37

  • Added labels: ****
  • Removed labels: ****
Added the following to security-onion-upgrade.sh:


sed -i 's| |=|g' $CONF
source $CONF
if [ "$VERSION" = "20120116" ]; then
        NEW="20120119"
        echo "**********************************************"   | $LOGGER
        echo "* Upgrading from $VERSION to $NEW."               | $LOGGER
        echo "**********************************************"   | $LOGGER
        DIR="/nsm/backup/$NEW"
        mkdir -p $DIR                                           | $LOGGER
        cd $DIR

    echo "* Disabling Snorby DB fix cronjob due to performance issues with large/busy DBs" | $LOG
GER
    FILE=securityonion-snorby-db-fix_"$NEW"_i386.deb
    wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
    dpkg -i $FILE                       | $LOGGER

    echo "* Installing new Setup script"            | $LOGGER
    FILE=securityonion-setup_"$NEW"_i386.deb
    wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
    dpkg -i $FILE                       | $LOGGER

    echo "* Installing new Suricata"                        | $LOGGER
        FILE=securityonion-suricata_"$NEW"-1_i386.deb
        wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
        dpkg -i $FILE                                           | $LOGGER

        echo "* Installing new Suricata config"                 | $LOGGER
        FILE=securityonion-suricata-config_"$NEW"_i386.deb
        wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
        dpkg -i $FILE                                           | $LOGGER

        echo "* Installing new Suricata-specific rules"         | $LOGGER
        FILE=securityonion-suricata-rules_"$NEW"_i386.deb
        wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
        dpkg -i $FILE                                           | $LOGGER

    apt-get -y remove securityonion-pulledpork-update
        echo "* Installing new PulledPork config"            | $LOGGER
        FILE=securityonion-pulledpork_"$NEW"_i386.deb
        wget -q http://sourceforge.net/projects/security-onion/files/$NEW/$FILE -O $FILE | $LOGGER
        dpkg -i $FILE                                           | $LOGGER

    cp /etc/crontab .
    sed -i '/pulledpork_update.sh/d' /etc/crontab
    service cron restart                    | $LOGGER

        echo "* Copying suricata.yaml to sensor directories"    | $LOGGER
        grep -v "^#" /etc/nsm/sensortab |awk '{print $1}' |while read SENSOR
        do
                mkdir -p $DIR/"$SENSOR"/
                cp /etc/nsm/"$SENSOR"/suricata.yaml $DIR/"$SENSOR"/
                cp /etc/suricata/suricata.yaml /etc/nsm/"$SENSOR"/
        sed -i "s|threshold-file: threshold.conf|threshold-file: /etc/nsm/$SENSOR/threshold.c
onf|g" /etc/nsm/"$SENSOR"/suricata.yaml
        done

    FILE=/etc/pulledpork/pulledpork.conf
    if [ -f $FILE ]
    then
        sed -i 's|local_rules=/etc/nsm/rules/local.rules,/etc/nsm/rules/decoder-events.rules,
/etc/nsm/rules/stream-events.rules|local_rules=/etc/nsm/rules/local.rules,/etc/n
sm/rules/decoder-even
ts.rules,/etc/nsm/rules/stream-events.rules,/etc/nsm/rules/http-events.rules,/et
c/nsm/rules/smtp-even
ts.rules|g' $FILE
        sed -i 's|rule_url=http://rules.emergingthreats.net|rule_url=https://rules.emergingth
reatspro.com|g' $FILE
        sed -i 's|distro=FreeBSD-8.0|distro=Ubuntu-10-4|g' $FILE
        /usr/local/bin/pulledpork_update.sh
    fi

        sed -i "s|VERSION=$VERSION|VERSION=$NEW|g" $CONF        | $LOGGER
        echo "* Upgrade to $NEW complete."                      | $LOGGER
        echo 
fi

Original comment by doug.bu...@gmail.com on 22 Jan 2012 at 5:37

  • Added labels: ****
  • Removed labels: ****
@GoogleCodeExporter

This comment has been minimized.

Show comment
Hide comment
@GoogleCodeExporter

GoogleCodeExporter Mar 24, 2015

Published:
http://securityonion.blogspot.com/2012/01/security-onion-20120119-now-available.
html

Original comment by doug.bu...@gmail.com on 22 Jan 2012 at 5:38

  • Changed state: Verified
  • Added labels: ****
  • Removed labels: ****
Published:
http://securityonion.blogspot.com/2012/01/security-onion-20120119-now-available.
html

Original comment by doug.bu...@gmail.com on 22 Jan 2012 at 5:38

  • Changed state: Verified
  • Added labels: ****
  • Removed labels: ****
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment