Installation

Fedele Mantuano edited this page Jan 5, 2017 · 6 revisions
Clone this wiki locally

System packages

This is a complete guide for SpamScope installation.

The first step is install all requirements and tools that SpamScope uses to analyze every mail.

For Debian system:

apt-get -yqq update
apt-get -yqq install cmake libffi-dev libfuzzy-dev libxml2-dev libxslt1-dev libyaml-dev p7zip-full python-dev python-pip unrar-free unzip zlib1g-dev

SpamScope automatically extracts attachments from mail and unzip (unrar, ...) the archives, so it uses unrar-free, unzip, p7zip-full.

Leiningen

SpamScope uses streamparse project, that uses Leiningen to use Clojure.

With Clojure you can define Apache Storm topologies.

wget https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein -O /opt/lein
chmod 755 /opt/lein
ln -s /opt/lein /usr/local/bin/lein
lein version

Faup

Faup stands for Finally An Url Parser and is a library and command line tool to parse URLs and normalize fields.

wget https://github.com/stricaud/faup/archive/master.tar.gz -O /opt/faup.tar.gz
tar zxf /opt/faup.tar.gz -C /opt
rm -rf /opt/faup.tar.gz
mkdir -p /opt/faup-master/build
cd /opt/faup-master/build
cmake ..
make
make install
echo '/usr/local/lib' | tee -a /etc/ld.so.conf.d/faup.conf
ldconfig
cd /opt/faup-master/src/lib/bindings/python
python setup.py install

Tika App (optional)

SpamScope can use Tika App to parse every attachment mail.

The Apache Tika toolkit detects and extracts metadata and text from over a thousand different file types (such as PPT, XLS, and PDF).

wget https://archive.apache.org/dist/tika/tika-app-1.13.jar -O /opt/tika-app-1.13.jar

To use Tika App JAR with Python you should use tika-app wrapper.

Thug (optional)

If you want to analyze the attachments with Thug, follow these instructions to install it.

VirusTotal (optional)

It's possible add to results (for mail attachments) VirusTotal report. You need a private API key.

Redis (optional)

It's possible to store the results in Redis. In this case you should install redis package.

Last step: SpamScope

Clone repository:

git clone https://github.com/SpamScope/spamscope.git

Install requirements in file requirements.txt with python-pip:

pip install -r requirements.txt

I suggest you to use virtualenv.