Addded license banner

Improved init scripts
Improved README

README.md

@@ -1,48 +1,141 @@
 # Dorothy2
 
-A botnet analysis framework.
+A malware/botnet analysis framework written in Ruby.
 
 
 ##Requirements
 
-WARNING:
-The current version of Dorothy, is based on VMWare ESX5. ESXi is not supported due to its limitations in using the VMWare API.
-However, the overall framework could be easily customized in order to use another virtualization engine. Dorothy2 is very modular,
-and any customization or modification is very welcome.
+>WARNING:
+The current version of Dorothy, is based on VMWare ESX5. ESXi is not supported due to its limitations in using the
+VMWare API.
+However, the overall framework could be easily customized in order to use another virtualization engine. Dorothy2 is
+very modular,and any customization or modification is very welcome.
 
+Dorothy needs the following software (not expressly in the same host) in order to be executed:
 
--VMWare ESX 5.0
--Ruby 1.8.7
--At least one WindowsXP virtual machine
--One unix-like machine dedicated to the Network Analysis Engine (tcpdump/ssh needed)
+* VMWare ESX >= 5.0  (tip: if you download ESXi, you can evaluate ESX for 30 days)
+* Ruby 1.8.7
+* Postgres >= 9.0
+* At least one Windows virtual machine
+* One unix-like machine dedicated to the Network Analysis Engine(NAM) (tcpdump/ssh needed)
+* [pcapr-local](https://github.com/mudynamics/pcapr-local )  (only used by doroParser)
+* MaxMind libraries (only used by doroParser)
 
 
 ## Installation
 
-Add this line to your application's Gemfile:
+It is raccommended to follow this step2step process:
 
-    gem 'dorothy2'
+1. Set your ESX environment
+2. Install Postgres >= 9.0
+3. Install Dorothy and libmagic libraries
+4. Start Dorothy, and configure it
+5. Use Dorothy
 
-And then execute:
+### 1. Set your ESX environment
+1. Basic configuration (ssh)
+ * From vSphere:
 
-    $ bundle
+            Configuration->Security Profile->Services->Proprieties->SSH->Options->Start and Stop with host->Start->OK
 
-Or install it yourself as:
+2. Configure the Windows VMs used for sandboxing
+ * Create a test_ping.bat file into C:\ folder, with the following content:
 
-    $ gem install dorothy2
+            ping -n 1 google.com
+>This file will be used for checking if the VM has internet access. You can substitute "google.com" with whatever host you like. Just a suggestion: use hostnames instead of IP addresses. The aim of this test doesn't care if the DNS is not resolving, or the IP addresses is unreachable. It cares only if *everything* works.
 
+ * Disable Windows firewall (preferred)
+ * VMWare Tools must be installed in the Guest system.
+3. Configure the unix VM used by the NAM
+     * Install tcpdump and sudo
 
-Install libmagic :
-    $ brew install libmagic
-    $ brew link libmagic
+                #apt-get install tcpdump sudo
 
+     * Create a dedicated user for dorothy (e.g. "dorothy")
 
-VMWare Tools must be installed in the Guest system.
+                #useradd dorothy
+     * Add dorothy's user permission to execute/kill tcpdump to the sudoers file:
+
+                #visudo
+                add the following line:
+                dorothy  ALL = NOPASSWD: /usr/sbin/tcpdump, /bin/kill
+
+     * Add the pubblic key of the user who will execute Dorothy in /home/dorothy/.ssh/authorized_keys
+
+     > Consider that you are going to execute Dorothy on your machine, and that HOST2 is the NAM. In order to access
+     > to NAM in an automatic mode, Dorothy needs to authenticate to HOST2's ssh service through its public key in order
+     > to avoid interactive authentication.
+
+
+### 2. Install Postgres >= 9.0
+1. Install Postgress
+
+        $sudo apt-get install pg
+or
+
+        http://www.postgresql.org/download/
+
+2. Configure a dedicated user for Dorothy (or use root user instead, up to you :)
+
+Add a user dedicated to dorothy (or use the root one, up to you :)
+
+### 3. Install Dorothy and libmagic libraries
+
+1.    Install Dorothy gem
+
+        $ gem install dorothy2
+2. Install libmagic ruby libraries
+
+        $ brew install libmagic
+        $ brew link libmagic
+
+### 4. Start Dorothy, and configure it!
+
+0. Install MaxMind libraries
+    * [GeoLiteCity](http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz)
+    * [GeoLite ASN](http://download.maxmind.com/download/geoip/database/asnum/GeoIPASNum.dat.gz)
+    * Copy GeoLiteCity.dat and GeoIPASNum.dat into Dorothy's etc/geo/ folder
+
+1. Start Dorothy
+
+        $ dorothy_start -v
+The following message should appear
+
+        [WARNING] It seems that the Dorothy configuration file is not present,
+        please answer to the following question in order to create it now.
+
+2. Follow the instruction to configure
+    * The environment variables (db, esx server, etc)
+    * The Dorothy sources (where to get new binaries)
+    * The ESX Virtual machines used for the analysis
+
+The first time you execute Dorothy, it will ask you to fill those information in order to create the required configuration files into the etc/ folder. However, you are free to modify/create such files directly - configuration example files can be found there too.
+
+###5. Use Dorothy
+1. Copy a .exe or .bat file into $yourdorothyhome/opt/bins/manual/
+2. Execute dorothy with the malwarefolder source type (if you left the default one)
+
+    $ dorothy_start -v -s malwarefolder
 
 
 ## Usage
 
+	Usage:
+	$./dorothy_start [options]
+	where [options] are:
+       --verbose, -v:   Enable verbose mode
+      --infoflow, -i:   Print the analysis flow
+    --source, -s <s>:   Choose a source (from the ones defined in etc/sources.yml)
+        --daemon, -d:   Stay in the backround, by constantly pooling datasources
+ --SandboxUpdate, -S:   Update Dorothive with the new Sandbox file
+ --DorothiveInit, -D:   (RE)Install the Dorothy Database (Dorothive)
+          --help, -h:   Show this message
+
+
+ >Example
 
+    ./dorothy_start -v -s malwarefolder
+    ./dorothy_stop
 
 ------------------------------------------
 

bin/dorothy_start

@@ -1,34 +1,43 @@
 #!/usr/bin/env ruby
 
+# Copyright (C) 2013 marco riccardi.
+# This file is part of Dorothy - http://www.honeynet.it/dorothy
+# See the file 'LICENSE' for copying permission.
+
 require 'rubygems'
 require 'trollop'
 require 'dorothy2'          #comment for testing/developmnet
 
 #load '../lib/dorothy2.rb'  #uncomment for testing/developmnet
 
-include DoroEnv
 include Dorothy
 
 
 opts = Trollop.options do
   banner <<-EOS
 
-	The Dorothy Malware Analysis Framework 2.0
+   ####################################################
+   ##                                                ##
+   ##  The Dorothy Malware Analysis Framework 2.0    ##
+   ##                                                ##
+   ####################################################
 
-  marco.riccardi@
-	www.honeynet.it
+        marco.riccardi@honeynet.it
+	www.honeynet.it/dorothy
 
 
 	Usage:
-	Manager.rb [options]
+  dorothy_start [options]
 	where [options] are:
   EOS
 
 
   opt :verbose, "Enable verbose mode"
   opt :infoflow, "Print the analysis flow"
-  opt :source, "Choose a source (manual|honeypot|ztracker)", :type => :string
+  opt :source, "Choose a source (from the ones defined in etc/sources.yml)", :type => :string
   opt :daemon, "Stay in the backround, by constantly pooling datasources"
+  opt :SandboxUpdate, "Update Dorothive with the new Sandbox file"
+  opt :DorothiveInit, "(RE)Install the Dorothy Database (Dorothive)"
 
 end
 
@@ -41,23 +50,34 @@ if opts[:infoflow]
 	#2) Copy File to VM
 	#3) Start Sniffer
 	#4) Execute file into VM
-	#5) Make screenshop
+	#5) Make screenshot
 	#6) Wait X minutes (configure X in the conf file)
 	#7) Stop Sniffer
 	#8) Download Screenshot and trafficdump
 	#9) Try to retreive malware info from VirusTotal
-	#10) Insert data to Dorothy-DB
+	#10) Insert data into Dorothy-DB
 	------------------------------------------
 	"
   exit(0)
 end
 
+puts "
+
+   ####################################################
+   ##                                                ##
+   ##  The Dorothy Malware Analysis Framework 2.0    ##
+   ##                                                ##
+   ####################################################
+
+"
+
 #VARS
+HOME = File.expand_path("..",File.dirname(__FILE__))
 VERBOSE = (opts[:verbose] ? true : false)
 daemon = (opts[:daemon] ? true : false)
 
 #DEFAULT CONF FILES
-conf = '../etc/dorothy.yml'
+conf = HOME + '/etc/dorothy.yml'
 
 #LOAD ENV
 Util.exists?(conf) ? DoroSettings.load!(conf) : DoroConfig.create
@@ -71,12 +91,38 @@ LOGGER = DoroLogger.new(logout, DoroSettings.env[:logage])
 LOGGER.sev_threshold = DoroSettings.env[:loglevel]
 
 #INIT DB Connector
-db = Insertdb.new
+begin
+  db = Insertdb.new
+rescue => e
+  if e.inspect =~ /exist/
+    puts "WARNING".yellow + " The database doesn't exist yet. Press Enter to load the ddl into the DB"
+    gets
+    Util.init_db(true)
+    exit(0)
+  else
+    puts "ERROR".red + " Can't connect to the database"
+    puts e
+    exit(0)
+  end
+end
+
+
+if opts[:DorothiveInit]
+  Util.init_db
+  exit(0)
+end
+
+if opts[:SandboxUpdate]
+  puts "[Dorothy]".yellow + " Loading #{sboxfile} into Dorothive"
+  DoroConfig.init_sandbox(sboxfile)
+  puts "[Dorothy]".yellow + " Done."
+  exit(0)
+end
 
 if Util.exists?(sfile)
   sources = YAML.load_file(sfile)
 else
-  puts "[WARNING]".red + " A source file doesn't exist, please crate one in the /etc folder" #TODO ..or specify another one in the args -S
+  puts "[WARNING]".red + " A source file doesn't exist, please crate one in the /etc folder"
   exit(0)
 end
 

bin/dorothy_stop

@@ -1,13 +1,17 @@
 #!/usr/bin/env ruby
 
+# Copyright (C) 2013 marco riccardi.
+# This file is part of Dorothy - http://www.honeynet.it/dorothy
+# See the file 'LICENSE' for copying permission.
+
+
+
 require 'rubygems'
 require 'trollop'
 require 'dorothy2'
 
 #load '../lib/dorothy2.rb'
 
-
-include DoroEnv
 include Dorothy
 
 

bin/dparser_start

@@ -1,12 +1,15 @@
 #!/usr/bin/env ruby
 
-#libdir = File.dirname(__FILE__) + "./lib"
-#libdir = File.expand_path(libdir)
-#$: << libdir
+# Copyright (C) 2013 marco riccardi.
+# This file is part of Dorothy - http://www.honeynet.it/dorothy
+# See the file 'LICENSE' for copying permission.
 
 require 'rubygems'
-#require 'trollop'
-require '../lib/doroParser'
+require 'trollop'
+require 'dorothy2'
+require 'doroParser'
+
+#load '../lib/doroParser'
 
 include DoroEnv
 include DoroParser

bin/dparser_stop

@@ -1,11 +1,17 @@
 #!/usr/bin/env ruby
 
+# Copyright (C) 2013 marco riccardi.
+# This file is part of Dorothy - http://www.honeynet.it/dorothy
+# See the file 'LICENSE' for copying permission.
+
 
 require 'rubygems'
 require 'trollop'
-require '../lib/doroParser'
+require 'dorothy2'
+require 'doroParser'
+
+#load '../lib/doroParser'
 
-include DoroEnv
 include DoroParser
 
 

etc/ddl/dorothive.ddl

@@ -1520,15 +1520,6 @@ ALTER TABLE ONLY sandboxes
 ALTER TABLE ONLY sensors
     ADD CONSTRAINT sensors_pkey PRIMARY KEY (id);
 
-
---
--- Name: sightings_pk; Type: CONSTRAINT; Schema: dorothy; Owner: postgres; Tablespace: 
---
-
-ALTER TABLE ONLY sightings
-    ADD CONSTRAINT sightings_pk PRIMARY KEY (sample, sensor, date);
-
-
 --
 -- Name: traffic_dumps_pkey; Type: CONSTRAINT; Schema: dorothy; Owner: postgres; Tablespace: 
 --