Browse files

Updated README

  • Loading branch information...
1 parent c01fe9f commit 40c14609d0eff880247caf6744d578751d66c524 @pjlantz committed Aug 8, 2010
Showing with 59 additions and 37 deletions.
  1. +59 −37 README
View
96 README
@@ -1,12 +1,12 @@
-Contents
+Contents:
===========
-[1] INSTALL
-[2] SETUP
-[3] USAGE
-[4] DEV
+[1] - INSTALL
+[2] - SETUP
+[3] - USAGE
+[4] - DEVELOPMENT
-[1] INSTALL
+[1] - INSTALL
===========
Hale has the following dependencies:
@@ -27,36 +27,58 @@ httplib2 == 0.6.0
Additionally the monitor requires a database backend driver corresponding to the database used by django. When these libraries are installed download the source from here and extract it anywhere.
-[2] SETUP
+[2] - SETUP
=========
-1) First create a database that will be used by Hale, the database engine can be any of your choice like MySQL, PostgreSQL etc.
+1) First create a database that will be used by Hale, the database engine can be any of your choice. If you are using an existing database then skip this step.
-2) Next step is to install python database backend drivers corresponding to the one used by server engine.
+2) Next step is to install python database backend drivers corresponding to the one used by the server engine.
-3) Edit settings.py in hale/src/webdb/ and set the following configurations in DATABASES: ENGINE, NAME, USER, PASSWORD, HOST, PORT where the engine setting is for example django.db.backends.mysql if your server engine is MySQL. The name setting is the name of your database used when creating it.
+3) Edit settings.py in hale/src/webdb/ and edit the following configurations: ENGINE, NAME, USER, PASSWORD, HOST and PORT where the engine setting is for example django.db.backends.mysql if your server engine is MySQL. The name setting is the name of your database used when creating it.
-4) In the webdb directory run the following command: python manage.py syncdb. If you get any errors here its most likely that the database settings in settings.py are incorrect. Also set the superuser that will be used when administrating the users.
+4) If you dont want to start your own web ui then skip this step and go to 8). In the webdb directory run the following command: 'python manage.py syncdb'. If you get any errors here its most likely that the database settings in settings.py are incorrect. Also, during the sync set the superuser that will be used when administrating the users.
-5) Run python manage.py rebuild_index to let the search engine index first time. This search engine is used by the web ui to find queries.
+5) Run 'python manage.py rebuild_index' to let the search engine index first time. After this you run update_index instead and should put this as a cron job to update indexes in a regular interval.
-6) Run python manage.py runserver and head to http://127.0.0.1:8000 to check if setup was correctly done. Then go to to http://127.0.0.1/admin and login with your superuser account created before. Create some users if you wish so and then add your proxies. If no proxies are specified then the monitor will connect directly to the botnet.
+6) Run 'python manage.py runserver' and head to http://127.0.0.1:8000 to check if setup was correctly done. Then go to to http://127.0.0.1/admin and login with your superuser account created before. Create some users if you wish so and then add your proxies. If no proxies are specified then the monitor will connect directly to the botnets and URLs.
-7) The runserver command deploys a development server that is not recommended to use public since performance issues arise. Instead deploy the web ui with a web server of your choice. TODO: describe deploying
+7) The runserver command deploys a development server that is not recommended for public use since performance issues arise. Instead deploy the web ui with a web server of your choice as described here: http://www.djangobook.com/en/beta/chapter21/ for use with Apache.
-8) Before running the monitor edit hale.conf in hale/src/conf/ if you wish to use a XMPP server. If not then skip this step. To activate XMPP bot either edit login info to an existing account and server or start your own XMPP server. An important step when starting up a XMPP server is to increase the max stanza size from the default value to something like 10Mb. Otherwise malware sharing will not be possible. The channel settings in hale.conf are used for setting the share grouproom used by the bot and the coord setting is used for the grouproom where all coordination between sensors is done.
+8) Upload modules that will be used from hale/src/modules/ or write your own (see Development section). Upload the desired module in the admin interface and edit for example the module name to 'irc' and the filename to 'ircModule.py'. If you want others to see how to configure this module then copy the corresponding section config located in hale/conf/modules.conf and put it in the textbox, also add the uniqueKeys sections for the module being uploaded.
-[3] USAGE
-=========
-To start the monitor head to hale/src/ and execute python main.py. If it fires up with errors then the django settings.py file is not correctly set or some libraries are missing. When the monitor is running type help or ? to get the available commands. Type help command to get more info about the specific command. Starting up a monitor bot is done by first editing the hale/src/conf/modules.conf file, for example using a irc configuration as follow:
-
-[ircConf] module = irc botnet = irc.freenode.net port = 6667 password = None nick = testjg5534 username = agent007 realname = Spying channel = #testasdf channel_pass = kluczbork pass_grammar = PASS nick_grammar = NICK user_grammar = USER join_grammar = JOIN version_grammar = VERSION time_grammar = TIME privmsg_grammar = PRIVMSG topic_grammar = TOPIC currenttopic_grammar = 332 ping_grammar = PING pong_grammar = PONG
-
-Edit or create a new config by specifying a new uniquely named section ([ircConf] part). At the top of the conf file there is a section called uniqueKeys where all unique fields for a module are specified and used to generate the botnet hash, this should usually not be changed to preserve correct botnet tracking. When this is done run useconf section to load the configuration and then fire up the bot with exec modulename id where id is set by you to identify the botnet.
+9) Before running the monitor edit hale.conf in hale/src/conf/ if you wish to use a XMPP server. If not then skip this step. To activate XMPP bot set use setting to True and either edit login info to an existing account and server or start your own XMPP server. An important step when starting up a XMPP server is to increase the max stanza size from the default value to something like 10Mb. Otherwise malware sharing will not be possible. The channel settings in hale.conf are used for the share grouproom used by the bot and the coord setting is used for the grouproom where all coordination between sensors is done.
-The web interface provides access to all captured data in the sensor network. There is also a search function which enables the user to search for botnet and file hashes, related IP numbers, botnet IDs, botnet modules used and botnet hosts.
-
-[4] DEV
-=======
+[3] - USAGE
+=========
+To start the monitor head to hale/src/ and execute python main.py. If it fires up with errors then the django settings.py file is not correctly set or some libraries are missing. When the monitor is running type 'help' or '?' to get the available commands. Type help command to get more info about the specific command. Starting up a monitor bot is done by first editing the hale/src/conf/modules.conf file, for example using a irc configuration as follow:
+
+[ircConf]
+module = irc
+botnet = irc.freenode.net
+port = 6667
+password = None
+nick = testjg5534
+username = agent007
+realname = Spying
+channel = #testasdf
+channel_pass = kluczbork
+pass_grammar = PASS
+nick_grammar = NICK
+user_grammar = USER
+join_grammar = JOIN
+version_grammar = VERSION
+time_grammar = TIME
+privmsg_grammar = PRIVMSG
+topic_grammar = TOPIC
+currenttopic_grammar = 332
+ping_grammar = PING
+pong_grammar = PONG
+
+Edit or create a new config by specifying a new uniquely named section ([ircConf] part). At the top of the config file there is a section called uniqueKeys where all unique fields for a module are specified and used to generate the botnet hash, this should usually not be changed to preserve correct botnet tracking. When this is done run useconf section to load the configuration and then fire up the bot with exec modulename id where id is set by you to identify the botnet.
+
+The web interface provides access to all captured data in the database which is accessible from the index page. There is also a search function which enables the user to search for botnet and file hashes, related IP numbers, botnet IDs, botnet modules used and botnet hosts. If the user got access to edit proxies or modules then this can be done in the admin section, url to this is http://.../admin. The administrator can set user modes and also add consumers for the web API.
+
+[4] - DEVELOPMENT
+===============
How to add modules, the current module API work as follow:
1) Implement module, for example:
@@ -97,7 +119,7 @@ Add decorator for the register function (in this case module_setup) which will b
Also follow the naming convention nameModule.py and @moduleManager.register("name") and import the moduleManager, if not the moduleManager will notify you about any errors.
-Catch all possible exceptions and send them to the module coordinator error bucket which takes care of all errors from every module running in the monitor system. These errors can be shown with the 'showlog' command in the CLI.
+The rest of the module code is omitted but should create a twisted factory object and start this with the reactor.
2) Drag the file to the modules directory. The moduleManager will then automatically import it and check for errors.
@@ -119,7 +141,7 @@ channel = #irc
...
etc.
-4) Upload the module to the web ui by setting the module name to for exampel irc, filename ircModule.py and then add an config example for this module.
+4) Upload the module to the web ui by setting the module name to for example irc, filename ircModule.py and then add a config example for this module.
Feeder bot HOWTO:
@@ -157,14 +179,14 @@ RESTful Web API:
To get access to the api you need a consumer key and secret key, this can be created by the admin and are used with OAuth to authenticate. The following URLs are available to fetch data in JSON format:
- /api/botnet will reply with all botnets monitored
- /api/botnet/botnethash will reply with the botnet with hash equal to botnethash
- /api/host/hostname will reply with all botnets monitored with host equal to hostname
- /api/type/module will reply with all botnets monitored with the module
- /api/botips/hash will reply with all ips captured by botnet with the value hash
- /api/bologs/hash will reply with all logs for botnet with value hash
- /api/bofiles/hash will reply with file hashes captured by botnet with value hash
- /api/file/hash returns botnet(s) info for those that have captured file with the hash specified
- /api/ip/addr will reply with botnet(s) info for those that have detected an IP with number addr
+ http://.../api/botnet will reply with all botnets monitored
+ http://.../api/botnet/botnethash will reply with the botnet with hash equal to botnethash
+ http://.../api/host/hostname will reply with all botnets monitored with host equal to hostname
+ http://.../api/type/module will reply with all botnets monitored with the module
+ http://.../api/botips/hash will reply with all ips captured by botnet with the value hash
+ http://.../api/bologs/hash will reply with all logs for botnet with value hash
+ http://.../api/bofiles/hash will reply with file hashes captured by botnet with value hash
+ http://.../api/file/hash returns botnet(s) info for those that have captured file with the hash specified
+ http://.../api/ip/addr will reply with botnet(s) info for those that have detected an IP with number addr
Note that currently only GET operations are possible.

0 comments on commit 40c1460

Please sign in to comment.