Updated documentation

mushorg · Aug 17, 2020 · 8866850 · 8866850
1 parent a51b1be
commit 8866850
Show file tree

Hide file tree

Showing 8 changed files with 186 additions and 58 deletions.
diff --git a/docs/source/api.rst b/docs/source/api.rst
@@ -7,51 +7,95 @@ where, ``API_KEY`` is a JWT-token created by a particular tanner-api, which can
 ``API_KEY for full access: 'API_KEY'``
 
 How to create an API_KEY with desired signature?
-~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 * By default tanner's API_KEYs use the signature: 'tanner_api_auth'
 * This signature is veryfied on all the API requests.
 * It is highly recommended that every tanner-owner set their own signature.
 * This can be done by modifying tanner.config['API']['auth_signature'] to the desired one.
 
-/?key=API_KEY
-~~~~
+
+Using API Key
+~~~~~~~~~~~~~
+
+.. code-block::
+
+	/?key=API_KEY
+
 This is the index page which shows ``tanner api``.
 
-/snares
-~~~~~~~~~~
+List Snare instances
+~~~~~~~~~~~~~~~~~~~~
+.. code-block::
+
+	/snares
+
 This shows all the snares' uuid.
 
-/snare/<snare-uuid>?key=API_KEY
-~~~~~~~~~~~~~~~~~~~~~~
+Access single snare instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block::
+
+	/snare/<snare-uuid>?key=API_KEY
+
+
 Replace ``<snare-uuid>`` with a valid `snare-uuid` and it will show all the sessions related to that ``snare-uuid`` and their details.
 
-/snare-stats/<snare-uuid>?key=API_KEY
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Stats of a snare instance
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block::
+
+	/snare-stats/<snare-uuid>?key=API_KEY
+
 Replace ``<snare-uuid>`` with a valid `snare-uuid` and it will show some stats.
 
 	* No of sessions in the sanre
 	* Total duration for which snare remains active
 	* Attack frequency, which shows no of sessions which face different attacks.
 
+List all sessions
+~~~~~~~~~~~~~~~~~
+
+.. code-block::
+
+	/<snare-uuid>/<sessions-id>/?API_KEY
+
+This gives you a list of all the sessions UUID that are present on a give snare instance.
+
+
+Get single session
+~~~~~~~~~~~~~~~~~~~
+.. code-block::
+
+	/session/<sess-uuid>?key=API_KEY
+
+It gives all information about the session with given uuid.
+
+Get all information about all the sessions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 /<snare-uuid>/sessions?filters=<filters>&key=API_KEY
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 This shows all the sessions' uuid which follow the filters.
-Filters are sepatated by ``white-space`` and name-value pair are separated by ``:``. E.g ``?filters=filter1:value1 filter2:value2``.
+Filters are sepatated by ``white-space`` and name-value pair are separated by ``:``. 
 
-It supports 5 filters:
+E.g ``?filters=filter1:value1 filter2:value2``.
 
-	* **peer_ip** -- Sessions with given ip. E.g ``peer_ip:127.0.0.1 ``
-	* **user-agent** -- Sessions with given user-agent. E.g ``user-agent:Chrome``
-	* **attack_types** -- Sessions with given attack type such as lfi, rfi, xss, cmd_exec, sqli. E.g ``attack_types:lfi``
-	* **possible_owners** -- Sessions with given owner type such as user, tool, crawler, attacker. E.g ``possible_owners:attacker``
-	* **start_time** -- Sessions which started after `start_time`. E.g ``start_time:1480560``
-	* **end_time** -- Sessions which ended before `end_time`. E.g ``end_time:1480560``
+It supports 5 filters:
 
-Multiple filters can be applied as ``peer_ip:127.0.0.1 start_time:1480560 possible_owners:attacker``
+* **peer_ip** -- Sessions with given ip. 
+	E.g  ``peer_ip:127.0.0.1``
+* **user_agent** -- Sessions with given user-agent. 
+	E.g ``user_agent:Chrome``
+* **attack_type** -- Sessions with given attack type such as lfi, rfi, xss, cmd_exec, sqli. 
+	E.g ``attack_types:lfi``
+* **owners** -- Sessions with given owner type such as user, tool, crawler, attacker. 
+	E.g ``owners:attacker``
+* **start_time** -- Sessions which started after `start_time`. 
+	E.g ``start_time:26-06-2020``
+* **end_time** -- Sessions which ended before `end_time`. 
+	E.g ``end_time:26-06-2020``
 
-/api/session/<sess-uuid>?key=API_KEY
-~~~~~~~~~~~~~~~~~~~~~~~~
-It gives all information about the session with given uuid.
+**Multiple filters** can be applied as ``peer_ip:127.0.0.1 start_time:26-06-2020 owners:attacker``
 
-External hyperlinks, like Python_.
-.. _Python: http://www.python.org/
diff --git a/docs/source/config.rst b/docs/source/config.rst
@@ -33,6 +33,18 @@ There are 8 different sections :
     :port: The port at which which redis is running
     :poolsize: The poolsize of redis server
     :timeout: The duration of timeout for redis server
+
+  * **POSTGRES**
+    # Configure postgres if it's running on some different port or network.
+
+    :host: The host address at which postgres is running
+    :port: The port at which which postgres is running
+    :poolsize: The poolsize of postgres server
+    :timeout: The duration of timeout for postgres server
+    :db_name: The name of the DB to which the connection is to be made. 
+    :user: The name of the user which have read & write access to the above mentioned DB.
+    :password: The password of the user using which the connection would be made.
+
   * **EMULATORS**
 
     :root_dir: The root directory for emulators that need data storing such as SQLI and LFI. Data will be stored in this directory
@@ -78,7 +90,7 @@ There are 8 different sections :
 
 If no file is specified, following YAML will be used as default:
 
-.. code-block:: python
+.. code-block:: yaml
 
   DATA:
     db_config: /opt/tanner/db/db_config.json
@@ -112,6 +124,15 @@ If no file is specified, following YAML will be used as default:
     port: 6379
     poolsize: 80
     timeout: 1
+  
+  POSTGRES:
+    host: localhost
+    port: 5432
+    poolsize: 80
+    timeout: 1
+    db_name: tanner
+    user: honeypot
+    password: honeynet
 
   EMULATORS:
     root_dir: /opt/tanner

diff --git a/docs/source/db_setup.rst b/docs/source/db_setup.rst
@@ -44,6 +44,7 @@ Data tokens
 ~~~~~~~~~~~
 
 Data tokens are used for filling the database with dummy data.
+
 There are 4 default tokens:
         * **I** -- integer id
         * **L** -- login/username
@@ -53,4 +54,4 @@ There are 4 default tokens:
 
 
 **Note**: TANNER uses the default linux wordlist (``/usr/share/dict/words``) for data.
-If you don't have the default wordlist in your system, install it or put it manually in ``/usr/share/dict``.
+If you don't have the default wordlist in your system, install it or put it manually in ``/usr/share/dict``.
diff --git a/docs/source/emulators.rst b/docs/source/emulators.rst
@@ -63,7 +63,9 @@ It emulates `SQL injection`_ vulnerability. This attack is detected by ``libinje
 The emulator copies the original database (see :doc:`db_setup` for more info about db) to a dummy database for every attacker.
 It uses UUID of the session for the attacker's db name. Every query is executed on the attacker's db.
 The emulator returns the result of the execution and the page where SNARE should show the result.
+
 It supports two types of DBs.
+
 * **SQLITE**
   To enable it, set SQLI type to SQLITE in config
 * **MySQL**

diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -14,6 +14,7 @@ Contents:
    quick-start
    parameters
    emulators
+   db_setup
    sessions
    storage
    dorks

diff --git a/docs/source/quick-start.rst b/docs/source/quick-start.rst
@@ -11,11 +11,11 @@ Basic concept
 * Serve dorks.
 * Adopt and change the responses.
 
-Setup Redis
-"""""""""""
+Setup Databases
+"""""""""""""""
+
+See :doc:`storage` page to know how to setup the Databases.
 
-#. Install the Redis: ``sudo apt-get install redis-server``
-#. Start it on ``localhost`` with default ``port``: ``redis-server``
 
 Setup PHP Sanbox
 """"""""""""""""

diff --git a/docs/source/storage.rst b/docs/source/storage.rst
@@ -1,11 +1,35 @@
-Storage
-=======
+Old Method
+==========
+Until Tanner v0.7 we were using Redis_ for storing all the data i.e analyzed and unanalyzed sessions. But the issue with that Redis was that it's a ``in-memory`` database meaning it consume large amount of RAM if large amount of data has to be stored. This usually resulted in the unexpected crash of the tanner server.
 
-We use Redis_ as main storage.
-TANNER connects to the redis with default values: ``host='localhost', port=6379``
 
-You should install and start the Redis on the server before using TANNER.
-See :doc:`quick-start`
+New Method
+==========
+
+To solve the problem described above we decided to use the combination of Postgres_ and Redis_ for storage purpose.
+
+
+**Setup Both database**
+
+* Installation
+
+    * Install Redis
+        #. On ubuntu: ``sudo apt-get install redis-server``
+
+    * Install Postgres
+        #. On ubuntu: ``sudo apt install postgresql``
+
+* Configuration
+
+Once both the server are installed and are running, make the required changes to configuration file which should be present in `/opt/tanner/config.yaml`. See [config] section for more detail about configuration file.
+
+Migrating from old setup to new setup
+=====================================
+
+If you are currently using the old setup for tanner then you'll have to migrate all your data to the postgresql and have a properly configured database setup. To make your work easy we have made a migration-script_
+
 
 .. _Redis: http://redis.io/
+.. _Postgres: https://www.postgresql.org/
+.. _migration-script: TODO ADD LINK HERE
 
diff --git a/docs/source/web.rst b/docs/source/web.rst
@@ -1,54 +1,89 @@
 Tanner WEB
 ==========
-Tanner WEB provides various stats related to traffic captured by snare in UI form. It can be accessed at ``locahost:8091/``.
+Tanner WEB provides various stats related to traffic captured by snare in UI form. It can be accessed at ``locahost:8091/``. 
+
+The web uses the Tanner API in the background to get the data.
+
+Index page
+~~~~~~~~~~
+.. code-block::
+
+	/ 
 
-/
-~~~~
 This is the index page which has a logo (mushorg) with ``Tanner web`` written below it.
 
 Below that we can find general info of the tanner instance:
 
 * **Tanner version** -- Which shows the version of tanner instance
 * **No. of snares connected** -- Which shows the number of snares connected to the tanner instance
 * **Latest session** -- Which will navigate you to the latest session's info page
+
 Below that we can find a clickable which states, ``Click here to navigate to snares list``, clicking which will move you to the ``/snares`` page.
 
-/snares
-~~~~~~~~~~ 
+List snare instances
+~~~~~~~~~~~~~~~~~~~~
+.. code-block::
+
+	/snares
+
 This shows all the snares' uuid. Each snare object is clickable. Clicking displays the page **/snare/<snare-uuid>**
 
-/snare/<snare-uuid>
-~~~~~~~~~~~~~~~~~~~~~~
+Information about single snare instance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+.. code-block::
+
+	/snare/<snare-uuid>
+
 Replace ``<snare-uuid>`` with a valid `snare-uuid` and it will provide two options:
 	* **Snare-Stats** -- It will move you to **/snare-stats/<snare-uuid>**
 	* **Sessions** -- It will move you to **/<snare-uuid>/sessions**
 
-/snare-stats/<snare-uuid>
+Stats of a snare instance
 ~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block::
+
+	/snare-stats/<snare-uuid>
+
 This page shows some general stats about the snare
 
 	* **No of Sessions** - Total no of sessions of the snare
 	* **Total Duration** - Total durations during which sessions remain active
 	* **Attack Frequency** - Frequency of different attacks made on the snare
 
-/<snare-uuid>/sessions
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+List all sessions
+~~~~~~~~~~~~~~~~~
+
+.. code-block::
+	
+	/<snare-uuid>/sessions
+
 This shows all the sessions' uuid. Each is clickable. Clicking displays **/session/<sess-uuid>**
 Filters can be on the sessions using the input box and clicking the ``Apply`` button.
 Filters are sepatated by ``white-space`` and name-value pair are separated by ``:``. E.g ``filter1:value1 filter2:value2``.
 
-It supports 6 filters:
-	* **peer_ip** -- Sessions with given ip. E.g ``peer_ip:127.0.0.1 ``
-	* **user-agent** -- Sessions with given user-agent. E.g ``user-agent:Chrome``
-	* **attack_types** -- Sessions with given attack type such as lfi, rfi, xss, cmd_exec, sqli. E.g ``attack_types:lfi``
-	* **possible_owners** -- Sessions with given owner type such as user, tool, crawler, attacker. E.g ``possible_owners:attacker``
-	* **start_time** -- Sessions which started after `start_time`. E.g ``start_time:1480560``
-	* **end_time** -- Sessions which ended before `end_time`. E.g ``end_time:1480560``
-	* **location** -- Sessions which have been done from the specified geometric `location`. It can take value of either country, city, country_code or zip_code. E.g ``location:India``, ``location:Mumbai``, ``location:US``, ``location:636005`` etc
+It supports 5 filters:
+
+* **peer_ip** -- Sessions with given ip. 
+	E.g ``peer_ip:127.0.0.1``
+* **user_agent** -- Sessions with given user-agent. 
+	E.g ``user-agent:Chrome``
+* **attack_types** -- Sessions with given attack type such as lfi, rfi, xss, cmd_exec, sqli. 
+	E.g ``attack_types:lfi``
+* **owners** -- Sessions with given owner type such as user, tool, crawler, attacker. 
+	E.g ``possible_owners:attacker``
+* **start_time** -- Sessions which started after `start_time`. 
+	E.g ``start_time:1480560``
+* **end_time** -- Sessions which ended before `end_time`. 
+	E.g ``end_time:1480560``
+
+**Multiple filters** can be applied as ``peer_ip:127.0.0.1 start_time:1480560 possible_owners:attacker``
 
-Multiple filters can be applied as ``peer_ip:127.0.0.1 start_time:1480560 possible_owners:attacker``
+Information about single session
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+.. code-block::
+	
+	/session/<sess-uuid>
 
-/session/<sess-uuid>
-~~~~~~~~~~~~~~~~~~~~~~~~
 It gives all information about the session with given uuid. Here you may find some of the text clickable such as 
-``peer_ip``,``possible_owners``, ``start_time``, ``end_time``, ``attack_types``. Clicking on them will display all the sessions will same attribute value.
+``peer_ip``, ``possible_owners``, ``start_time``, ``end_time``, ``attack_types``. Clicking on them will display all the sessions will same attribute value.