Documentation update

docs/Core.rst

@@ -2,80 +2,105 @@
 Core
 ====
 
-At initialization of CapTipper,
-The PCAP file is being parsed by the pcap_parser.py library which was originally written by ... and was further modified to suit with the CapTipper.
+CapTipper is using a modified version of the pcap_parser library in order to parse the PCAP file.
 
-There are three main datasets handled in CapTipper Core:
-Conversations
-Objects
-Hosts
+There are three main datasets handled in CapTippers core:
+
+* Conversations
+* Objects
+* Hosts
 
 Conversations
 ===============
-The parsing poplulates a data structure called CTCore.Conversations.
+At first, the parsing populates a data structure called ``CTCore.Conversations``.
 A 'Conversation' is defined as Request and a Response.
 Conversations is a list of named tuples, each named tuple consists of the following values:
-id - Conversation ID
-server_ip_port - Server ip and port in the form of IP:PORT
-uri - Relative object on the host (e.g "/images/cat.jpg")
-req - Raw Request data
-res_body - Response raw body
-res_head - Response Header
-res_num - HTTP response number (e.g "200 OK")
-res_type - HTTP response type (e.g "application/octet-stream")
-host - Server Host name
-referer - Referer URL
-filename - Response object name (e.g "cat.jpg")
-method - Rquest method (GET/POST/etc..)
-redirect_to -
-req_microsec - Request microsec time
-res_len - Response size
-magic_name - Filetype as identified by Whatype (e.g "Windows executable file")
-magic_ext - Filetype extension as identified by Whatype (e.g "EXE")
-
- ['id','server_ip','uri','req','res_body','res_head','res_num','res_type','host','referer', 'filename','method','redirect_to','req_microsec', 'res_len','magic_name', 'magic_ext']))
+
+* id - Conversation ID
+* server_ip_port - Server ip and port in the form of IP:PORT
+* uri - Relative object on the host (e.g "/images/cat.jpg")
+* req - Raw Request data
+* res_body - Response raw body
+* res_head - Response Header
+* res_num - HTTP response number (e.g "200 OK")
+* res_type - HTTP response type (e.g "application/octet-stream")
+* host - Server Host name
+* referer - Referer URL
+* filename - Response object name (e.g "cat.jpg")
+* method - Rquest method (GET/POST/etc..)
+* redirect_to - The URL will be redirected to in case the ``location`` response header exists
+* req_microsec - Request microsec time
+* res_len - Response size
+* magic_name - Filetype as identified by Whatype (e.g "Windows executable file")
+* magic_ext - Filetype extension as identified by Whatype (e.g "EXE")
 
 A conversation is added to the list during the parsing process upon parsing the response packet.
 
-the function responsible for adding the conversation is finish_conversations(self) in CTCore.py.
-as part of the function, a refernce to the Whatype library (Read more) is called.
+The function responsible for adding the conversation is ``finish_conversations(self)`` in CTCore.py.
+as part of the function, a reference to the Whatype library (Read more) is called.
 
-A conversation may sometimes be refered as object, while object is meant to describe only the response.
 The file name set for each conversation is created with this priority:
-1. Filename given to the object from the server with "Content Disposition: filename=<file>"
-2. The last part of the URI, diregarding arguments.
-3. Number of object + ".html"
+
+ 1. Filename given to the object from the server with "Content Disposition: filename=<file>"
+ 2. The final part of the URI (ignoring arguments)
+ 3. Number of object + ".html"
+
+When requesting to display the conversation using the command ``convs``,
+a unique color is given to different conversations based on the type received in the responses ``content_type``:
+
+- RED - PDF Files
+- BLUE - JavaScript Files
+- GREEN - Images
+- YELLOW - Generic binary files
+
 
 Objects
 =========
-Object is a much thinner representation of the Conversations dataset, and contains only the following information:
+Object is a much thinner representation of the conversations dataset, and contains only the following information:
 
-type 
-value
-conv_id
-name
+* id - Object ID
+* type - Object type  [body / ungzip / jsbeautify] (default: 'body')
+* value - Object content
+* conv_id - Associated conversation id (alias: CID)
+* name - File name
 
-all conversations are stored in objects list with the 'type' of 'body'
-'value' contains the response body of each conversation
-'conv_id' is as reference for objects that were created from conversations (e.g ungzip)
-'name' will be the filename that was given to the conversation, except for the dynamically created objects, 
+All initial conversations are stored in the objects list with the string 'body' as ``type`` and ``value`` containing the the response data.
+``id`` is unique to each object and objects that are also conversations have the same id.
+``conv_id`` is a reference for objects that were created from conversations (e.g ungzip).
+``name`` will be the filename that was given to the conversation;
+dynamically created objects will have a prefix of how they were created,
 for example in case of ungzip, the name will be "ungzip-[filename]"
 
+Objects list example (using the command ``objects``):
+
+::
+
+    ID   CID     TYPE          NAME
+    ---- -----  -----------   --------
+      0 | 0   | body        | 0.html
+      1 | 1   | body        | jquery.js
+      2 | 2   | body        | logo.jpg
+      3 | 3   | body        | kitty.png
+      4 | 0   | ungzip      | ungzip-0.html
+
 Hosts
 =========
-The hosts data set is a dictionary with the key beind a tuple of (host, ip) and the value for each key is a live of the URIs.
+The hosts data set is a dictionary with the key being a tuple of (host, ip) and the value for each key is a list of the domains URIs.
+
 for example:
-[(google.com, )]
+::
+
+     (
+        ('www.example.com','93.184.216.34:80'),
+          ['/   [0]',
+          '/js/jquery.js?ver=1.7.3   [2]',
+          '/images/logo.jpg   [4]',
+          '/images/kitty.png   [5]']
+     )
 
-The conversations list
-=================
-- Colors
 
 Update
-============
-The update process check for a version difference between the local project to the github repository.
+=======
+The update process checks for a version difference between the local project to the github repository.
 In case such a difference exists, it downloads the master branch and expands it to the local path and overwrites current .py files.
 
-Send To VirusTotal
-=====================
-

docs/Plugins.rst

@@ -2,25 +2,22 @@
 Plugins
 =======
 
-CapTipper supports external plugins for easy use through the CapTipper
-console.
+CapTipper is also a framework who supports plugins as extensions for the console.
+If you have an idea for a plugin, you should implement it and send it to me or make a PULL request to the GitHub repository.
 
 Getting Started
 ===============
 
-All plugin modules must be placed inside the directory plugins/ The
-modules are loaded by CapTipper dynamically when launched. The
+All plugin modules must be placed inside the ``plugins/`` directory.
+The modules are loaded by CapTipper dynamically when launched.
 
-Plugins are python modules who implements the ConsolePlugin interface. A
-CapTipper plugin must import the ConsolePlugin class and inherint it.
+All Plugins are python modules who implements the ``ConsolePlugin`` interface. A
+CapTipper plugin must import the ``ConsolePlugin`` class from ``CTPlugin`` and inherent it.
 
-The run() function is the EntryPoint CapTipper will use to launch the
-plugin. Also, The class implementing the ConsolePlugin must have the
-same name as the py file.
+The ``run()`` function is the entry point CapTipper will use to launch the plugin.
+The class implementing the ``ConsolePlugin`` must have the same name as the ``.py`` file.
 
-Basic example:
-
-my\_first\_plugin.py
+Basic example (``my_first_plugin.py``):
 
 .. code-block:: python
 
@@ -34,18 +31,21 @@ my\_first\_plugin.py
         def run(self, args):
             print "Hello World"
 
-The plugin result can be printed through the plugin using "print", or
-returned from the RUN function; if run() result is different than None,
-the information will be printed to the screen.
+The plugin result can be printed by the plugin using "print", or returned from the Run function.
+In case result returned from ``run()`` is different than ``None``, the information will be printed out.
 
 Global Structurs
 ================
 
-CapTipper hold 3 main Data Structures conainting all information. 1.
-Conversations 2. Objects 3. Hosts
+CapTipper hold 3 main data structures containing all information:
 
-All of which, are accessible from the plugin class using
+1. Conversations
+2. Objects
+3. Hosts
 
+- See #Core for more information regarding the data sets.
+
+All of which, are accessible from the plugin class using
 
 .. code-block:: python
 
@@ -56,16 +56,16 @@ All of which, are accessible from the plugin class using
 Internal Functions
 ==================
 
-The ConsolePlugin interface contains some function for more convient
-use: get\_name\_by\_id - Returns response object name (e.g "index.html")
-of a given conversation id get\_body\_by\_id - Returns the raw response
-body of a given convesation get\_plaintext\_body\_by\_id - Returns
-plaintext response body (e.g ungzipped in case needed) is\_valid\_id -
-Checks if id sent to the plugin is a valid one
+The ``ConsolePlugin`` interface contains important function that should be used when accessing relevant information:
+
+* ``get_name_by_id(obj_id)`` - Returns response object name (e.g "index.html") of a given conversation id
+* ``get_body_by_id(obj_id)`` - Returns the raw response body of a given convesation
+* ``get_plaintext_body_by_id(obj_id)`` - Returns plaintext response body (e.g ungzipped in case needed)
+* ``is_valid_id(obj_id)`` - Checks if id sent to the plugin is a valid one
 
-It is easy to import other function from the CapTipper Core.
+It is also easy to import other function from the CapTipper Core.
 
-For example:
+Let's take a look at an example of importing the ``ungzip`` function from ``CTCore``:
 
 .. code-block:: python
 
@@ -87,15 +87,15 @@ For example:
             else:
                 print "invalid id"
 
-You can also see here the use of the conversations internal variable
-"magic\_ext" Best practice for ungzip would be to use the
-get\_plaintext\_body\_by\_id() Function.
+You can also see here the use of the conversation internal variable ``magic_ext``.
+
+It is important to mention that the best practice for getting the conversation body is using the
+``get_plaintext_body_by_id()`` function which will also ungzip the data if necessary.
 
 Example #1
 ==========
 
-The following plugin imports the srcHTMLParser from CTCore, and searches
-external javascript referenced in a given conversation
+The following plugin imports the srcHTMLParser from CTCore, and searches external javascript referenced in a given conversation
 
 .. code-block:: python
 
@@ -111,7 +111,11 @@ external javascript referenced in a given conversation
             if len(args) > 0:
                 # Get the conversation ID
                 id = int(args[0])
+
+                # Checks if id is value
                 if self.is_valid_id(id):
+
+                    # Gets conversation name
                     name = self.get_name_by_id(id)
                     print "[.] Searching for external scripts in object {} ({})...".format(str(id),name)
 
@@ -121,6 +125,8 @@ external javascript referenced in a given conversation
                     # Create Parser instance and search for <script src="...
                     parser = srcHTMLParser("script")
                     parser.feed(response_body)
+
+                    # Prints results
                     parser.print_objects()
                 else:
                     print "Invalid conversation ID {}".format(str(id))
@@ -130,8 +136,7 @@ external javascript referenced in a given conversation
 Example #2
 ==========
 
-The following plugin checks if the host involved in a given conversation
-is still connectable using the stored IP and Port
+The following plugin checks if the host involved in a given conversation is still alive using a socket object and the conversations stored IP and Port.
 
 .. code-block:: python