Merge pull request #35 from automationator/master

Updates documentation

README.rst

@@ -1,81 +1,30 @@
-SIP: Simple Intel Platform
-==========================
+Simple Intel Platform
+=====================
 
 .. image:: https://readthedocs.org/projects/simple-intel-platform/badge/?version=latest
   :target: https://simple-intel-platform.readthedocs.io/en/latest/?badge=latest
   :alt: Documentation Status
 
-Read the full documentation at `Read the Docs <https://simple-intel-platform.readthedocs.io/en/latest/>`_.
+What is SIP?
+------------
 
-Run the included setup script to begin configuration.
+SIP is a Dockerized threat intelligence platform for managing security events, indicators, and the relationships between them. It is designed for ease of use and a low barrier of entry so that anyone can quickly deploy a production-ready intel platform for their organization.
 
-::
+At its core, SIP is a RESTful API sitting atop a database that has been carefully designed to hold the information we have found over the years to be the most impactful in developing a successful threat intelligence program.
 
-   ./setup.py
+Why use SIP?
+------------
 
-The setup script will walk you through:
+Security is not easy. It is not enough to simply purchase the Latest And Greatest™ product and believe that your organization is protected. There is an endless supply of financially-motivated attackers, and their techniques are always evolving.
 
--  Creating dev/test/production Docker environments
--  Setting the MySQL root user password
--  Creating a non-root MySQL user and password
--  Creating an optional self-signed HTTPS certificate
+The best way to protect your organization is to ensure that you learn something from every attack that you detect and feed that knowledge back into your detection tools. This is the detection+intel feedback loop, and mastering it is key to building an effective intel program for your organization.
 
-Once you answer the setup script's questions, it will generate files
-containing the environment variables for the MySQL and Flask Docker
-containers.
+SIP is the product of years worth of hands-on experience and insight from some of the most successful security and intel teams and was built with the hope that sharing this experience makes us all more secure.
 
-**MAKE SURE TO NOTE THE ADMIN PASSWORD REPORTED BY THE SETUP SCRIPT!**
+Getting Started
+---------------
 
-If you created a dev or a production environment, their containers will
-be built and initialized with the default database values specified in
-the setup.ini file:
-
-::
-
-   services/web/etc/setup.ini
-
-To run the unit/integration tests:
-----------------------------------
-
-NOTE: A separate test Docker environment will be automatically built if
-you chose to build either a dev or production environment.
-
-::
-
-   bin/test.sh
-
-To initialize DB migrations (Flask-Migrate):
---------------------------------------------
-
-Note: This should only be performed for the DEV environment.
-
-::
-
-   bin/db-init-DEV.sh
-
-To generate DB migrations (Flask-Migrate):
-------------------------------------------
-
-Note: This should only be performed for the DEV environment.
-
-::
-
-   bin/db-migrate-DEV.sh
-
-To upgrade the DB (Flask-Migrate):
-----------------------------------
-
-Note: This can be performed on either the DEV or PROD environment.
-
-::
-
-   bin/db-upgrade-DEV.sh
-   bin/db-upgrade-PROD.sh
-
-To enter a shell into one of the docker containers:
----------------------------------------------------
-
-::
-
-   docker exec -i -t sip_web-dev_1 /bin/sh
-   docker exec -i -t <container_name> /bin/sh
+1. :ref:`setup` SIP
+2. Use the included :ref:`scripts` to start and stop SIP
+3. Access the SIP :ref:`api`
+4. Access the SIP :ref:`gui`

services/web/docs/api.rst

@@ -1,6 +1,12 @@
+.. _api:
+
 API
 ***
 
+**DEV**: :code:`https://127.0.0.1:4443/api`
+
+**PRODUCTION**: :code:`https://127.0.0.1/api`
+
 .. toctree::
    :maxdepth: 1
    :caption: Contents:

services/web/docs/conf.py

@@ -44,8 +44,6 @@
 # ones.
 extensions = [
     'sphinx.ext.autodoc',
-    'sphinx.ext.doctest',
-    'sphinx.ext.coverage',
     'sphinxcontrib.httpdomain',
     'sphinxcontrib.autohttp.flask',
     'sphinxcontrib.autohttp.flaskqref',
@@ -91,7 +89,12 @@
 # further.  For a list of options available for each theme, see the
 # documentation.
 #
-# html_theme_options = {}
+html_theme_options = {
+    'description': 'Simple Intel Platform',
+    'github_user': 'IntegralDefense',
+    'github_repo': 'SIP',
+    'github_button': True
+}
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,

services/web/docs/gui.rst

@@ -0,0 +1,10 @@
+.. _gui:
+
+GUI
+***
+
+SIP uses `Flask-Admin <https://flask-admin.readthedocs.io/en/latest/>`_ for a basic web interface. Some of the functionality is restricted within the GUI for performance reasons, as SIP is primarily meant to be used via its :ref:`api`.
+
+**DEV**: :code:`https://127.0.0.1:4443/SIP`
+
+**PRODUCTION**: :code:`https://127.0.0.1/SIP`

services/web/docs/index.rst

@@ -6,8 +6,12 @@
 .. include:: ../../../README.rst
 
 .. toctree::
+   :hidden:
    :maxdepth: 2
    :caption: Contents:
 
    Overview <self>
    API <api>
+   GUI <gui>
+   Scripts <scripts>
+   Setup <setup>

services/web/docs/scripts.rst

@@ -0,0 +1,74 @@
+.. _scripts:
+
+Scripts
+*******
+
+.. contents::
+  :backlinks: none
+
+After you have setup SIP, the included scripts will help you to easily run and maintain SIP.
+
+**Start**
+
+::
+   
+   $ bin/start-DEV.sh
+   $ bin/start-PROD.sh
+
+**Stop**
+
+::
+
+   $ bin/stop-DEV.sh
+   $ bin/stop-PROD.sh
+
+Integration Tests
+-----------------
+
+A separate TEST Docker environment will be automatically built if you chose to build either a DEV or PRODUCTION environment during the setup process.
+
+::
+
+   $ bin/test.sh
+
+Database Migrations
+-------------------
+
+SIP uses `Flask-Migrate <https://flask-migrate.readthedocs.io/en/latest/>`_ to handle any database schema changes and migrations.
+
+*NOTE*: This should only be performed for the DEV environment.
+
+**Initialize**
+
+This should never need to be performed unless you delete the included migrations directory. Additionally, it is only meant to run on the DEV environment.
+
+::
+
+   $ bin/db-init-DEV.sh
+
+**Migrate**
+
+Once you have made some changes to the database schema in the models.py file, you need to generate the schema migrations. This is also only meant to run on the DEV environment.
+
+::
+
+   $ bin/db-migrate-DEV.sh
+
+**Upgrade**
+
+After you have created the schema migrations (or have received migrations through a git pull), you need to upgrade your database. This can run on the DEV or PRODUCTION environments.
+
+::
+
+   $ bin/db-upgrade-DEV.sh
+   $ bin/db-upgrade-PROD.sh
+
+Debugging
+---------
+
+If SIP refuses to start, and you suspect the issue is with the Docker container itself, you can enter a shell to the containers:
+
+::
+
+   $ docker exec -i -t sip_web-dev_1 /bin/sh
+   $ docker exec -i -t <container_name> /bin/sh