Skip to content
This repository has been archived by the owner on Mar 29, 2019. It is now read-only.

Commit

Permalink
Merge pull request #39 from marumari/master
Browse files Browse the repository at this point in the history 
Massive rewrite of the docs
  • Loading branch information
@april
april committed Sep 14, 2015
2 parents dcf651c + 315301b commit 7bc69c0
Show file tree
Hide file tree
Showing 17 changed files with 447 additions and 772 deletions.
91 changes: 3 additions & 88 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
About Minion
============

Minion is a security testing framework built by Mozilla to bridge the gap between developers and security testers. To do so, it enables developers to scan their projects using a friendly interface.
Minion is a security testing framework built by Mozilla to bridge the gap between developers and security testers.
To do so, it enables developers to scan their projects using a friendly interface.

For complete user and developer documentation see http://minion.readthedocs.org/en/latest/

Expand All @@ -25,94 +26,8 @@ The following projects are optional plugins for minion that add more functionali
* https://github.com/mozilla/minion-skipfish-plugin
* https://github.com/mozilla/minion-nmap-plugin

This project contains a script that will check out the above projects. See below in the setup instructions.

Setting up a development environment
------------------------------------

Whatever platform you use, you will need the following tools:

* Python 2.7
* virtualenv
* git

For task distribution and data storage, Minion uses the following services:

* rabbitmq
* mongodb

If you work on Ubuntu, install the following packages:

$ sudo apt-get install git build-essential python-virtualenv python-dev rabbitmq-server mongodb-server curl libcurl4-openssl-dev
$ sudo apt-get install nmap skipfish

If you work on Fedora 19, install the following packages:

$ sudo yum groupinstall 'Development Tools'
$ sudo yum install python-devel python-virtualenv mongodb-server mongodb-devel rabbitmq-server libcurl-devel openssl-devel
$ sudo yum install nmap skipfish

You can make Minion ready for development by following these steps:

$ git clone https://github.com/mozilla/minion
$ cd minion
$ ./setup.sh clone
$ ./setup.sh develop

You can also run ``./setup.sh install`` if you choose to make Minion available to the global Python interpreter.
This option will run ``python setup.py install`` instead of ``python setup.py develop``.

Running Minion in Development Mode
----------------------------------

To run Minion you need to have five things up and running:

* The Frontend
* The Backend REST API
* The Backend Scan Worker
* The Backend State Worker
* The Backend Plugin Worker

The order is not important, just start them all up in separate terminal windows. The easiest way to start them in development mode is to use the provided `setup.sh` script.

Start the frontend in a new shell window:

$ cd minion
$ ./setup.sh run-frontend

Start the backend in a new shell window:

$ cd minion
$ ./setup.sh run-backend

Start the backend scan worker in a new shell window:

$ cd minion
$ ./setup.sh run-scan-worker

Start the backend state worker in a new shell window:

$ cd minion
$ ./setup.sh run-state-worker

Start the backend plugin worker in a new shell window:

$ cd minion
$ ./setup.sh run-plugin-worker

Start the scan scheduler worker in a new shell window:

$ cd minion
$ ./setup.sh run-scheduler-worker

Start the scan scheduler in a new shell window:

$ cd minion
$ ./setup.sh run-scheduler


License
-------
This software is licensed under the MPL License. For more
information, read the file ``LICENSE``.
information, read the file ``LICENSE.md``.

2 changes: 1 addition & 1 deletion docs/source/conf.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@

# General information about the project.
project = u'Minion'
copyright = u'2013-2014, Mozilla'
copyright = u'2013-2015, Mozilla'

# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
Expand Down
190 changes: 129 additions & 61 deletions docs/source/configure_minion.rst
View file
Original file line number Diff line number Diff line change
@@ -1,72 +1,74 @@
Configure Minion
################
Configuring Minion
##################

Minion backend and frontend can be configured after installation. This document will
explain how to configure them.
This document explains how to configure the Minion frontend and backend.

As a convention, Minion will look at ``/etc/minion/`` and ``/home/user/.minion``. In the second
case, the user home directory is the user that runs Minion backend server.
As a convention, Minion will look in ``/etc/minion/`` and ``~minion/.minion`` for its configuration files.

.. _whitelist_blacklist_hostname_label:
.. _configure_minion_frontend_label:

Hostname Whitelist and Blacklist
================================
Configure Minion Frontend
=========================

As of Minion 0.3 release, Minion will blacklist the following IP addresses from scanning.
Here is the `default configuration <https://github.com/mozilla/minion-vm/blob/master/frontend.json>`_ for the Minion frontend server::

.. code-block:: python
{
"backend-api": {
"url": "http://minion-backend:8383"
},

'10.0.0.0/8',
'127.0.0.0/8',
'172.16.0.0/12',
'192.168.0.0/16',
'169.254.0.0/16'
"login": {
"type": "persona",

You can check the latest list from https://github.com/mozilla/minion-backend/blob/master/minion/backend/utils.py.
"ldap": {
"uri": "ldaps://ldap.server/",
"baseDN": "ou=test,dc=test_dc",

The effect of this is that Minion will refuse to scan any target site whose hostname falls in one of the ranges.
For example, when Minion resolve the hostname ``localhost`` to ``127.0.0.1``, Minion will abort the scan because
it is blacklisted.
"emailAttribute": "mail",
"groupMembershipAttribute": "member",
"usernameAttribute": "uid",

To configure the blacklist and whitelist, you can supply a file called **scan.json** in either ``/etc/minion/``
or ``/home/user/.minion/``.
"checkAuthorizedGroups": false,
"authorizedGroups": [
"ou=groupTest1,ou=test,dc=test_dc",
"ou=groupTest2,ou=test,dc=test_dc"
]
}
}
}

.. code-block:: python
To configure the frontend, place your configuration in a file called ``frontend.json`` in either ``/etc/minion`` or ``/home/user/.minion``.

{
"whitelist": [
"192.168.0.0/16",
"127.0.0.1"
]
}
- ``backend-api``

In this configuration, we allowed scanning LAN network and localhost. This is useful when you are testing your
own web application from home. However, ``172.16.0.0/12`` range is still restricted from scanning.
- ``uri``: URI of the Minion backend server

You can supply your own blaclist as well.
- ``login``

.. code-block:: python
- ``type``: the type of authentication to use; support types are currently ``persona``, which requires no configuration, and ``ldap``

{
"whitelist": [
"192.168.0.0/16",
"127.0.0.1"
],
"blacklist": [
"foobar.com"
]
}
- ``ldap``: the configuration for LDAP, if ``ldap`` is the chosen authentication method in ``login -> ``type``

- ``uri``: URI to ldap server

- ``baseDN``: baseDN for users; not needed for Active Directory

- ``emailAttribute``: typically ``mail`` in OpenLDAP or ``userPrincipalName`` in Active Directory

In this example, foobar.com is not scannable. When we specify our own blacklist, we replace the default one
entirely with our own. So we can omit the whitelist in our example.
- ``groupMembershipAttribute``: typically ``member`` in OpenLDAP or ``uniqueMember`` in Active Directory

- ``usernameAttribute``: typically ``uid`` in OpenLDAP or ``samAccountName`` in AD

Configure Backend
=================
- ``checkAuthorizedGroups``: if true, require group membership in addition to valid user id

Here is the default configuration for the backend server (see https://github.com/mozilla/minion-backend/blob/master/minion/backend/utils.py)
- ``authorizedGroups``: list of groups where users are authorized to use Minion (if ``checkAuthorizedGroups`` is true)

.. code-block:: python
.. _configure_minion_backend_label:

Configure Minion Backend
========================

Here is the `default configuration <https://github.com/mozilla/minion-backend/blob/master/etc/backend.json>`_ for the Minion backend server::

{
'api': {
Expand All @@ -83,29 +85,95 @@ Here is the default configuration for the backend server (see https://github.com
'email': {
'host': '127.0.0.1',
'port': 25,
'max_time_allowed': 3600 * 24 * 7 # seconds in 7 days
'max_time_allowed': 604800 * 24 * 7 # 60 * 60 * 24 * 7 seconds
}
}

To configure the backend, supply all the options in a file called ``backend.json`` at either ``/etc/minion`` or
To configure the backend, place your configuration in a file called ``backend.json`` at either ``/etc/minion`` or
``/home/user/.minion``.

The ``api/url`` is the full authority (hostname and port) of the backend server.
- ``api``

- ``url``: the full authority (hostname and port) of the backend server.

- ``celery``

- ``broker``: URI of the celery broker

- ``backend``: protocol used to speak to backend

- ``mongodb``:

The ``max_time_allowed`` determines the life time of an invitation; by default it will remain valid for seven days.
- ``host``: hostname of MongoDB server

Configure Frontend
==================
- ``port``: port of the MongoDB server

The frontend is much simpler.
- ``email``

.. code-block:: python
- ``host``: hostname of mail server

- ``port``: port of mail server

- ``max_time_allowed``: determines the life time of an invitation; by default it will remain valid for seven days.


.. _whitelist_blacklist_hostname_label:

Whitelisting and Blacklisting Hosts
===================================

By default, `Minion will blacklist <https://github.com/mozilla/minion-backend/blob/master/etc/scan.json>`_ the following IP addresses from being scanned:

.. code-block:: javascript
"blacklist": [
"10.0.0.0/8",
"127.0.0.0/8",
"172.16.0.0/12",
"192.168.0.0/16",
"169.254.0.0/16"
]
You can check the latest list at: .

The effect of this is that Minion will refuse to scan any target site whose hostname falls in one of the ranges.
For example, when Minion resolve the hostname ``localhost`` to ``127.0.0.1``, Minion will abort the scan because
it is blacklisted.

To configure the blacklist and whitelist, you can copy ``etc/scan.json`` into either ``/etc/minion/`` or ``~minion/.minion/``. Note that the whitelist will override the blacklist, so in this example, IP addresses in 192.168.1.0/24 can be scanned, despite 192.168.0.0/16 being in the blacklist:

.. code-block:: javascript
{
'backend-api': {
'url': 'http://127.0.0.1:8383'
}
"whitelist": [
"192.168.1.0/24"
],
"blacklist": [
"10.0.0.0/8",
"127.0.0.0/8",
"172.16.0.0/12",
"192.168.0.0/16",
"169.254.0.0/16"
]
}
If the backend server is on a different server, then put this configuration in a file called ``frontend.json``
at either ``/etc/minion`` or ``/home/user/.minion``.
Any host that does not fall within the blacklist can be scanned.

IP address blacklisting and whitelist also supports hostnames and hostname wildcards. For example:

.. code-block:: javascript
"blacklist": [
"mozilla.com",
"*.mozilla.org"
]
In this configuration, we allowed scanning LAN network and localhost, but we removed the ability to scan mozilla.com and any subdomain of mozilla.org. Note that if we wanted to block mozilla.org and subdomains, we would need entries for ``mozilla.org`` and ``*.mozilla.org``:

.. code-block:: javascript
"blacklist": [
"mozilla.org",
"*.mozilla.org"
]
17 changes: 6 additions & 11 deletions docs/source/contribute_to_minion.rst
View file
Original file line number Diff line number Diff line change
@@ -1,17 +1,12 @@
Contribute
##########

Minion is built using open source projects such as Python, AngularJS and Twsited, so
we want to keep it open source. Furthermore, we believe that
**"secrecy does not buy additional security."** (*Kerckhoff's principle*) Security testing
is not an exclusive privilege. Instead, security testing should be as agile as possible.
Minion is built using open source projects such as Python, angular.js, Flask, Celery, and Twisted. In order to make Minion feature richer and secure, we invite the community to contribute in any way they can. If you have ideas to submit, or have issues to report, you encourage to contact us via:

In order to make Minion feature richer and secure, we invite the community to contribute
in any way they can. If you have ideas to submit, or have issues to report, you encoruage
to contact us via:
* IRC: #websectools channel on irc.mozilla.org
* Email list: http://groups.google.com/group/mozilla-minion-dev
* Any of our GitHub repos:

.. include:: include/minion_repos.rst

* Source code: https://github.com/mozilla/minion
* Email list: http://groups.google.com/group/mozilla-minion-dev
* `Security/Projects/Minion/Roadmap <https://wiki.mozilla.org/Security/Projects/Minion>`_
* We also use the **#websectools** channel on irc.mozilla.org
.. include:: include/plugin_repos.rst

0 comments on commit 7bc69c0

Please sign in to comment.