Add some documentation on writing OONI Tests using

the new framework.
hellais · Sep 28, 2012 · ffc44ce · ffc44ce
1 parent 839564d
commit ffc44ce
Show file tree

Hide file tree

Showing 7 changed files with 174 additions and 129 deletions.
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -16,7 +16,9 @@
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
-#sys.path.insert(0, os.path.abspath('.'))
+sys.path.insert(0,
+            os.path.join(os.path.dirname(__file__), '..', '..'))
+
 
 # -- General configuration -----------------------------------------------------
 
@@ -25,7 +27,8 @@
 
 # Add any Sphinx extension module names here, as strings. They can be extensions
 # coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
-extensions = ['sphinx.ext.todo', 'sphinx.ext.coverage', 'sphinx.ext.pngmath', 'sphinx.ext.viewcode']
+extensions = ['sphinx.ext.todo', 'sphinx.ext.coverage', 'sphinx.ext.pngmath',
+'sphinx.ext.viewcode', 'sphinx.ext.autodoc']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']

diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -1,96 +1,50 @@
 .. OONI documentation master file.
 
-==========
-About OONI
-==========
+Welcome to the OONI documentation!
+==================================
 
+    The Net interprets censorship as damage and routes around it.
+                John Gilmore; TIME magazine (6 December 1993)
 
-Dependencies
-************
+OONI, the Open Observatory of Network Interference, is a global observation
+network which aims is to collect high quality data using open methodologies,
+using Free and Open Source Software (FL/OSS) to share observations and data
+about the various types, methods, and amounts of network tampering in the world.
 
-* Twisted: http://twistedmatrix.com/trac/
-* PyYAML: http://pyyaml.org/
-* Scapy: http://www.secdev.org/projects/scapy/
-    * pypcap: http://code.google.com/p/pypcap/
-    * libdnet: http://code.google.com/p/libdnet/
 
-*Optional*
+Getting started
+***************
 
-* BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/
+If you choose to use virtualenv to setup your development environment you will
+need to do the following::
 
-Installation
-************
+    virtualenv ENV
+    source ENV/bin/activate
+    pip install twisted Scapy pyyaml
 
-On debian you can install all the dependecies with apt-get with this command:
+To get the latest version of scapy you will need mercurial. You can then install
+it with::
 
-    apt-get install python-twisted python-twisted-names python-yaml python-scapy python-beautifulsoup
-
-*The "hard" way*
-
-This involves installing the dependencies installable via easy_install/pip and
-the ones that are not by building them from source.
-
-"simple" dependencies via easy_install:
-
-    sudo easy_install pyyaml
-    sudo easy_install twisted
-    sudo easy_install beautifulsoup
-
-"simple" dependencies via pip:
-
-    sudo pip install pyyaml
-    sudo pip install twisted
-    sudo pip install beautifulsoup
+    pip install hg+http://hg.secdev.org/scapy
 
-libdnet:
+On debian you can install all the dependecies with apt-get with this command::
 
-    wget http://libdnet.googlecode.com/files/libdnet-1.12.tgz
-    tar xzf libdnet-1.12.tgz
-    cd libdnet-1.12
-    ./configure  && make
-    cd python/
-    sudo python setup.py install
-    cd ../../ && rm -rf libdnet-1.12*
-
-pypcap:
-
-    svn checkout http://pypcap.googlecode.com/svn/trunk/ pypcap-read-only
-    cd pypcap-read-only/
-    sudo pip install pyrex
-    make
-    sudo python setup.py install
-    cd ../ && rm -rf pypcap-read-only
-
-scapy:
-
-    wget http://www.secdev.org/projects/scapy/files/scapy-latest.zip
-    unzip scapy-latest.zip
-    cd scapy-2.2.0/
-    sudo python setup.py install
-    cd ../ && rm -rf scapy-*
-
-Running
-*******
+    apt-get install python-twisted python-twisted-names python-yaml python-scapy python-beautifulsoup
 
-To run ooni probe do
+Once you have installed all the dependencies OONI tests can be run like so::
 
-$ export PYTHONPATH=`pwd`
+    bin/ooniprobe path/to/test.py --cmd1 foo --cmd2 bar
 
-$ python ooni/ooniprobe.py
 
+Contents
+********
 
 .. toctree::
-   :maxdepth: 2
+    :maxdepth: 2
+    :glob:
 
-    intro
+    install
     tutorial
     writing_tests
-    ...
-
-Indices and tables
-==================
 
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`
 
diff --git a/docs/source/install.rst b/docs/source/install.rst
@@ -0,0 +1,55 @@
+
+Installing OONI
+===============
+
+Currently no installation documentation is present, since OONI is not meant to
+be installed and should be handled with care.
+
+Dependencies
+************
+
+OONI depends on the following pieces of software.
+
+* Twisted: http://twistedmatrix.com/trac/
+* PyYAML: http://pyyaml.org/
+* Scapy: http://www.secdev.org/projects/scapy/
+    * pypcap: http://code.google.com/p/pypcap/
+    * libdnet: http://code.google.com/p/libdnet/
+
+*Optional*
+
+* BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/
+
+Manual installation of scapy
+----------------------------
+
+It is optimal to install scapy, libdnet and pypcap from source. This can be
+done with the following code snippets.
+
+libdnet::
+
+    wget http://libdnet.googlecode.com/files/libdnet-1.12.tgz
+    tar xzf libdnet-1.12.tgz
+    cd libdnet-1.12
+    ./configure  && make
+    cd python/
+    sudo python setup.py install
+    cd ../../ && rm -rf libdnet-1.12*
+
+pypcap::
+
+    svn checkout http://pypcap.googlecode.com/svn/trunk/ pypcap-read-only
+    cd pypcap-read-only/
+    sudo pip install pyrex
+    make
+    sudo python setup.py install
+    cd ../ && rm -rf pypcap-read-only
+
+scapy::
+
+    wget http://www.secdev.org/projects/scapy/files/scapy-latest.zip
+    unzip scapy-latest.zip
+    cd scapy-2.2.0/
+    sudo python setup.py install
+    cd ../ && rm -rf scapy-*
+
diff --git a/docs/source/tutorial.rst b/docs/source/tutorial.rst
@@ -1,4 +1,3 @@
-========
 Tutorial
 ========
 

diff --git a/docs/source/writing_tests.rst b/docs/source/writing_tests.rst
@@ -1,26 +1,76 @@
-.. OONI documentation master file.
-
-==================
 Writing OONI tests
 ==================
 
-OONIProbe tests can be written in two modes: blocking or non-blocking.
 
-Going the blocking route is not advised and all tests in the end should end up
-being written in the non-blocking way.
+The OONI testing API is heavily influenced and partially based on the python
+:class:`unittest` module and :class:`twsted.trial`.
+
+
+Test Cases
+----------
+
+The atom of OONI Testing is called a Test Case. A test case class may contain
+multiple Test Functions.
+
+.. autoclass:: ooni.nettest.TestCase
+
+:class:`ooni.nettest.TestCase` is a subclass of :class:`unittest.TestCase` so
+the assert methods that apply to :class:`unittest.TestCase` will also apply to
+:class:`ooni.nettest.TestCase`.
+
+If the test you plan to write is not listed on the `Tor OONI trac page
+<https://trac.torproject.org/projects/tor/wiki/doc/OONI/Tests>`_, you should
+add it to the list and following the `test template
+<https://trac.torproject.org/projects/tor/wiki/doc/OONI/Tests/TestTemplate>`_
+write up a description about it.
+
+
+Inputs
+------
+
+Inputs are what is given as input to every iteration of the Test Case. You have
+100 inputs, then every test case will be run 100 times.
+
+To configure a static set of inputs you should define the
+:class:`ooni.nettest.TestCase` attribute ``inputs``. The test will be run ``len(inputs)`` times. Any iterable object is a valid ``inputs`` attribute.
+
+If you would like to have inputs be determined from a user specified input
+file, then you must set the ``inputFile`` attribute. This is an array that
+specifies what command line option may be used to control this value.
+
+By default the ``inputProcessor`` is set to read the file line by line and
+strip newline characters. To change this behavior you must set the
+``inputProcessor`` attribute to a function that takes as arugment a file
+descriptor and yield the next item. The default ``inputProcessor`` looks like
+this::
+
+
+    def lineByLine(fp):
+        for x in fp.readlines():
+            yield x.strip()
+        fp.close()
+
+
+Test Functions
+--------------
+
+These shall be defined inside of your :class:`ooni.nettest.TestCase` subclass.
+These will be class methods.
 
-A good way to understand how to write a test is also to take a look at the OONI
-Test Interface in the following files:
+To add data to the test report you may write directly to the report object like
+so::
 
-* ooni/plugoo/interface.py
+    def my_test_function():
+        result = do_something()
+        self.report['something'] = result
 
-* ooni/plugoo/tests.py
+OONI will then handle the writing of the data to the final test report.
 
-Writing non-blocking tests
---------------------------
+To access the current input you can use the ``input`` attribute, for example::
 
-To bootstrap the process of creating a new tests you can run the scaffolding
-script in ooni/scaffolding.py.
+    def my_test_with_input():
+        do_something_with_input(self.input)
 
-This will create a new plugin with the specified name inside of ooni/plugins/.
+This will at each iteration over the list of inputs do something with the
+input.
 
diff --git a/docs/writing_tests.md b/docs/writing_tests.md
diff --git a/ooni/nettest.py b/ooni/nettest.py
@@ -34,39 +34,40 @@ class TestCase(unittest.TestCase):
     This is the monad of the OONI nettest universe. When you write a nettest
     you will subclass this object.
 
-    _inputs_ can be set to a static set of inputs. All the tests (the methods
-    starting with the "test_" prefix) will be run once per input.  At every run
-    the _input_ attribute of the TestCase instance will be set to the value of
-    the current iteration over inputs.  Any python iterable object can be set
-    to inputs.
+    * inputs: can be set to a static set of inputs. All the tests (the methods
+      starting with the "test_" prefix) will be run once per input.  At every run
+      the _input_ attribute of the TestCase instance will be set to the value of
+      the current iteration over inputs.  Any python iterable object can be set
+      to inputs.
 
-    _inputFile_ attribute should be set to an array containing the command line
-    argument that should be used as the input file. Such array looks like this:
+    * inputFile: attribute should be set to an array containing the command line
+      argument that should be used as the input file. Such array looks like
+      this:
 
-        ["commandlinearg", "c", "The description"]
+          ``["commandlinearg", "c", "The description"]``
 
-    The second value of such arrray is the shorthand for the command line arg.
-    The user will then be able to specify inputs to the test via:
+      The second value of such arrray is the shorthand for the command line arg.
+      The user will then be able to specify inputs to the test via:
 
-        ooniprobe mytest.py --commandlinearg path/to/file.txt
+          ``ooniprobe mytest.py --commandlinearg path/to/file.txt``
 
-    or
+      or
 
-        ooniprobe mytest.py -c path/to/file.txt
+          ``ooniprobe mytest.py -c path/to/file.txt``
 
 
-    _inputProcessor_ should be set to a function that takes as argument an
-    open file descriptor and it will yield the input to be passed to the test
-    instance.
+    * inputProcessor: should be set to a function that takes as argument an
+      open file descriptor and it will yield the input to be passed to the test
+      instance.
 
-    _name_ should be set to the name of the test.
+    * name: should be set to the name of the test.
 
-    _author_ should contain the name and contact details for the test author.
-    The format for such string is as follows:
+    * author: should contain the name and contact details for the test author.
+      The format for such string is as follows:
 
-        The Name <email@example.com>
+          ``The Name <email@example.com>``
 
-    _version_ is the version string of the test.
+    * version: is the version string of the test.
     """
     name = "IDidNotChangeTheName"
     author = "John Doe <foo@example.com>"