New README and documentation strategy

.gitignore

@@ -1,13 +1,19 @@
 *~
 *.swp
 *.pyc
+*.so
+*.dylib
 bin
 eggs
 parts
 .installed.cfg
 .mr.developer.cfg
 *.egg-info
-src
 develop-eggs
-dist
 sphinx
+dist
+.nfs*
+.gdb_history
+build
+*.egg
+src/

.travis.yml

@@ -6,6 +6,7 @@ matrix:
     env:
     - secure: qL1i5WsXhScWH6AT3bThx17Puw90+w9vKVxS0fVRTo2HOQ4dpofZEnwFia+QBg5ARSiELE7Yl1kyHXyFWT4RcES0TJP5KYqhDL4LToYwKHzmlizWHN7cTPWF24fs8M06IMjhvHr0TL6VKh3rtFHhGDV6hjCNiB3OShupnQtnVN4=
     - secure: V0v5m68Ed8LQufe421JwKobtVw3EE2iKbAR2JhNzwCWjej16qc4olPN3g+wPXYyTNpseC98qe2DKdl7NuREgsOarfYJs//wn7te2FSFegsbNHMoHPTeDr6UY8O7qWjnqrPAssaE0dFLPhippRCGZ8ajaJOhEOO07zYkqa10eOPo=
+    - BOB_DOCUMENTATION_SERVER=https://www.idiap.ch/software/bob/docs/latest/bioidiap/%s/master
   - python: 3.2
     env:
     - NUMPYSPEC===1.8.0
@@ -15,11 +16,11 @@ matrix:
 before_install:
 - sudo add-apt-repository -y ppa:biometrics/bob
 - sudo apt-get update -qq
-- sudo apt-get install -qq --force-yes libboost-all-dev libblitz1-dev libhdf5-serial-dev libatlas-dev libatlas-base-dev liblapack-dev
+- sudo apt-get install -qq --force-yes libboost-all-dev libblitz1-dev libhdf5-serial-dev libatlas-dev libatlas-base-dev liblapack-dev texlive-latex-recommended texlive-latex-extra texlive-fonts-recommended
 - if [ -n "${NUMPYSPEC}" ]; then sudo apt-get install -qq gfortran; fi
 - if [ -n "${NUMPYSPEC}" ]; then pip install --upgrade pip setuptools; fi
-- if [ -n "${NUMPYSPEC}" ]; then pip install --find-links http://wheels.astropy.org/ --find-links http://wheels2.astropy.org/ --use-wheel numpy$NUMPYSPEC ; fi
-- pip install --find-links http://wheels.astropy.org/ --find-links http://wheels2.astropy.org/ --use-wheel matplotlib==1.3.0 sphinx nose==1.3.0 jinja2==2.6 coveralls
+- if [ -n "${NUMPYSPEC}" ]; then pip install --find-links http://wheels.astropy.org/ --find-links http://wheels2.astropy.org/ --use-wheel numpy$NUMPYSPEC; fi
+- pip install --find-links http://wheels.astropy.org/ --find-links http://wheels2.astropy.org/ --use-wheel matplotlib==1.3.0 sphinx nose==1.3.0 jinja2==2.6 coveralls setuptools==7.0
 install:
 - python bootstrap.py
 - ./bin/buildout

README.rst

@@ -1,136 +1,50 @@
 .. vim: set fileencoding=utf-8 :
 .. Manuel Guenther <manuel.guenther@idiap.ch>
-.. Thu Sep  4 11:35:05 CEST 2014
+.. Fri Oct 31 14:18:57 CET 2014
 
+.. image:: http://img.shields.io/badge/docs-stable-yellow.png
+   :target: http://pythonhosted.org/bob.db.mnist/index.html
+.. image:: http://img.shields.io/badge/docs-latest-orange.png
+   :target: https://www.idiap.ch/software/bob/docs/latest/bioidiap/bob.db.mnist/master/index.html
 .. image:: https://travis-ci.org/bioidiap/bob.db.mnist.svg?branch=master
    :target: https://travis-ci.org/bioidiap/bob.db.mnist
 .. image:: https://coveralls.io/repos/bioidiap/bob.db.mnist/badge.png
    :target: https://coveralls.io/r/bioidiap/bob.db.mnist
-.. image:: http://img.shields.io/github/tag/bioidiap/bob.db.mnist.png
-   :target: https://github.com/bioidiap/bob.db.mnist
+.. image:: https://img.shields.io/badge/github-master-0000c0.png
+   :target: https://github.com/bioidiap/bob.db.mnist/tree/master
 .. image:: http://img.shields.io/pypi/v/bob.db.mnist.png
    :target: https://pypi.python.org/pypi/bob.db.mnist
 .. image:: http://img.shields.io/pypi/dm/bob.db.mnist.png
    :target: https://pypi.python.org/pypi/bob.db.mnist
+.. image:: https://img.shields.io/badge/original-data--files-a000a0.png
+   :target: http://yann.lecun.com/exdb/mnist
 
-================
- MNIST Database
-================
+==================================
+ MNIST Database Interface for Bob
+==================================
 
-The MNIST database is a database of handwritten digits, which consists of a
-training set of 60,000 examples, and a test set of 10,000 examples. It was
-made available by Yann Le Cun and Corinna Cortes (`MNIST database
-<http://yann.lecun.com/exdb/mnist/>`_). The data was originally extracted
-from a larger set made available by `NIST <http://www.nist.gov/>`_, before
-being size-normalized and centered in a fixed-size image (28x28 pixels).
+The MNIST_ database is a database of handwritten digits, which consists of a training set of 60,000 examples, and a test set of 10,000 examples.
+It was made available by Yann Le Cun and Corinna Cortes.
+The data was originally extracted from a larger set made available by NIST_, before being size-normalized and centered in a fixed-size image (28x28 pixels).
 
-The actual raw data for the database should be downloaded from the `original
-website <http://yann.lecun.com/exdb/mnist/>`_. This package only contains
-the `Bob <http://www.idiap.ch/software/bob/>`_ accessor methods to use this
-database directly from python.
+This package only contains the Bob_ accessor methods to use this database directly from Python.
+It does not contain the original raw data files, which need to be obtained through the link above.
 
-You would normally not install this package unless you are maintaining it. What
-you would do instead is to tie it in at the package you need to **use** it.
-There are a few ways to achieve this:
+Installation
+------------
+To install this package -- alone or together with other `Packages of Bob <https://github.com/idiap/bob/wiki/Packages>`_ -- please read the `Installation Instructions <https://github.com/idiap/bob/wiki/Installation>`_.
+For Bob_ to be able to work properly, some dependent packages are required to be installed.
+Please make sure that you have read the `Dependencies <https://github.com/idiap/bob/wiki/Dependencies>`_ for your operating system.
 
-1. You can add this package as a requirement at the ``setup.py`` for your own
-   `satellite package
-   <https://github.com/idiap/bob/wiki/Virtual-Work-Environments-with-Buildout>`_
-   or to your Buildout ``.cfg`` file, if you prefer it that way. With this
-   method, this package gets automatically downloaded and installed on your
-   working environment, or
+Documentation
+-------------
+For further documentation on this package, please read the `Stable Version <http://pythonhosted.org/bob.db.mnist/index.html>`_ or the `Latest Version <https://www.idiap.ch/software/bob/docs/latest/bioidiap/bob.db.mnist/master/index.html>`_ of the documentation.
+For a list of tutorials on this or the other packages ob Bob_, or information on submitting issues, asking questions and starting discussions, please visit its website.
 
-2. You can manually download and install this package using commands like
-   ``easy_install`` or ``pip``.
+.. _bob: https://www.idiap.ch/software/bob
+.. _mnist: http://yann.lecun.com/exdb/mnist
+.. _nist: http://www.nist.gov
 
-The package is available in two different distribution formats:
 
-1. You can download it from `PyPI <http://pypi.python.org/pypi/bob.db.mnist>`_, or
 
-2. You can download it in its source form from `its git repository
-   <https://github.com/bioidiap/bob.db.mnist>`_.
-
-The database raw files must be installed somewhere in your environment.
-
-You can mix and match points 1/2 above based on your requirements. Here
-are some examples:
-
-Modify your setup.py and download from PyPI
-===========================================
-
-That is the easiest. Edit your ``setup.py`` in your satellite package and add
-the following entry in the ``install_requires`` section (note: ``...`` means
-`whatever extra stuff you may have in-between`, don't put that on your
-script)::
-
-    install_requires=[
-      ...
-      "bob.db.mnist",
-    ],
-
-Proceed normally with your ``bootstrap/buildout`` steps and you should be all
-set. That means you can now import the namespace ``bob.db.mnist`` into your scripts.
-
-Modify your buildout.cfg and download from git
-==============================================
-
-You will need to add a dependence to `mr.developer
-<http://pypi.python.org/pypi/mr.developer/>`_ to be able to install from our
-git repositories. Your ``buildout.cfg`` file should contain the following
-lines::
-
-  [buildout]
-  ...
-  extensions = mr.developer
-  auto-checkout = *
-  eggs = bob
-         ...
-         bob.db.mnist
-
-  [sources]
-  bob.db.mnist = git https://github.com/bioidiap/bob.db.mnist.git
-  ...
-
-
-How to use this database API
-============================
-
-After launching the python interpreter (assuming that the environment is properly set up),
-you could get the training set as follows::
-
-  >>> import bob.db.mnist
-  >>> db = bob.db.mnist.Database('PATH_TO_DATA_FROM_YANN_LECUN_WEBSITE') # 4 binary .gz compressed files
-  >>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
-
-In this case, this should return two NumPy arrays:
-
-1. `images` contain the raw data (60,000 samples of dimension 784 [28x28 pixels images])
-
-2. `labels` are the corresponding classes (digits 0 to 9) for each of the 60,000 samples
-
-
-If you don't have the data installed on your machine, you can also use the following
-set of commands that will:
-
-1. first look for the database in the bob/db/mnist/ subdirectory and use it if is available
-
-2. automatically download it from Yann Lecun's website into a temporary folder that will
-   be erased when the destructor of the bob.db.mnist database is called.
-
-3. automatically download it into the provided directory that will **not** be deleted.
-
-::
-
-  >>> import bob.db.mnist
-  >>> db = bob.db.mnist.Database() # Check for the data files locally, and download them if required
-  >>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
-  >>> del db # delete the temporary downloaded files if any
-
-or
-
-::
-
-  >>> db = bob.db.mnist.Database("Directory") # Persistently downloads files into the folder "Directory"
-  >>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
-  >>> del db # The download directory stays
 

doc/conf.py

@@ -3,21 +3,13 @@
 # Laurent El Shafey <Laurent.El-Shafey@idiap.ch>
 # Wed May 8 19:05:23 CEST 2013
 #
-# Copyright (C) 2011-2013 Idiap Research Institute, Martigny, Switzerland
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, version 3 of the License.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+# Copyright (C) 2011-2014 Idiap Research Institute, Martigny, Switzerland
+
+import os
+import sys
+import glob
+import pkg_resources
 
-import sys, os
 
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
@@ -71,15 +63,17 @@
 import time
 copyright = u'%s, Idiap Research Institute' % time.strftime('%Y')
 
+# Grab the setup entry
+distribution = pkg_resources.require('bob.db.mnist')[0]
+
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
 # built documents.
 #
 # The short X.Y version.
-from bob.db.mnist.driver import Interface
-version = Interface().version()
+version = distribution.version
 # The full version, including alpha/beta/rc tags.
-release = version
+release = distribution.version
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
@@ -140,12 +134,12 @@
 
 # The name of an image file (relative to this directory) to place at the top
 # of the sidebar.
-html_logo = ''
+html_logo = 'img/logo.png'
 
 # The name of an image file (within the static path) to use as favicon of the
 # docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
 # pixels large.
-html_favicon = ''
+html_favicon = 'img/favicon.ico'
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
@@ -194,7 +188,7 @@
 #html_file_suffix = None
 
 # Output file base name for HTML help builder.
-htmlhelp_basename = 'bobdbdoc'
+htmlhelp_basename = 'bob_db_mnist_doc'
 
 
 # -- Options for LaTeX output --------------------------------------------------
@@ -208,7 +202,7 @@
 # Grouping the document tree into LaTeX files. List of tuples
 # (source start file, target name, title, author, documentclass [howto/manual]).
 latex_documents = [
-  ('index', 'bobdbman.tex', u'Bob',
+  ('index', 'bob_db_mnist.tex', u'Bob',
    u'Biometrics Group, Idiap Research Institute', 'manual'),
 ]
 
@@ -246,40 +240,15 @@
     ('index', 'bob', u'MNIST Database (Bob API) Documentation', [u'Idiap Research Institute'], 1)
 ]
 
-
-# We want to remove all private (i.e. _. or __.__) members
-# that are not in the list of accepted functions
-accepted_private_functions = ['__call__']
-
-def member_function_test(app, what, name, obj, skip, options):
-  # test if we have a private function
-  if len(name) > 1 and name[0] == '_':
-    # test if this private function should be allowed
-    if name not in accepted_private_functions:
-      # omit privat functions that are not in the list of accepted private functions
-      return True
-    else:
-      # test if the method is documented
-      if not hasattr(obj, '__doc__') or not obj.__doc__:
-        return True
-
-  # Skips selected members in auto-generated documentation. Unfortunately, old
-  # versions of Boost.Python will not generate a __self__ member for static
-  # methods and that screws-up Sphinx processing.
-  if sphinx.__version__ < "1.0":
-    # We have to remove objects that do not have a __self__ attribute set
-    import types
-    if isinstance(obj, types.BuiltinFunctionType) and \
-      not hasattr(obj, '__self__') and what == 'class':
-        app.warn("Skipping %s %s (no __self__)" % (what, name))
-        return True
-
-    return False
-
 # Default processing flags for sphinx
 autoclass_content = 'both'
 autodoc_member_order = 'bysource'
-autodoc_default_flags = ['members', 'undoc-members', 'special-members', 'inherited-members', 'show-inheritance']
+autodoc_default_flags = ['members', 'undoc-members', 'inherited-members', 'show-inheritance']
+
+# For inter-documentation mapping:
+from bob.extension.utils import link_documentation
+intersphinx_mapping = link_documentation()
+
 
 def setup(app):
-  app.connect('autodoc-skip-member', member_function_test)
+  pass

doc/guide.rst

@@ -6,5 +6,43 @@
  User's Guide
 ==============
 
+After launching the python interpreter (assuming that the environment is properly set up), you could get the training set as follows:
+
+.. code-block:: py
+
+  >>> import bob.db.mnist
+  >>> db = bob.db.mnist.Database('PATH_TO_DATA_FROM_YANN_LECUN_WEBSITE') # 4 binary .gz compressed files
+  >>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
+
+In this case, this should return two :py:class:`numpy.ndarray`\s:
+
+1. `images` contain the raw data (60,000 samples of dimension 784 [28x28 pixels images])
+
+2. `labels` are the corresponding classes (digits 0 to 9) for each of the 60,000 samples
+
+
+If you don't have the data installed on your machine, you can also use the following set of commands that will:
+
+1. first look for the database in the ``bob/db/mnist`` subdirectory and use it if is available
+
+2. automatically download it from Yann Lecun's website into a temporary folder that will be erased when the destructor of the :py:class:`bob.db.mnist.Database` is called.
+
+3. automatically download it into the provided directory that will **not** be deleted.
+
+.. code-block:: py
+
+  >>> import bob.db.mnist
+  >>> db = bob.db.mnist.Database() # Check for the data files locally, and download them if required
+  >>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
+  >>> del db # delete the temporary downloaded files if any
+
+or:
+
+.. code-block:: py
+
+  >>> db = bob.db.mnist.Database("Directory") # Persistently downloads files into the folder "Directory"
+  >>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
+  >>> del db # The download directory stays
+
 .. todo::
    Write users guide.

doc/index.rst

@@ -2,8 +2,10 @@
 .. @author: Manuel Guenther <Manuel.Guenther@idiap.ch>
 .. @date:   Tue Aug 26 09:42:18 CEST 2014
 
+.. _bob.db.mnist:
+
 ==========================
- MNIST Database Protocols
+ MNIST Database Interface
 ==========================
 
 .. todolist::