Major doc update + Update all challenges' constructor to add verbose …

…+ download

+ kargs parametrs. Besides, the download parameter allows to retrieve info
without downloading the data.

README.rst

@@ -18,7 +18,10 @@ DREAMTools
    :target: http://dreamtools.readthedocs.org/en/latest/?badge=latest
    :alt: Documentation Status
 
-:Python version: DREAMTools is supported for Python 2.7, 3.4 and 3.5
+:Python version: DREAMTools is supported for Python 2.7, 3.4 and 3.5.
+    Pre-compiled versions are available for Linux and MAC platforms through Anaconda
+    and the **bioconda** channel.
+
 :Note about coverage: We do not run the entire test suite on Travis, which
                       reports a 40% test coverage. Note however, that the actual
                       test coverage is about 80%.
@@ -47,25 +50,24 @@ Motivation
 
 The main goals of **DREAMTools** are to provide:
 
-#. scoring functions equivalent to those used during past DREAM challenges for **end-users** via a standalone application (called **dreamtools**).
-#. a common place for **developers** involved in the DREAM challenges to share code
+#. Scoring functions equivalent to those used during past DREAM challenges for **end-users** via a standalone application (called **dreamtools**).
+#. A common place for **developers** involved in the DREAM challenges to share code
 
 **DREAMTools** does not provide code related to aggregation,
 leaderboards, or more complex analysis even though such code
 may be provided (e.g., in D8C1 challenge).
 
-Note that many scoring functions requires data hosted on `Synapse <www.synapse.org>`_ . We therefore strongly encourage you to **register to Synapse**. Depending on the challenge, you may be requested to accept terms of agreements to use the data.
+Note that many scoring functions requires data hosted on `Synapse <http://www.synapse.org>`_ . We therefore strongly encourage you to **register to Synapse**. Depending on the challenge, you may be requested to accept terms of agreements to use the data.
 
 Installation
 -----------------
 
-The first method requires the **pip** tools to be installed and compilation of
-dependencies::
+For those familiar with Python, you may use the `pip executable <https://pypi.python.org/pypi/pip>`_ provided with Python. It will install the latest release of **DREAMTools** and the dependencies::
 
     pip install cython
     pip install dreamtools
 
-If you are not familiar with compilation and Python, you may use `conda <https://www.continuum.io/downloads>`_::
+If you are not familiar with compilation and/or Python, you may use `conda <https://www.continuum.io/downloads>`_ since we have pre-compiled packages with a conda channel called **bioconda**::
 
     conda config --add channels r
     conda config --add channels bioconda

doc/source/changelog.rst

@@ -2,15 +2,24 @@ ChangeLog
 ==============
 
 
-1.2.6
--------
+1.3.0 16/03/2016
+---------------------
+
+* stable version synchronised with F1000 v2 paper.
+* add a new notebook example (D9C1)
+* all Challenge constructor have now the following parameters: verbose, download
+  and **kargs so that if another parameter is added, no code will need to be
+  changed. 
+* documentation added for D7C4 to install Perl dependencies
 
-:2 March 2016:
+1.2.6 02/03/2016
+-------------------------
 
 * CHANGES: 
     - since dreamtools is now in bioconda, we removed the installation
       scripts conda_install.bat and conda_install.sh. 
     - Doc updates.
+    - conda recipes in ./conda removed (now in bioconda-recipes project)
 
 
 1.2.5

doc/source/core.rst

@@ -4,6 +4,13 @@ CORE modules
 .. contents::
 
 
+Challenge
+----------------
+
+.. automodule:: dreamtools.core.challenge
+    :members:
+    :undoc-members:
+
 
 Rocs
 -----------

doc/source/credits.rst

@@ -1,7 +1,7 @@
 Credits
 ----------
 
-Contributions to **DREAMTools** are direct (e.g. by contributing to the code in  the github repository http://github.com/dreamtools/dreamtools) or indirectly (e.g., by developing an original scoring functions in DREAM challenges). Since **DREAMTools** framework is dynamic by nature, we won't keep a list of contributors, which would be outdated quickly. Instead, we recommend to check the scoring module themselves (e.g., dreamtools/dream5/D5C2/scoring.py) or the github log or the DREAM challenges website (http://dreamchallenges.org).
+Contributions to **DREAMTools** are direct (e.g. by contributing to the code in  the github repository http://github.com/dreamtools/dreamtools) or indirectly (e.g., by developing an original scoring functions in DREAM challenges). Since **DREAMTools** framework is dynamic by nature, we won't keep a list of contributors, which would be outdated quickly. Instead, we recommend to check the scoring module themselves (e.g., dreamtools/dream5/D5C2/scoring.py) or the github log, the DREAM website (http://dreamchallenges.org), or the Synapse web of the challenge itself.
 
 
 

doc/source/developers.rst

@@ -19,7 +19,7 @@ called **__init__.py**. Then, all you need to do is to go to *dreamtools/dream8*
 
     dreamtools-layout --challenge-name D8C4
 
-Some sub directories and files are created including the **scoring.py** with a basic class were to code or wrap your scoring function.
+Some sub directories and files are created including the **scoring.py** with a basic class where to code or wrap your scoring function.
 
 
 If data file or templates are too large, we strongly recommend to store them in a project on Synapse. We have created a synapse project called `dreamtools <https://www.synapse.org/#!Synapse:syn4483180>`_
@@ -58,9 +58,9 @@ Basic Structure of the Challenge class
         Data and templates are downloaded from Synapse. You must have a login.
 
         """
-        def __init__(self):
+        def __init__(self, verbose=True, download=True, **kargs):
             """.. rubric:: constructor"""
-            super(D7C4, self).__init__('D7C4')
+            super(D7C4, self).__init__('D7C4', verbose, download, **kargs)
             self._init()
             # if several sub-challenges, name them here
             self.sub_challenges = []
@@ -84,10 +84,16 @@ Basic Structure of the Challenge class
 Storing large files
 ---------------------
 
-In the example above, templates and goldstandard locations are in the directory
-of DREAMTools. Using the methods :meth:`getpath_template` automatically found
-the path of directory and one simply needs to provide the filename. However,
-large files should not be stored in DREAMTools repository but on synapse in
+Templates and gold standards are either stored within the **DREAMTools**
+package or, if there are too large, on the Synapse web site. In the latter
+case, the files will be downloaded on request (only once). You should therefore
+not change those files, which are located in the **DREAMTools** directory 
+(e.g., /home/user/.config/dreamtools). The downloaded files are stored in
+specific directories. For instance, files related to the D9C1 challenge are 
+stored in /home/user/.config/dreamtools/dream9/D9C1.
+
+So, as developers, you should also figure out if a file should be stored in
+Synapse or not. Large files are currently stored in this synapse page
 `dreamtools <https://www.synapse.org/#!Synapse:syn4483180>`_
 
 If your class inherits from :class:`dreamtools.core.challenge.Challenge`,
@@ -98,9 +104,8 @@ local location::
                 'syn2898469')
 
 
-
-All files downloaded from Synapse are then stored in a local directory. For
-instance in /home/user/.config/dreamtools/dream5/D5C2 for the example above.
+In this example, the file *DREAM5_GoldStandard_probes.zip* is stored in this
+directory: /home/user/.config/dreamtools/dream5/D5C2 for the example above.
 
 
 License/header

doc/source/header.txt

@@ -3,7 +3,7 @@
 #
 #  This file is part of DREAMTools software
 #
-#  Copyright (c) 2015, DREAMTools Development Team
+#  Copyright (c) 2015-2016, DREAMTools Development Team
 #  All rights reserved
 #
 #  Distributed under the BSD 3-Clause License.

doc/source/index.rst

@@ -18,12 +18,14 @@ DREAMTools
    :target: http://dreamtools.readthedocs.org/en/latest/?badge=latest
    :alt: Documentation Status
 
-:Python version: DREAMTools is supported for Python 2.7, 3.4 and 3.5
+:Python version: DREAMTools is supported for Python 2.7, 3.4 and 3.5.
+    Pre-compiled versions are available for Linux and MAC platforms through Anaconda
+    and the **bioconda** channel.
+
 :Note about coverage: We do not run the entire test suite on Travis, which
                       reports a 40% test coverage. Note however, that the actual
                       test coverage is about 80%.
-:Contributions: Please join https://github.com/dreamtools/dreamtools and share your notebooks https://github.com/dreamtools/dreamtools/notebooks
-
+:Contributions: Please join https://github.com/dreamtools/dreamtools
 :Online documentation: `On readthedocs <http://dreamtools.readthedocs.org/>`_
 :Issues and bug reports: `On github <https://github.com/dreamtools/dreamtools/issues>`_
 :How to cite: Cokelaer T, Bansal M, Bare C et al. DREAMTools: a Python
@@ -48,23 +50,25 @@ Motivation
 
 The main goals of **DREAMTools** are to provide:
 
-#. scoring functions equivalent to those used during past DREAM challenges for **end-users** via a standalone application (called **dreamtools**).
-#. a common place for **developers** involved in the DREAM challenges to share code
+#. Scoring functions equivalent to those used during past DREAM challenges for **end-users** via a standalone application (called **dreamtools**).
+#. A common place for **developers** involved in the DREAM challenges to share code
 
 **DREAMTools** does not provide code related to aggregation,
 leaderboards, or more complex analysis even though such code
 may be provided (e.g., in D8C1 challenge).
 
-Note that many scoring functions requires data hosted on `Synapse <www.synapse.org>`_ . We therefore strongly encourage you to **register to Synapse**. Depending on the challenge, you may be requested to accept terms of agreements to use the data.
+Note that many scoring functions requires data hosted on `Synapse <http://www.synapse.org>`_ . We therefore strongly encourage you to **register to Synapse**. Depending on the challenge, you may be requested to accept terms of agreements to use the data.
 
 Installation
 -----------------
 
-::
+For those familiar with Python, you may use the `pip executable <https://pypi.python.org/pypi/pip>`_ provided with Python. It will installed the
+latest release and the dependencies::
 
+    pip install cython
     pip install dreamtools
 
-or with conda::
+If you are not familiar with compilation and/or Python, you may use `conda <https://www.continuum.io/downloads>`_ since we have pre-compiled packages with a conda channel called **bioconda**::
 
     conda config --add channels r
     conda config --add channels bioconda
@@ -74,7 +78,10 @@ See :ref:`Installation` section for details.
 
 Usage
 ------------
-**DREAMTools** can be used by developers as a Python package::
+
+Every DREAM challenge is different. We will not explain here the scientific goal
+but show how one could score its own prediction. Developers can you **DREAMTools** 
+as a Python package::
 
     >>> from dreamtools import D6C3
     >>> s = D6C3()
@@ -85,11 +92,11 @@ Usage
     Pearson(Cp)      0.647516
     dtype: float64}
 
-A standalone application can be used from a terminal. The executable is called **dreamtools**. Here is an example::
+Besides, a standalone application can be used from a terminal. The executable is called **dreamtools**. Here is an example::
 
     dreamtools --challenge D6C3 --submission path_to_a_file
 
-See below for more details about the usage of the standalone application.
+See :ref:`userguide` for more details about the usage of the standalone application.
 
 
 
@@ -103,10 +110,11 @@ Available challenges, templates and gold standards
 **DREAMTools** includes about 80% of DREAM challenges from DREAM2 to DREAM9.5
 Please visit `F1000 link <http://f1000research.com/articles/4-1030/v1>`_  (Table 1).
 
-
-All gold standards and templates are retrieved automatically. You can obtain the location of a gold standard file as follows::
+All gold standards and templates are retrieved automatically. Once downloaded, you 
+can obtain the location of a gold standard or template as follows::
 
     dreamtools --challenge D6C3 --download-gold-standard
+    dreamtools --challenge D6C3 --download-template
 
 
 Full documentation

doc/source/installation.rst

@@ -10,16 +10,16 @@ Familiar with Python ecosystem ?
 -----------------------------------
 
 If you are familiar with Python and the **pip** application and your system
-is already configured (compilers, development libraries available)), these
+is already configured (compilers, development libraries available), these
 two commands should install **DREAMTools** and its dependencies (in unix or
 windows terminal)::
 
     pip install cython
     pip install dreamtools
 
-If you do not have dependencies installed yet (e.g pandas, numpy, scipy), this
-make take a while (e.g., 10-15 minutes). If you are in a hurry, see the Anaconda
-solution here below.
+If you do not have dependencies installed yet (e.g., pandas, numpy, scipy), this
+may take a while depending on your system (typically 10-15 minutes). If you are 
+in a hurry or do not want to compile libraries, see the Anaconda solution here below.
 
 If you are new to Python
 -----------------------------------
@@ -30,11 +30,12 @@ use the `Anaconda <https://www.continuum.io/downloads>`_ solution.
 
 Anaconda is a free Python distribution. It includes most popular Python packages
 for science and data analysis and has dedicated channels. One such channel is
-called **bioconda** and complements the default channel (conda) with a set of 
-packages dedicated to life science.
+called `bioconda >https://bioconda.github.io/>`_ and complements the default
+channel (conda) with a set of packages dedicated to life science.
 
-We have included **DREAMTools** in **bioconda**. So, once Anaconda is installed, 
-you first need to add **bioconda** channel to your environment (and R)::
+We have included **DREAMTools** in **bioconda** channel. So, once Anaconda is installed,
+you first need to add the **bioconda** channel to your environment (and R
+channel)::
 
     conda config --add channels r
     conda config --add channels bioconda
@@ -44,10 +45,8 @@ This should be done only once. Then, install **DREAMTools** itself::
     conda install dreamtools
 
 This command should install **DREAMTools** in your default conda environment. If
-you wish
-to try **DREAMTools** in another environment (e.g different python version), you
-would need to create a new one and then install **DREAMTools** in that
-environment::
+you wish to try **DREAMTools** in another (independent) environment (e.g., a 
+different python version), you would need to create and activate the environment first::
 
     conda create --name test_dreamtools python=3.5
     source activate test_dreamtools
@@ -58,44 +57,30 @@ environment::
 Installation from source
 -----------------------------------
 
-The command::
-
-    pip install dreamtools
-
-install the latest release of **DREAMTools**. If you prefer to use the
-source code, you can also get     the github repository and install
-**DREAMTools** as
-follows (dependencies such as numpy or scipy will need to be compiled if
-not found)::
-
+The previous methods relies on released versions of **DREAMTools**. If a new
+feature is only available in the source code, then you will need to get the
+source code, which is available in the github repository::
 
    git clone git@github.com:dreamtools/dreamtools.git
    cd dreamtools
    python setup.py install
 
+Dependencies (e.g. Pandas) will need to be compiled or pre-installed (see
+above).
 
 
-Note for Windows
+Note for Windows and Anaconda
 -----------------------------------
 
-If you decide to compile the source yourself under windows, you will 
-have to install a compiler that is compatible with what
-was used by Anaconda to compile the libraries such as numpy. This should not
-be a worry under Linux or Mac platforms. However, under Windows, pre-compiled 
-packages (e.g., Cython) used a specific version of 
-a compiler (http://docs.continuum.io/anaconda/faq#how-did-you-compile-cpython).
-
-It appears to be Visual Studio version 2008 for Python 2.7 and is provided by Microsoft (http://www.microsoft.com/en-us/download/details.aspx?id=44266) for free. However, for python3, there is no specific compiled provided (Jan 2016). If you still want to go for Python3, you should get Visual C version 2010 (http://stackoverflow.com/questions/29909330/microsoft-visual-c-compiler-for-python-3-4).
+We do not provide any DREAMTools package on **bioconda** for Windows.
 
-From Anaconda documentation, Cython was compiled under windows with (March 2016):
+However, if you use Anaconda and decide to compile the source yourself under Windows, then you will
+have to install a compiler that is compatible with Anaconda. In other words, you will have to use the same compiled as the one used by Anaconda.
 
-#. Visual Studio 2008 for Python 2.6 and 2.7
-#. VS 2010 for Python 3.3 and 3.4
-#. VS 2015 for Python 3.5
 
-Even though you do not have Microsoft Visual Studio, you can get the Visual Redistributable Packages that install run-time components that are required to tun C++ applications built using Visual Studio. 
+For Python 2.7, compilation should work easily. You need to know that pre-compiled  packages (e.g., Cython) used a specific version of a compiler (http://docs.continuum.io/anaconda/faq#how-did-you-compile-cpython), which is Visual Studio version 2008  and is provided by Microsoft (http://www.microsoft.com/en-us/download/details.aspx?id=44266) for free.
 
-For VS2015, see http://www.microsoft.com/en-us/download/details.aspx?id=44266
+For Python3.4 and 3.5, this is a bit more difficult. You should get Visual C version 2010 (http://stackoverflow.com/questions/29909330/microsoft-visual-c-compiler-for-python-3-4) or for Python 3.5 another Visual C version 2015. This may change with time but this information was found on Anaconda documentation (March 2016). You may found useful information here as well for VS2015:  http://www.microsoft.com/en-us/download/details.aspx?id=44266
 
 
 Note for Python2.X and Python3.X

doc/source/references.rst

@@ -1,3 +1,5 @@
+.. _references:
+
 References
 ==============