Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

DOC: Beautify/extend docs

  • Loading branch information...
commit 134dbc494257ad0d90b0a9efd3423c103450d109 1 parent d1d161d
@GaelVaroquaux GaelVaroquaux authored
View
1  .gitignore
@@ -29,5 +29,6 @@ dist
doc/documentation.zip
doc/generated
doc/CHANGES.rst
+doc/README.rst
# Coverage report
.coverage
View
4 CHANGES.rst
@@ -1,5 +1,5 @@
-Changes
-=========
+Latest changes
+===============
Release 0.4.4
----------------
View
27 README.rst
@@ -1,6 +1,3 @@
-Joblib
-=============
-
The homepage of joblib with user documentation is located on:
http://packages.python.org/joblib/
@@ -15,11 +12,17 @@ To get the latest code using git, simply type::
If you don't have git installed, you can download a zip or tarball
of the latest code: http://github.com/joblib/joblib/archives/master
-Development
-===================
+Installing
+=========================
+
+As any Python packages, to install joblib, simply do::
+
+ python setup.py install
+
+in the source code directory.
Workflow to contribute
-----------------------------
+=========================
To contribute to joblib, first create an account on `github
<http://github.com/>`_. Once this is done, fork the `joblib repository
@@ -30,7 +33,7 @@ on several computer, and when you are happy with them, send a pull
request to the main repository.
Running the test suite
-------------------------
+=========================
To run the test suite, you need nosetests and the coverage modules.
Run the test suite using::
@@ -41,7 +44,7 @@ from the root of the project.
Building the docs
-----------------------
+=========================
To build the docs you need to have setuptools and sphinx (>=0.5) installed.
Run the command::
@@ -52,7 +55,7 @@ The docs are built in the build/sphinx/html directory.
Making a source tarball
-----------------------------
+=========================
To create a source tarball, eg for packaging or distributing, run the
following command::
@@ -65,7 +68,7 @@ no extra dependencies than the Python standard library. You will need
setuptool and sphinx.
Making a release and uploading it to PyPI
-------------------------------------------
+==================================================
This command is only run by project manager, to make a release, and
upload in to PyPI::
@@ -73,7 +76,7 @@ upload in to PyPI::
python setup.py sdist bdist_egg register upload
Updating the changelog
-------------------------
+========================
Changes are listed in the CHANGES.rst file. They must be manually updated
but, the following git command may be used to generate the lines::
@@ -81,7 +84,7 @@ but, the following git command may be used to generate the lines::
git log --abbrev-commit --date=short --no-merges --sparse
Licensing
-==========
+----------
joblib is **BSD-licenced** (3 clause):
View
1  doc/conf.py
@@ -215,4 +215,5 @@
# Hack to copy the CHANGES.rst file
import shutil
shutil.copyfile('../CHANGES.rst', 'CHANGES.rst')
+shutil.copyfile('../README.rst', 'README.rst')
View
9 doc/developing.rst
@@ -0,0 +1,9 @@
+
+===============
+Development
+===============
+
+.. include:: README.rst
+
+.. include:: CHANGES.rst
+
View
6 doc/index.rst
@@ -21,6 +21,10 @@
text-align: center;
font-size: 270% ;
}
+
+ div.bodywrapper blockquote {
+ margin: 0 ;
+ }
</style>
@@ -44,7 +48,7 @@ User manual
installing.rst
memory.rst
parallel.rst
- CHANGES.rst
+ developing.rst
Module reference
-----------------
View
78 joblib/__init__.py
@@ -8,8 +8,9 @@
3. logging and tracing of the execution
-Joblib is optimized to be **fast** and **robust** in particular on large,
-long-running functions and has specific optimizations for `numpy` arrays.
+Joblib is optimized to be **fast** and **robust** in particular on large
+data and has specific optimizations for `numpy` arrays. It is
+**BSD-licensed**.
============================== ==========================================
@@ -22,46 +23,41 @@
**Report issues**: http://github.com/joblib/joblib/issues
============================== ==========================================
-Joblib is **BSD-licensed**.
Vision
--------
-Joblib came out of long-running data-analysis Python scripts. The long
-term vision is to provide tools to achieve better
-reproducibility when running jobs, without changing the way
-code looks like. However, Joblib can also be used to provide a
-light-weight make replacement.
-
-The main problems identified are:
-
- 1) **Lazy evaluation:** People need to rerun over and over the same
- script as it is tuned, but end up commenting out steps, or
- uncommenting steps, as they are needed, as they take long to run.
-
- 2) **Persistence:** It is difficult to persist in an efficient way
- arbitrary objects containing large numpy arrays. In addition,
- hand-written persistence to disk does not link easily the file on
- disk to the corresponding Python object it was persists from in the
- script. This leads to people not a having a hard time resuming the
- job, eg after a crash and persistence getting in the way of work.
-
-The approach taken by Joblib to address these problems is not to build a
-heavy framework and coerce user into using it (e.g. with an explicit
-pipeline). It strives to leave your code and your flow control as
-unmodified as possible.
-
-`More ... <http://packages.python.org/joblib/why.html>`__
-
-Current features
+The vision is to provide tools to easily achieve better performance and
+reproducibility when working with long running jobs. In addition, Joblib
+can also be used to provide a light-weight make replacement or caching
+solution.
+
+ 1) **Avoid computing twice the same thing**: code is rerun over an
+ over, for instance when prototyping computational-heavy jobs (as in
+ scientific development), but hand-crafted solution to aleviate this
+ issue is error-prone and often leads to unreproducible results
+
+ 2) **Persist to disk transparently**: persisting in an efficient way
+ arbitrary objects containing large data is hard. In addition,
+ hand-written persistence does not link easily the file on disk to the
+ execution context of the original Python object. As a result, it is
+ challenging to resume a application status or computational job, eg
+ after a crash.
+
+It strives to address these problems while **leaving your code and your
+flow control as unmodified as possible** (no framework, no new
+paradigms).
+
+Main features
------------------
-1) **Transparent and fast disk-caching of output value:** a make-like
- functionality for Python functions that works well for arbitrary
- Python objects, including very large numpy arrays. The goal is
- to separate operations in a set of steps with well-defined inputs and
- outputs, that are saved and reran only if necessary, by using standard
- Python functions::
+1) **Transparent and fast disk-caching of output value:** a memoize or
+ make-like functionality for Python functions that works well for
+ arbitrary Python objects, including very large numpy arrays. Separate
+ persistence and flow-execution logic from domain logic or algorithmic
+ code by writing the operations as a set of steps with well-defined
+ inputs and outputs: Python functions. Joblib can save their
+ computation to disk and rerun it only if necessary::
>>> from joblib import Memory
>>> mem = Memory(cachedir='/tmp/joblib')
@@ -95,16 +91,6 @@
display streams, and provide a way of compiling a report.
We want to be able to quickly inspect what has been run.
-Development
--------------
-
-The code is `hosted <http://github.com/joblib/joblib>`_ on github.
-It is easy to clone the project and experiment with making your own
-modifications. If you need extra features, don't hesitate to contribute
-them.
-
-`More ... <http://github.com/joblib/joblib/blob/master/README.rst>`__
-
..
>>> import shutil ; shutil.rmtree('/tmp/joblib/')
Please sign in to comment.
Something went wrong with that request. Please try again.