Skip to content
Permalink
Browse files

Update Pythran documentation

* Fix several typos / mistakes
* (Manually!) cleanup logo
* Update titles and rendering of several blocks
* Move to Sphinx for static website generation.
  • Loading branch information...
serge-sans-paille committed Feb 20, 2014
1 parent a49b8d0 commit 1398dc3b17f0be647b772e5e17539227458285e4
Showing with 299 additions and 382 deletions.
  1. +3 −34 TODO
  2. +14 −1 doc/CLI.rst
  3. +17 −17 doc/DEVGUIDE.rst
  4. +15 −16 doc/INTERNAL.rst
  5. +14 −13 doc/MANUAL.rst
  6. +3 −3 doc/TUTORIAL.rst
  7. +20 −68 logo.svg
  8. +33 −30 website/Makefile
  9. +180 −0 website/conf.py
  10. +0 −200 website/os.css
37 TODO
@@ -1,6 +1,6 @@
=================
Pythran TODO list
=================
====
TODO
====

Plenty of ideas for the brave! If you want to work on one of these, please
tell ``pythran@freelists.org`` before, as it is good to discuss before coding!
@@ -34,14 +34,6 @@ Support ``str`` module
Because current implementation sucks. Not that important for the kind of code
we target, but still...

Support ``operator`` module
---------------------------

*homework for xmar*

This is an easy one. A good candidate to discover Pythran's runtime internals.


Automatic vectorization of list comprehension
---------------------------------------------

@@ -51,14 +43,6 @@ Because when you see::

you **must** think: what a good vectorization opportunity!

Support singly assigned global variables
----------------------------------------

There is no support for global variables in Pythran, but there is no good point
for it. This is gonna be tough because it impacts many analysis (including
aliasing analysis, memory effects and typing) so only advanced pythraners
should go there.

Support ``import a_user_module``
--------------------------------

@@ -73,14 +57,6 @@ There is an extension mechanism in ``distutils`` to compile native modules when
running ``setup.py``. Pythran should provide a compatibility layer. There is a
first draft in the europar2013 branch, this is an easy go for newcomers.

Jit typing
----------

*warming-up for joker-eph*

Pythran could be called at runtime (with a huge slowdown :() for each function
marked with the appropriate decorator. Pretty good job for a new pythraner too.

Pythran tutorial
----------------

@@ -94,13 +70,6 @@ Named arguments

Named arguments are not supported in Pythran... Yet! Alias aliasing to the rescue!

Automatic generation of (part of) Pythonic
------------------------------------------

Some part of ``pythonic`` are really boring to write. For instance the
``PROXY`` macro calls. Developers should be lazy: there must be a way (think
about it!) to automate part of this work. Good for smart new pythraners!

Performance issues detection
----------------------------

@@ -1,6 +1,11 @@
======================
Command Line Interface
======================

This file shows some use case of Pythran on the command line.

Firstly lets clear the working space::

$> rm -f cli_*

.. Small hack to setup the $PATH in a compatible way
@@ -9,36 +14,43 @@ Firstly lets clear the working space::
.. >>> os.environ['PATH'] = './scripts:' + os.environ['PATH']
One of the most classic use case in Pythran is to generate a native .so module::

$> printf '#pythran export foo()\ndef foo(): print \"hello world\"' > cli_foo.py
$> pythran cli_foo.py
$> ls cli_foo.so
cli_foo.so

The generated native ``.so`` module can then be called with the Python interpreter::

$> python -c 'import cli_foo ; cli_foo.foo()'
hello world

You can choose your optimization level by using ``-O`` flag::

$> rm cli_foo.so
$> pythran cli_foo.py -O2
$> ls cli_foo.so
cli_foo.so

Out of curiosity, you can check the generated output::

$> pythran -E cli_foo.py

That's some heavily templated code ;-) Pythran can then compile it for you to a Python module::

$> pythran cli_foo.cpp

Pythran can also generate raw C++ code, using the ``-e`` switch::

$> pythran -e cli_foo.py -o cli_foo.hpp
$> printf '#include \"cli_foo.hpp\"\nusing namespace __pythran_cli_foo ; int main() { foo()(); return 0 ; }' > cli_foo.cpp
$> g++ cli_foo.cpp `pythran-config --cflags --libs` -o cli_foo
$> ./cli_foo
hello world

You can use ``-p`` option to apply a Pythran optimization. For example, the python
code translated by Pythran can be generated by using python backend::
code translated by Pythran can be generated by using python back-end::

$> pythran -e cli_foo.py -p pythran.backend.Python
import itertools
def foo():
@@ -49,6 +61,7 @@ code translated by Pythran can be generated by using python backend::
__init__()

If you want to specify the path of generated file::

$> pythran cli_foo.py -o /tmp/cli_foo.so
$> ls /tmp/cli_foo.so
/tmp/cli_foo.so
@@ -1,8 +1,8 @@
========================
Pythran Developper Guide
========================
===============
Developer Guide
===============

Do not hang around in Pythran code base without your developper guide! It is
Do not hang around in Pythran code base without your developer guide! It is
the compass that will guide you in the code jungle!

Disclaimer
@@ -14,7 +14,7 @@ Coding Style
------------

All Python code must be conform to the PEP 8, and the ``pep8`` command must not
yield any message when run on our database. Additionnaly, avoid backslashes,
yield any message when run on our database. Additionally, avoid backslashes,
and try to make your code as concise as possible.

C++ code use spaces (no tabs) and a tab width of 4.
@@ -25,18 +25,18 @@ File Hierarchy
Listing the top level directory yields the following entries:

setup.py
The files that describels what gets installed, that holds ``PyPI`` entries
The files that describes what gets installed, that holds ``PyPI`` entries
and such.

doc/
If you're reading this document, you know what it's all about! ``MANUAL``
is the user documentation and ``DEVGUIDE`` is the developper documentation.
is the user documentation and ``DEVGUIDE`` is the developer documentation.

LICENSE
Boring but important stuff.

MANIFEST.in
Describe additionnal stuff to package there.
Describe additional stuff to package there.

README
Quick introduction and description of _pythran_. The ``README.rst`` file is
@@ -49,7 +49,7 @@ pythran/tests/
The source of all issues.

pythran/pythonic/
Where C++ backend lies.
Where C++ back-end lies.

scripts/
Where Python scripts calling the ``pythran`` module lies.
@@ -95,7 +95,7 @@ There are two kinds of tests in ``pythran``:
call the ``run_test(function_to_translate, *effective_parameters,
**name_to_signature)`` method [1]_. It translates ``function_to_translate``
into a native function using the type annotations given in the
``name_to_signature`` dictionnary, runs both the python and the native
``name_to_signature`` dictionary, runs both the python and the native
version with ``effective_parameters`` as arguments and asserts the results
are the same.

@@ -110,7 +110,7 @@ There are two kinds of tests in ``pythran``:
C++ runtime
-----------

The C++ code generated by ``pythran`` relies on a specific backend,
The C++ code generated by ``pythran`` relies on a specific back-end,
``pythonic``. It is a set of headers that mimics Python's intrinsics and
collections behavior in C++. It lies in ``pythran/pythonic/``. There is one
directory per module, e.g. ``pythran/pythonic/numpy`` for the ``numpy`` module,
@@ -120,7 +120,7 @@ and one file per function, e.g. ``pythran/pythonic/numpy/ones.hpp`` for the
must be ``#includ``-able independently, i.e. it itself includes all the type
and function definition it needs. This helps keeping compilation time low.

All pythran functions and types live in the ``pythonic`` namespace. Each extra
All Pythran functions and types live in the ``pythonic`` namespace. Each extra
module defines a new namespace, like ``pythonic::math`` or
``pythonic::random``, and each type is defined in the ``pythonic::types``
namespace. The ``PROXY`` macro from ``pythonic/utils/proxy.hpp`` is commonly
@@ -129,7 +129,7 @@ used to convert functions into functors.
The pythonic runtime can be used without Python support, so it is important to
protect all Python-specific stuff inside ``USE_BOOST_PYTHON`` guard.

All methods are represented by functions in pythran. The associated
All methods are represented by functions in Pythran. The associated
pseudo-modules are prefixed and suffixed by a double underscore ``__``, as in
``pythran/pythonic/__list__``.

@@ -138,7 +138,7 @@ Benchmarking and Testing
------------------------

Stand-alone algorithms are put into ``pythran/tests/cases``. They must be valid
pythran input (including spec annotations). To be taken into account by the
Pythran input (including spec annotations). To be taken into account by the
validation suite, they must be listed in ``pythran/tests/test_cases.py``. To be
taken into account by the benchmarking suite, they must have a line starting
with the ``#runas`` directive. Check ``pythran/tests/matmul.py`` for a complete
@@ -154,10 +154,10 @@ python
Uses the interpreter used to run ``setup.py``.

pythran
Uses the pythran compiler.
Uses the Pythran compiler.

pythran+omp
Uses the pythran compiler in OpenMP mode.
Uses the Pythran compiler in OpenMP mode.

All measurements are made using the ``timeit`` module. The number of iterations
is customizable through the ``--nb-iter`` switch.
@@ -179,7 +179,7 @@ How to
1. Subclass one of ``ModuleAnalysis``, ``FunctionAnalysis`` or ``NodeAnalysis``.
2. List analysis required by yours in the parent constructor, they will be built automatically and stored in the attribute with the corresponding uncameled name.
3. Write your analysis as a regular ``ast.NodeVisitor``. The analysis result must be stored in ``self.result``.
4. Use it either from another pass's constructor, or throught the ``passmanager.gather`` function.
4. Use it either from another pass's constructor, or through the ``passmanager.gather`` function.


:Push changes into the holy trunk:
@@ -1,6 +1,6 @@
=================
Pythran Internals
=================
=========
Internals
=========

This document describes some internals of Pythran compiler.

@@ -147,9 +147,9 @@ When the scope can be attached to an assignment, Pythran uses this piece of info
>>> scopes[foo_tree.body[0].body[1].body[0]]
set(['a'])

Additionnaly, some OpenMP directives, when applied to a single statement, are
Additionally, some OpenMP directives, when applied to a single statement, are
treated by Pythran as if they created a bloc, emulated by a dummy
conditionnal::
conditional::

>>> def foo(n):
... "omp parallel"
@@ -164,8 +164,8 @@ conditionnal::
s = 1
return s

Note that the OpenMP directives are not pretty-printed by the Python backend!
However the additionnal if bloc makes it clear that ``s`` should have function
Note that the OpenMP directives are not pretty-printed by the Python back-end!
However the additional if bloc makes it clear that ``s`` should have function
scope, and the scope is not attached to the first assignment::

>>> scopes = pm.gather(analysis.Scope, foo_tree)
@@ -254,10 +254,10 @@ programs!::
Lazyness
--------

``Expressions templates`` used by numpy internal representation enable lazyness
``Expressions templates`` used by numpy internal representation enable laziness
computation. It means that operations will be computed only during assignation
to avoid intermediate array allocation and improve data locality.
Lazyness analysis enable Expression template even if there is multiple
Laziness analysis enable Expression template even if there is multiple
assignment in some case.

Let's go for some examples.
@@ -268,7 +268,7 @@ for each elements, two operations are apply at once instead of one by one::
... return array * 5 + 3

It also apply for other unary operations with numpy array.
In this example, lazyness doesn't change anything as is it a typical case for
In this example, laziness doesn't change anything as is it a typical case for
Expression templates but peoples may write::

>>> def foo(array):
@@ -309,12 +309,11 @@ In this case, ``b`` can't be lazy so its values is ``inf``::
>>> lazyness
{'a': 1, 'array': 2, 'b': inf}

We can notice that a reassignment reinit its value so even if ``a`` is used twice,
its counters returns ``1``.
``inf`` also happen in case of subscript use as we need to compute the value to
subscript on it. AugAssigned values can't be lazy too and variables used in
loops too.
Lazyness also cares about aliased values::
We can notice that a reassignment reinitializes its value so even if ``a`` is
used twice, its counters returns ``1``. ``inf`` also happen in case of
subscript use as we need to compute the value to subscript on it. Updated
values can't be lazy too and variables used in loops too. Laziness also cares
about aliased values::

>>> def foo(array):
... a = array * 2

0 comments on commit 1398dc3

Please sign in to comment.
You can’t perform that action at this time.