Permalink
Browse files

update mpi doc

  • Loading branch information...
minrk committed Mar 11, 2011
1 parent 33bd9f2 commit 194772e65f9a7ecb9335b41836674cb5a63bdd8c
Showing with 24 additions and 56 deletions.
  1. +22 −53 docs/source/parallelz/parallel_mpi.txt
  2. +2 −3 docs/source/parallelz/parallel_process.txt
@@ -68,17 +68,17 @@ Manual starting using :command:`mpiexec`
If you want to start the IPython engines using the :command:`mpiexec`, just
do::
- $ mpiexec -n 4 ipengine --mpi=mpi4py
+ $ mpiexec -n 4 ipenginez --mpi=mpi4py
This requires that you already have a controller running and that the FURL
files for the engines are in place. We also have built in support for
PyTrilinos [PyTrilinos]_, which can be used (assuming is installed) by
starting the engines with::
- mpiexec -n 4 ipengine --mpi=pytrilinos
+ $ mpiexec -n 4 ipenginez --mpi=pytrilinos
Automatic starting using PBS and :command:`ipclusterz`
------------------------------------------------------
+------------------------------------------------------
The :command:`ipclusterz` command also has built-in integration with PBS. For
more information on this approach, see our documentation on :ref:`ipclusterz
@@ -108,9 +108,14 @@ distributed array. Save the following text in a file called :file:`psum.py`:
op=MPI.SUM)
return rcvBuf
-Now, start an IPython cluster in the same directory as :file:`psum.py`::
+Now, start an IPython cluster::
- $ ipclusterz mpiexec -n 4
+ $ ipclusterz start -p mpi -n 4
+
+.. note::
+
+ It is assumed here that the mpi profile has been set up, as described :ref:`here
+ <parallel_process>`.
Finally, connect to the cluster and use this function interactively. In this
case, we create a random array on each engine and sum up all the random arrays
@@ -120,61 +125,25 @@ using our :func:`psum` function:
In [1]: from IPython.zmq.parallel import client
- In [2]: c = client.Client()
-
- In [3]: mec.activate()
+ In [2]: %load_ext parallel_magic
- In [4]: px import numpy as np
- Parallel execution on engines: all
- Out[4]:
- <Results List>
- [0] In [13]: import numpy as np
- [1] In [13]: import numpy as np
- [2] In [13]: import numpy as np
- [3] In [13]: import numpy as np
+ In [3]: c = client.Client(profile='mpi')
- In [6]: px a = np.random.rand(100)
- Parallel execution on engines: all
- Out[6]:
- <Results List>
- [0] In [15]: a = np.random.rand(100)
- [1] In [15]: a = np.random.rand(100)
- [2] In [15]: a = np.random.rand(100)
- [3] In [15]: a = np.random.rand(100)
+ In [4]: view = c[:]
- In [7]: px from psum import psum
- Parallel execution on engines: all
- Out[7]:
- <Results List>
- [0] In [16]: from psum import psum
- [1] In [16]: from psum import psum
- [2] In [16]: from psum import psum
- [3] In [16]: from psum import psum
+ In [5]: view.activate()
- In [8]: px s = psum(a)
- Parallel execution on engines: all
- Out[8]:
- <Results List>
- [0] In [17]: s = psum(a)
- [1] In [17]: s = psum(a)
- [2] In [17]: s = psum(a)
- [3] In [17]: s = psum(a)
+ # run the contents of the file on each engine:
+ In [6]: view.run('psum.py')
- In [9]: px print s
- Parallel execution on engines: all
- Out[9]:
- <Results List>
- [0] In [18]: print s
- [0] Out[18]: 187.451545803
+ In [6]: px a = np.random.rand(100)
+ Parallel execution on engines: [0,1,2,3]
- [1] In [18]: print s
- [1] Out[18]: 187.451545803
+ In [8]: px s = psum(a)
+ Parallel execution on engines: [0,1,2,3]
- [2] In [18]: print s
- [2] Out[18]: 187.451545803
-
- [3] In [18]: print s
- [3] Out[18]: 187.451545803
+ In [9]: view['s']
+ Out[9]: [187.451545803,187.451545803,187.451545803,187.451545803]
Any Python code that makes calls to MPI can be used in this manner, including
compiled C, C++ and Fortran libraries that have been exposed to Python.
@@ -274,8 +274,7 @@ connections on all its interfaces, by adding in :file:`ipcontrollerz_config`:
.. sourcecode:: python
- c.HubFactory.client_ip = '*'
- c.HubFactory.engine_ip = '*'
+ c.RegistrationFactory.ip = '*'
You can now run the cluster with::
@@ -447,7 +446,7 @@ any point in the future.
To do this, the only thing you have to do is specify the `-r` flag, so that
the connection information in the JSON files remains accurate::
- $ ipcontrollerz -r --regport 12345
+ $ ipcontrollerz -r
Then, just copy the JSON files over the first time and you are set. You can
start and stop the controller and engines any many times as you want in the

0 comments on commit 194772e

Please sign in to comment.