Permalink
Browse files

Adding parallel tutorial notebooks.

  • Loading branch information...
1 parent 296eb30 commit 860113bbdd766ca9a55f7c2376374a7a741267f7 @ellisonbg ellisonbg committed Oct 24, 2012
View
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

Large diffs are not rendered by default.

Oops, something went wrong.

Large diffs are not rendered by default.

Oops, something went wrong.

Large diffs are not rendered by default.

Oops, something went wrong.

Large diffs are not rendered by default.

Oops, something went wrong.

Large diffs are not rendered by default.

Oops, something went wrong.

Large diffs are not rendered by default.

Oops, something went wrong.

Large diffs are not rendered by default.

Oops, something went wrong.

Large diffs are not rendered by default.

Oops, something went wrong.
@@ -0,0 +1,56 @@
+{
+ "metadata": {
+ "name": "Part 9 - Summary"
+ },
+ "nbformat": 3,
+ "nbformat_minor": 0,
+ "worksheets": [
+ {
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Summary of what we have covered\n",
+ "\n",
+ "* Simple remote execution of functions, or statements\n",
+ "* Distribution of data\n",
+ "* Parallel map\n",
+ "* Multiplexed execution\n",
+ "* Load-Balanced task farming\n",
+ "* Asynchronous task submission and result retrieval"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# And further things we have *not* covered\n",
+ "\n",
+ "* Functional Task Dependencies\n",
+ "* Configuring the IPython cluster (working with profiles, SGE, MongoDB, etc.)\n",
+ "* Tuning the IPython Cluster (TaskScheduler.hwm, custom serialization, etc.)\n",
+ "* Inter-engine communication with PyZMQ or MPI\n",
+ "* decorators for parallel and remote functions\n",
+ "* IPython `%magics` for some parallel execution\n",
+ "* Result caching, and task metadata\n",
+ "* The TaskResult Database\n",
+ "* Task resubmission\n",
+ "* Queue monitoring and management\n",
+ "\n",
+ "See the [IPython.parallel docs](http://ipython.org/ipython-doc/dev/parallel) for more detailed coverage of these topics, and check out our [examples](https://www.github.com/ipython/ipython/tree/master/docs/examples/parallel)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": true,
+ "input": [],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ }
+ ],
+ "metadata": {}
+ }
+ ]
+}
View
@@ -0,0 +1,8 @@
+import math
+import numpy
+import sys
+
+a=5
+
+def mysquare(x):
+ return x*x
@@ -0,0 +1,5 @@
+class ItalicStr(str):
+ def _repr_html_(self):
+ return '<i>%s</i>' % self
+
+ItalicStr('Hello World')
View
@@ -0,0 +1,7 @@
+%%file lnum.py
+#!/usr/bin/env python
+import sys
+for i, line in enumerate(sys.stdin.readlines()):
+ print i+1, ':', line,
+
+print '\n---- END ---'
@@ -0,0 +1,21 @@
+# To parallelize every call with map, you just need to get a list for each argument.
+# You can use `itertools.product` + `zip` to get this:
+
+
+import itertools
+
+product = list(itertools.product(widths, heights))
+# [(1, 6), (1, 7), (2, 6), (2, 7), (3, 6), (3, 7)]
+
+# So we have a "list of pairs",
+# but what we really want is a single list for each argument, i.e. a "pair of lists".
+# This is exactly what the slightly weird `zip(*product)` syntax gets us:
+
+allwidths, allheights = zip(*itertools.product(widths, heights))
+
+print " widths", allwidths
+print "heights", allheights
+
+# Now we just map our function onto those two lists, to parallelize nested for loops:
+
+ar = lview.map_async(area, allwidths, allheights)
@@ -0,0 +1,16 @@
+from IPython import parallel
+
+def remote_iterator(view, name):
+ """Return an iterator on an object living on a remote engine."""
+ it_name = '_%s_iter' % name
+ view.execute('%s = iter(%s)' % (it_name,name), block=True)
+ ref = parallel.Reference(it_name)
+ while True:
+ try:
+ yield view.apply_sync(lambda x: x.next(), ref)
+ # This causes the StopIteration exception to be raised.
+ except parallel.RemoteError as e:
+ if e.ename == 'StopIteration':
+ raise StopIteration
+ else:
+ raise e
@@ -0,0 +1,17 @@
+from IPython import display
+
+t_minus = range(10,0,-1)
+
+def lazy_iterator(name):
+ seq = eval(name)
+ it = iter(seq)
+ while True:
+ try:
+ yield it.next()
+ # this looks silly locally, but it will be useful for the remote version:
+ except StopIteration:
+ raise StopIteration
+
+lzit = lazy_iterator('t_minus')
+display(lzit)
+list(lzit)
@@ -0,0 +1,16 @@
+from IPython import parallel
+
+def remote_iterator(view, name):
+ """Return an iterator on an object living on a remote engine."""
+ it_name = '_%s_iter' % name
+ view.execute('%s = iter(%s)' % (it_name,name), block=True)
+ next_ref = parallel.Reference(it_name + '.next')
+ while True:
+ try:
+ yield view.apply_sync(next_ref)
+ # This causes the StopIteration exception to be raised.
+ except parallel.RemoteError as e:
+ if e.ename == 'StopIteration':
+ raise StopIteration
+ else:
+ raise e
View
@@ -0,0 +1,3 @@
+# For an expression made from elementary functions, we must first make it into
+# a callable function, the simplest way is to use the Python lambda construct.
+plot_taylor_approximations(lambda x: 1/cos(x), 0, [2,4,6], (0, 2*pi), (-5,5))
Oops, something went wrong.

0 comments on commit 860113b

Please sign in to comment.