ipython.parallel doesn't discover globals under Python 3.3

[kmike]

I have the latest master installed; this code doesn't work under Python 3.3 in ipython notebook:

from IPython.parallel import Client, interactive

def bar():
    return "bar"

def foo(x):
    return bar()

rf = Client(debug=False)
dview = rf[:] 
dview.map(foo, range(10)).get()

It raises the following exception:

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)<string> in <module>()
/Users/kmike/envs/pymorphy-33/lib/python3.3/site-packages/ipython-0.14.dev-py3.3.egg/IPython/parallel/client/remotefunction.py in func(f, *sequences)
    205                     f = lambda f, *sequences: list(map(f, *sequences))
    206                 else:
--> 207                     f = map
    208                 args = [self.func]+args
    209             else:
<ipython-input-3-702776eb8dec> in foo(x)
NameError: global name 'bar' is not defined

The code works if foo doesn't call bar(). It also works if 'bar' is injected to dview manually (via dview['bar'] = bar)

[dhirschfeld]

I believe that's because your engines (separate python processes) don't have the bar function in their namespaces - you only defined it locally. I think this will work if you push the function to the engine namespaces as follows:

rf = Client(debug=False)
dview = rf[:]
dview.push({'bar': bar})
dview.map_sync(foo, range(10))

[minrk]

Has nothing to do with Python 3, this is just a namespace issue, and it is actually behaving as expected.
foo doesn't have a hard reference to bar, it has a reference to whatever __main__.bar is at call time. The __main__ module is the user namespace, but the user namespace on the engine is not generally the same as the user namespace at the client. For this reason, bar can be different when you call foo locally or remotely.

Basically: sending foo does not send bar.

For instance:

bar = 5

def foo():
    return bar

view = rc[-1]
view['bar'] = 4

foo() # 5
view.apply_sync(foo) # 4

Here's a case where resolving names on the engines is actually the goal:

%px import os
%px pid = os.getpid()

def double_pid():
    return pid * 2

rc[:].apply_sync(double_pid) # returns a list of PIDs (*2 for fun) of each engine

[kmike]

Thanks for the explanation and sorry for noise!

[minrk]

No worries, that part is confusing.