NameError Issue in Multiprocessing

[frobnitzem]

BPO	14704
Nosy	@mdickinson

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-05-01.19:16:35.589>
created_at = <Date 2012-05-01.18:56:30.181>
labels = ['invalid']
title = 'NameError Issue in Multiprocessing'
updated_at = <Date 2012-05-01.19:16:35.587>
user = 'https://bugs.python.org/frobnitzem'

bugs.python.org fields:

activity = <Date 2012-05-01.19:16:35.587>
actor = 'mark.dickinson'
assignee = 'none'
closed = True
closed_date = <Date 2012-05-01.19:16:35.589>
closer = 'mark.dickinson'
components = ['None']
creation = <Date 2012-05-01.18:56:30.181>
creator = 'frobnitzem'
dependencies = []
files = []
hgrepos = []
issue_num = 14704
keywords = []
message_count = 2.0
messages = ['159764', '159766']
nosy_count = 2.0
nosy_names = ['mark.dickinson', 'frobnitzem']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue14704'
versions = ['Python 2.6']

[frobnitzem]

Python Devs,

There is an issue relating to variable lookup using exec from within multiprocessing's fork()-ed process. I'm attempting to use the forked process as a generic remote python shell, but exec is unable to reach variables from within functions. This issue makes it impossible to define a function which uses un-passed variables defined in the remote process.

The simplest way to reproduce the error is:

--- err.py ---

from multiprocessing import Process, Pipe

def run_remote(con, name):
  my_name = name
  for i in range(2):
    code = con.recv()
    exec code

me, he = Pipe()
p = Process(target=run_remote,
args=(he, "Sono Inglese de Gerrards Cross."))
p.start()

me.send("print my_name") # works

me.send("""
def show_name():
print my_name
show_name() # doesn't work
""")
--- end err.py ---

This program prints:
$ python2.6 err.py
Sono Inglese de Gerrards Cross.
Process Process-1:
Traceback (most recent call last):
  File "/sw/lib/python2.6/multiprocessing/process.py", line 232, in _bootstrap
    self.run()
  File "/sw/lib/python2.6/multiprocessing/process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
  File "err.py", line 7, in run_remote
    exec code
  File "<string>", line 4, in <module>
  File "<string>", line 3, in show_name
NameError: global name 'my_name' is not defined

I'm using Mac OSX (10.6.8) and
Python 2.6.5 (r265:79063, Sep 23 2010, 14:05:02)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin

The issue (with the same traceback) also occurs for:
Python 2.7 (r27:82500, Sep 29 2010, 15:34:46)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin

Using exactly the same set of exec calls locally results in the correct behavior.

--- noerr.py ---

my_name = "Sono Inglese de Gerrards Cross."
exec "print my_name"

exec """
def show_name():
print my_name
show_name()
"""
--- end noerr.py ---

[mdickinson]

Thanks for the report.

This is expected behaviour. It isn't actually anything to do with multiprocessing; it's to do with invoking exec from within a function scope. You can see the same effect with code like this:

code = """\
def show_name():
    print my_name
show_name()
"""

def run():
    my_name = "me"
    exec code

run()

See

http://docs.python.org/reference/executionmodel.html#interaction-with-dynamic-features

for more explanation.