exec() issue when used inside function

[schperplata]

BPO	36426
Nosy	@terryjreedy, @pfmoore, @ncoghlan, @tjguk, @zware, @zooba, @eamanu, @schperplata

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-03-30.11:56:11.897>
created_at = <Date 2019-03-25.17:31:05.785>
labels = ['interpreter-core', 'invalid', 'type-feature', '3.8', '3.9']
title = 'exec() issue when used inside function'
updated_at = <Date 2019-03-30.11:56:11.895>
user = 'https://github.com/schperplata'

bugs.python.org fields:

activity = <Date 2019-03-30.11:56:11.895>
actor = 'ncoghlan'
assignee = 'none'
closed = True
closed_date = <Date 2019-03-30.11:56:11.897>
closer = 'ncoghlan'
components = ['Interpreter Core']
creation = <Date 2019-03-25.17:31:05.785>
creator = 'schperplata'
dependencies = []
files = []
hgrepos = []
issue_num = 36426
keywords = []
message_count = 8.0
messages = ['338812', '338813', '338844', '338845', '338849', '339173', '339183', '339189']
nosy_count = 8.0
nosy_names = ['terry.reedy', 'paul.moore', 'ncoghlan', 'tim.golden', 'zach.ware', 'steve.dower', 'eamanu', 'schperplata']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue36426'
versions = ['Python 3.8', 'Python 3.9']

[schperplata]

I've reported a stack overflow question and no reasonable explation was offered. Here is what I've discovered:
.. code-block:: python
def func():
varName = 'bar'
varValue = 42
localVarToEvaluate = varName + ' = varValue'

    try:
        exec(localVarToEvaluate)
    except Exception as err:
        print(str(err))
    
    if 'bar' in locals():
        # print(locals()['bar']) # (1) OK
        # print(bar)  # (2) ERR
        #print("'bar' OK:", bar)  # (3) ERR
        pass # uncomment any line above

    func()

After exec() is executed, bar can be seen in locals(), but not accessible by intereter. Also, It can be accessed by directly calling print(locals()['bar'](, but not print(bar).

This is the problem as long as the code is wrapped in function. If the same code is placed in the module body, works as expected.

Is there any exaplanation for such behaviour, or is this a bug?

[schperplata]

Seems like I don't know how to write a code here.
Anyway, issue created on stack overflow can be found on:
https://stackoverflow.com/questions/55239875/python-exec-function-broken-in-versions-above-2-7-error-name-not-defined/55240000?noredirect=1#comment97362021_55240000

Works on 2.7, fails on everything above 3.x

[eamanu]

I test on 3.5 and 3.8 running not in an func and I don't have the problem:

Python 3.8.0a2+ (heads/bpo-36287:ba8f342623, Mar 25 2019, 21:57:16) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a = 'bar'
>>> b = 42
>>> c  = a + ' = c'
>>> c
'bar = c'
>>> exec(c)
>>> 'bar' in locals()
True
>>> print(locals()['bar'])
bar = c
>>> print(bar)
bar = c
>>> print("'bar' OK:", bar)
'bar' OK: bar = c

[eamanu]

But I confirmed the behavior reported

uhmm weird

[zooba]

This is currently by design, which means 3.8 is likely the only viable place it can change. It's also not Windows specific so I removed that component (people may remove themselves from nosy).

But +Nick, since I know he has some interest in making locals() behave more consistently. Currently it's basically a read-only proxy, as locals are optimized within functions which is why you can't see updates via the duct.

[terryjreedy]

This is not a bug; the behavior is covered in the docs. I am surprised that this works in 2.7, as it has the same warning as 3.7:
"
locals()
Update and return a dictionary representing the current local symbol table. Free variables are returned by locals() when it is called in function blocks, but not in class blocks.
Note The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter."

[schperplata]

Are there any workarounds - how can we use exec() and on-the-fly created variables?
Also, why 'locals()['bar']' is seen by the interpretter, but not ony 'bar'?

[ncoghlan]

This is not a bug - to enable function level optimisations, the compiler must be able to see all local variable names at compile time.

In Python 2.x the exec statement implementation included special code to allow even function local variables to be rebound, but that special-casing was removed as part of the Python 3 change to convert exec from a statement to a builtin function (as per https://www.python.org/dev/peps/pep-3100/#core-language )

This means that to use exec() and reliably see the changes it makes to a namespace, you have to supply a reliably read/write dictionary of your own.

Object instance dictionaries work well for this purpose, as you can then access the results as attributes on the instance:

>>> class Namespace:
...     pass
... 
>>> def f():
...     ns = Namespace()
...     exec("x = 1; y = 2", vars(ns))
...     print(ns.x)
...     print(ns.y)
... 
>>> f()
1
2