list comprehension expression raises "TypeError: 'NoneType' object is not subscriptable"

[davidemoro]

Tested with Python 3.6.1 with RestrictedPython 4.0b2:

(Pdb) RestrictionCapableEval('[item for item in [1,2]]').eval(context)
*** TypeError: 'NoneType' object is not subscriptable
(Pdb) [item for item in [1,2]]
[1, 2]

where context is:

(Pdb) context
{'variables': {'test_run_identifier': 'QA-240dc1b5-f6bd-11e7-888e-080027edbcd8-skin1', 'skin': 'skin1', 'datetime': '2018-01-11T11:49:48.439194', 'base_url': 'http://'}, 'len': <built-in function len>, 'list': <class 'list'>, 'match': <function match at 0x7f2b3c55bd90>}

Any suggestion?

Thanks in advance

[davidemoro]

Do you think it is a bug or list comprehensions are not allowed? Is there any workaround?

[loechel]

Hi @davidemoro I am not sure what you try to archive.

please follow the documentation at http://restrictedpython.readthedocs.io/en/latest/

I guess what you try to get is:

from RestrictedPython import compile_restricted_eval
from RestrictedPython import safe_builtins

byte_code = compile_restricted_eval( '[item for item in [1,2]]')
eval(byte_code, safe_builtins, {})

that should work

[davidemoro]

Hi @loechel,

thanks.

I think there are some issues with python3.6 as you can see:

(Pdb) eval(compile_restricted_eval('1+1'), safe_builtins, {})                          
*** TypeError: eval() arg 1 must be a string, bytes or code object
(Pdb) eval(compile_restricted_eval('1+1').code, safe_builtins, {})                     
2
(Pdb) eval(compile_restricted_eval('[item for item in [1,2]]').code, safe_builtins, {})
*** NameError: name '_getiter_' is not defined
(Pdb) RestrictionCapableEval('1+1').eval(context)
2
(Pdb) RestrictionCapableEval('[item for item in [1,2]]').eval(context)
*** TypeError: 'NoneType' object is not subscriptable

so it seems that using both methods list comprehensions doesn't work with Python3.6.
Meantime I'm suggesting my users to use a combination of map/filter instead of using list comprehensions.

Also reading the documentation I see that with RestrictionCapableEval you can avoid to import and use the compile_restricted_eval, safe_builtins and it works fine with both python2.7 and 3.6:

• https://github.com/tierratelematics/play_python/blob/62e4220ceead408bacee7409cc6f104a126f406e/play_python/providers.py#L127

[icemac]

@davidemoro I added a fix for RestrictionCapableEval in #97.
But be advised that RestrictionCapableEval.eval() is not meant to be used as is, see
https://github.com/zopefoundation/RestrictedPython/blob/master/src/RestrictedPython/Eval.py#L96-L97. You should create a subclass of RestrictionCapableEval and use functions for _getitem_, _getattr_ and _getiter_ which actually do at least some restrictions.

PS.: This is not a new issue in RestrictedPython 4. In 3.6 I got the same exception.

[davidemoro]

Hi @icemac, @loechel,

thanks :)

Just a curiosity (sorry, I don't know the technological details of RestrictedPython): I still don't understand why RestrictionCapableEval is not meant to be use as is and I'm just wondering if there is a reason for that comment (don't know, may be it is an old comment or it can causes some security issues).

As a RestrictedPython user perspective the RestrictionCapableEval API seems more clean (2 neat lines instead of 4 lines importing a ``safe_builtinsand acompile_restricted_eval` function). I think it would be more user friendly exposing a simple api that gets and expression plus a context and returns execute it.

I think that safe_builtins and compile_restricted_eval could be hided as inner implementation details. In addition with python3.6 you get an error because eval accepts a play byte_code but byte_code.code

Anyway thanks for your tips: if you are so kind to confirm that the compile_restricted_eval way is better I'll follow your suggestion as soon as possible!

Thanks :)

PS: the play-python package I've linked int this ticket is deprecated now because its features were included in https://github.com/tierratelematics/pytest-play

[icemac]

@davidemoro RestrictionCapableEval.eval() can be used but it does not prevent the code from accessing most of the attributes or items in the context. This might be okay in your use case. Zope (for which RestrictedPython was developed) uses its own implementations for _getitem_, _getattr_ and _getiter_ which restrict the access of the user the attributes resp. items he is allowed to access via ACLs.

RestrictedPython prevents access using some basic rules (like access to attributes those name starts with underscore is prevented) but there might be some other application specific rules which have to be implemented in functions used for _getitem_, _getattr_ and _getiter_ to get additional security assertions.

[davidemoro]

Ok, thanks a lot!