eval - 'Call' nodes are not implemented

[glaucouri]

df = P.DataFrame({'a':[1,1,2,2,3,3]})
df.eval("a.mean()")

raise:
NotImplementedError: 'Call' nodes are not implemented

i hope this will be possible
thank's
Gla

[cpcloud]

What you've got there doesn't make a whole lot of sense (maybe you're doing something that does and this is just an example):

df = DataFrame({'a': [1, 1, 2, 2, 3, 3]})
m = df.a.mean()
df.eval('m') == m
pd.eval('m') == m

so just evaluate it in Python.

No new features will be added to eval, query for the next release (0.13) and possibly even 0.14

[cpcloud]

Slightly OT: @glaucouri Just in case that error message is a bit cryptic: A "Call" node is a node in the abstract syntax tree created by parsing Python source. Here is the language reference on callable objects.

[glaucouri]

What i'm trying to explain (with my poor english) is that in an eval environment if [a] is a P.Series i expect that i can use a this syntax : ( this is my working program, that users can use for do personalized equations)

safe_env = dict( [(k,df[k]) for k in df.keys() ])
safe_env # some other injections from numpy
val = eval(equation,safe_env)
#now val is the result of the equation, so with a little of broadcasting i can insert it in the df

Actually, with this solution users can do filtering, grouping and equation in a DataFrame environment.

Thank's
Gla

[cpcloud]

@glaucouri What you've just described is essentially what happens under the hood in DataFrame.query()/DataFrame.eval(). Currently, you cannot call functions with either of those methods. Does that make sense?

[cpcloud]

Closing as stale ... not really a demand for this right now

[xenoscopic]

Out of curiosity, would a (well-written) pull request adding support for the numexpr intrinsic functions be accepted? I've noticed that there is a variable defined in the expression parser called _numexpr_supported_calls, which seems to enumerate these functions, but this variable doesn't seem to be used anywhere. Stuff like abs or some of the transcendental functions might be quite useful.

[jreback]

we always accept a well-written PR (even some not so well-written!) :)

[cpcloud]

@havoc-io Yes, a PR would be great! To get started (if you haven't already) take a look in pandas/computation/expr.py. This is where the parsing logic is implemented. You'd need to allow Call nodes by removing them from the set exclusion happening in the decorator.

Here's an example of what a Call node looks like for the code f(a, b, c=1):

import ast
from ast import *  # forgive the star import here
node = ast.parse('f(a, b, c=1)', mode='eval').body
node.func == Name(id='f', ctx=Load())
node.args == [Name(id='a', ctx=Load()), Name(id='b', ctx=Load())]
node.keywords == [keyword(arg='c', value=Num(n=1))]

You could probably skip implementing keyword arguments for now and just implement unary or binary functions to get a feel for how difficult (or not) it would be to do this.

[xenoscopic]

Eish, this may be a bit complicated actually, given the conditional handling of Call nodes (not to mention their subnodes). It might bloat the parsing code quite a bit, maybe more than it's worth. Perhaps simply transforming Series objects before evaling is going to be easier for now for my use case.

It may also not be as well-defined as I initially imagined. E.g., would you want the behavior for only the 'numexpr' engine or the 'python' engine as well (with appropriate replacement functions from math or NumPy)? I'm also not sure what the behavior should be if local_dict overrides one of the intrinsics - I'd have to see what numexpr does in this case.

Maybe best to leave this issue dead for now :(.

[cpcloud]

@havoc-io There are quite a few details to be worked out here, which is part of the reason I didn't implement these right away.

It would be nice to have the python engine for testing and speed comparisons, but you could also just test using regular numpy based, toplevel python code. Maybe see how hard it would be to do the simplest thing.

• only numexpr for now
• add the funcs to the global namespace (just for validation) ... this may require some kind of custom "delayed" call node that doesn't do anything because you don't want to actually evaluate it

[cpcloud]

I believe numexpr overrides intrinsics so a variable called sin wouldn't work.