eval doesn't have access to variables in function scope

[stevengj]

The following works:

julia> x = 3;
julia> eval(:x)
3

But this gives an error:

julia> bar(y) = eval(:y)
julia> bar(7)
ERROR: y not defined
 in bar at none:1

It makes no sense to me that eval does not evaluate its expressions in the scope where it is called (and the manual seems to indicate this as well).

[pao]

See also #1651.

[mlubin]

A possible workaround is to use a macro and esc.

[Keno]

Eval always evaluates in global scope, since it's essentially impossible to JIT compile eval. What is your usecase?

julia> y=3
3

julia> bar() = eval(:y)
# method added to generic function bar

julia> bar()
3

[stevengj]

I don't care if it is JIT compiled. I just need it to be evaluated in the correct scope. I can't use a macro because macros evaluate at parse-time rather than run-time, and I need the latter here. I can use global variables to work around this, but that is a total hack. This is a bug.

I'm using it for my PyCall module, in order to have a function that generates an anonymous composite type to wrap around around a Python object: using introspection, it determines the object's methods, and generates the corresponding structure fields. This way I can do e.g. math = pywrap(pyimport("math")); math.sin(3). I have a macro @pyimport math that converts to math = pywrap(pyimport("math")), but the actual work needs to be done inside the function pywrap because:

• Issue Julia hangs on shell command in macro body #2378 means that pywrap cannot be evaluated at parse-time without hanging, because it may need to initialize Python which means calling run which means deadlocking with the parser.
• Even if Julia hangs on shell command in macro body #2378 were fixed, I wouldn't want to do this at parse time. As @JeffBezanson pointed out, it is better if macro bodies are pure functions (or nearly so). Initializing Python, or even just importing a Python module, or even just calling anything in Python, will inevitably have side effects. I want these side effects to occur when the code is executed, not when it is parsed: for code in a function or a module (as opposed to code in the REPL), parse and execution time may be very different.
• It doesn't have to be fast, because this is only done once for a module, and the Julia execution time is negligible compared to the time spent munging Python stuff anyway.
• To work around this issue, I am using global variables (see the pywrap code), but this is ugly.

[mlubin]

It's a bit hard to quickly dig into the PyCall code to determine if this is reasonable, but you can embed the values of the variables you need in the expression, e.g.:

julia> bar(y) = @eval $y
# method added to generic function bar

julia> bar(7)
7

[JeffBezanson]

This is not a bug, because it was a deliberate choice to make it possible to analyze local variables. eval essentially needs variables to be in some kind of lookup structure like a dictionary, so for a function scope to support eval it basically needs to be rewritten so that variable accesses are dictionary gets and puts. I deem this silly since it deprives you of most of the value of lexical scope. It is also hard to prove that a function wouldn't need eval.

I think there are at least two good ways to deal with this use case: insert the values of pywrap_members[i][2], quoted, directly into the AST as @mlubin suggests. Or, a common technique is to use eval to generate a function that accepts the external values it needs; wrap the expression in function (pywrap_o,pywrap_members), then call that function on the values. This is basically manual closure-conversion to handle the fact that the compiler can't see which variables are used inside eval since they are quoted. It also looks to me like most of the call does not need eval. tname is global, so eval(tname)(pywrap_o, [ ...comprehension without quote...]...) I think would work.

[stevengj]

Ah, I didn't realize that one could insert non-Expr values directly into the AST (but now I see that this is indeed in the manual); that simplifies things, and removes the need to access local symbols in the AST.

Okay, I see why it might be impractical to lookup local symbols, and why it is not needed.

[toivoh]

@mlubin: Though you should write that as

julia> bar(y) = @eval $(quot(y))

otherwise it will not work as expected for e.g. symbols, and might fail for other types of y as well.
(Where quot(x) = expr(:quote, {x}) now lives in Base.Meta)

[pao]

@toivoh might look into redefining quot(x) = Expr(:quote, x), see 396e6c4

[toivoh]

I guess that would be

quot(x) = Expr(:quote, {x}, Any)

or does Expr have other constructors by itself?
I don't quite understand Jeff's change, or at least why it's more
efficient. But wouldn't the current definition benefit from the new expr
methods as it stands?

[pao]

I get the impression that not having to wrap an Array around x is part of the efficiency gain. The patch Jeff committed uses a constructor for which Expr(hd, args...) is a valid invocation; in quot(), there's just one argument. I haven't had the chance to try it out yet, though.

[toivoh]

Perhaps. I need to be sure that it constructs a single argument expr
though. Before, it was quot(x) = expr(:quote, x), but then I tried to
quote a cell array, which gave a bug that was not very fun to track down.

[JeffBezanson]

Sorry about the sudden change, but I think it will be better to remove expr and just use Expr in its new form, which is more lispy: just Expr(:call, :+, x, y). I think this is actually easier to use. It is faster because quote emits huge nested calls that build expressions, and before it was using expr, which meant an extra call including varargs for every expression node. And often those expressions are top-level and run in the interpreter. Using Expr directly was much faster.

[pao]

Thanks for that clarification!

[toivoh]

Ok, seems like a good change. So Expr(head, x) will always be a single arg expression, even if x is a Vector{Any}? Is this new signature documented anywhere?

[toivoh]

Then I agree with you, @pao. Feel free to do it if you like.

[JeffBezanson]

Correct, it will always be a single-arg expression.

[pdenapo]

I have an expression that I want to differentiate using the Calculus package, and then create a Julia function that evaluate the expression of the derivative at a given value of x, like in the example:

using Calculus
expression_F="x^3"
expression_f= differentiate(expression_F,:x)
f(x)=eval(expression_f)

The last function definition is broken by this issue:

julia> f(8)
ERROR: UndefVarError: x not defined

How am I supposed to do this in Julia?

[KristofferC]

Please ask questions on https://discourse.julialang.org