Metaprogramming

The strongest legacy of Lisp in the Julia language lies in its metaprogramming capabilities. Like Lisp, Julia is homoiconic: it represents its own code as a data structure of the language itself. Since code is represented by objects that can be created and manipulated from within the language, it is possible for a program to transform and generate its own code. This allows first-class code generation without bolted on build steps, and also allows first-class, Lisp-like macros, as compared to preprocessor "macro" systems like that of C or C++, which simply perform textual manipulation as a separate pass before any compilation occurs. Another aspect of metaprogramming is reflection: allowing the running program to dynamically discover properties of itself. Again, this falls naturally out of the fact that all data types and code are represented by first-class Julia data structures: one can simply programmatically access the data structures representing the running system to gain run-time knowledge if its structure and properties.

## Expressions and Eval

Julia code is represented as a syntax tree built out of Julia data structures of type Expr. This makes it easy to construct and manipulate Julia code from within Julia, without generating or parsing source text. Here is the definition of Expr:

type Expr
    head::Symbol
    args::Array{Any,1}
    typ
end

The head is a symbol identifying the kind of expression, and args is an array of subexpressions. typ is used by type inference to store type annotations (it can generally be ignored).

There is special syntax for "quoting" code (analogous to quoting strings) that makes it easy to create expression objects without explicitly constructing Expr objects. There are two forms: a short form for inline expressions using : followed by a single expression, and a long form for blocks of code, enclosed in quote ... end. Here is an example of the short form used to quote an arithmetic expression:

julia> ex = :(a+b*c+1)
+(a,*(b,c),1)

julia> typeof(ex)
Expr

julia> ex.head
call

julia> ex.args
{+,a,*(b,c),1}

julia> typeof(ex.args[1])
Symbol

julia> typeof(ex.args[2])
Symbol

julia> typeof(ex.args[3])
Expr

julia> typeof(ex.args[4])
Int64

Expressions provided by the parser generally only have symbols, other expressions, or literal values as their args, whereas expressions constructed by Julia code can have arbitrary values as args. In this case, + and a are symbols, *(b,c) is a subexpression, and 1 is a literal 64-bit signed integer. Here's an example of the longer expression quoting form:

julia> quote
     x = 1
     y = 2
     x + y
   end

begin
  x = 1
  y = 2
  +(x,y)
end

When the argument to : is just a symbol, a Symbol object results instead of an Expr:

julia> :foo
foo

julia> typeof(ans)
Symbol

This is precisely the traditional Lisp syntax for symbol literals, which Ruby has also inherited. Since the : prefix in Julia can be used to quote entire expressions, however, it is generally more similar to Lisp quoting operators than to : in Lisp, which can only be used to prefix symbol literals. In Julia, the two uses dovetail nicely enough to only warrant a single operator notation.

### Eval and Interpolation

Given an expression object, one can cause Julia to evaluate (execute) it at the top level scope (equivalent to loading from a file or typing at the interactive prompt) using the eval function:

julia> :(1 + 2)
+(1,2)

julia> eval(ans)
3

julia> ex = :(a + b)
+(a,b)

julia> eval(ex)
a not defined

julia> a = 1; b = 2;

julia> eval(ex)
3

Expressions passed to eval are not limited to only returning values — they can have side-effects:

julia> ex = :(x = 1)
x = 1

julia> x
x not defined

julia> eval(ex)
1

julia> x
1

Here, the evaluation of an expression object causes a value to be assigned to the top-level variable, x. Since expressions are just Expr objects, one can construct expressions completely programmatically and then evaluate them, thereby allowing completely dynamically generated code to be run. Here is a trivial example:

julia> a = 1;

julia> ex = Expr(:call, {:+,a,:b}, Any)
+(1,b)

julia> a = 0; b = 2;

julia> eval(ex)
3

The value of a is used to construct the expression ex which applies the + function to the value 1 and the variable b. Constructing Expr objects like this allows construction of any expression at all, but is somewhat tedious and ugly. Since the Julia parser is already excellent at producing expression objects, Julia allows "splicing" or interpolation of expression objects, prefixed with $, into larger expressions, written using normal syntax. The above example can be written more clearly and concisely using interpolation:

julia> a = 1;
1

julia> ex = :($a + b)
+(1,b)

Here, the construction of the Expr object is handled by the Julia parser, using the value of a in the appropriate slot of the syntax tree. The use of $ for expression interpolation is intentionally reminiscent of string interpolation and command interpolation. Expression interpolation allows convenient, readable programmatic construction of complex Julia expressions.

### Code Generation

It is a common strategy, when a significant amount of repetitive boilerplate code is required, to programmatically generate such code, minimizing the redundant repetition of information. In most languages, this requires an extra build step, and a separate program to generate the repetitive code. In Julia, expression interpolation and eval allow such code generation to take place in the normal course of program execution. For example, the following code defines a series of operators on three arguments in terms of their 2-argument forms:

for op = (:+, :*, :&, :|, :$)
  eval(quote
    ($op)(a,b,c) = ($op)(($op)(a,b),c)
  end)
end

In this manner, Julia acts as its own preprocessor, and allows code generation from inside the language, instead of requiring the programmer to actually generate source files as part of an external build process. The above code could be written slightly more tersely using the : prefix quoting form:

for op = (:+, :*, :&, :|, :$)
  eval(:(($op)(a,b,c) = ($op)(($op)(a,b),c)))
end

This sort of in-language code generation, using the eval(quote(...)) pattern, is common enough, however, that Julia comes with a macro to abbreviate this pattern:

for op = (:+, :*, :&, :|, :$)
  @eval ($op)(a,b,c) = ($op)(($op)(a,b),c)
end

The @eval macro just rewrites this call to be precisely equivalent to the above longer version. Interpolating into an unquoted expression is not supported and will cause a compile-time error:

julia> $a + b
not supported

For longer blocks of generated code, the expression argument given to @eval can be a block:

@eval begin
  # multiple lines
end

## Macros

Macros allow the programmer to automatically generate code at compile time, transforming zero or more argument expressions into a single result expression, which then takes the place of the macro call in the final syntax tree. Macros are invoked with the following general syntax:

@name expr1 expr2 ...

Note the distinguishing @ before the macro name and the lack of commas between the argument expressions. Before the program runs, this statement will be replaced with the result of calling an expander function for name on the expression arguments. Expanders are defined with the macro keyword:

macro name(expr1, expr2, ...)
    ...
end

Here, for example, is very nearly the definition of Julia's @assert macro (see error.j for the actual definition, which allows @assert to work on booleans arrays as well):

macro assert(ex)
    :($ex ? nothing : error("Assertion failed: ", $string(ex)))
end

This macro can be used like this:

julia> @assert 1==1.0

julia> @assert 1==0
Assertion failed: 1==0

Macro calls are expanded so that the above calls are precisely equivalent to writing

1==1.0 ? nothing : error("Assertion failed: ", "1==1.0")
1==0 ? nothing : error("Assertion failed: ", "1==0")

That is, in the first call, the expression :(1==1.0) is spliced into the test condition slot, while the value of string(:(1==1.0)) is spliced into the assertion message slot. The entire expression, thus constructed, is placed into the syntax tree where the @assert macro call occurs. Thus, if the test expression is true when evaluated, the entire expression evaluates to nothing, whereas if the test expression is false, an error is raised indicating the asserted expression that was false. Notice that it would not be possible to write this as a function, since only the value of the condition and not the expression that computed it would be available.

## Reflection

---

Previous: Parallel Computing — Next: Calling C Code