Find file Copy path
1c2c986 Jun 6, 2018
5 contributors

Users who have contributed to this file

@markknol @Simn @Gama11 @shohei909 @ibilon
338 lines (216 sloc) 22.6 KB
Macros are without a doubt the most advanced feature in Haxe. They are often perceived as dark magic that only a select few are capable of mastering, yet there is nothing magical (and certainly nothing dark) about them.
\define{Abstract Syntax Tree (AST)}{define-ast}{The AST is the result of \emph{parsing} Haxe code into a typed structure. This structure is exposed to macros through the types defined in the file \expr{haxe/macro/Expr.hx} of the Haxe Standard Library.}
A basic macro is a \emph{syntax-transformation}. It receives zero or more \tref{expressions}{expression} and also returns an expression. If a macro is called, it effectively inserts code at the place it was called from. In that respect, it could be compared to a preprocessor like \expr{\#define} in C++, but a Haxe macro is not a textual replacement tool.
We can identify different kinds of macros, which are run at specific compilation stages:
\item[Initialization Macros:] These are provided by command line using the \ic{--macro} compiler parameter. They are executed after the compiler arguments were processed and the \emph{typer context} has been created, but before any typing was done (see \Fullref{macro-initialization}).
\item[Build Macros:] These are defined for classes, enums and abstracts through the \expr{@:build} or \expr{@:autoBuild} \tref{metadata}{lf-metadata}. They are executed per-type, after the type has been set up (including its relation to other types, such as inheritance for classes) but before its fields are typed (see \Fullref{macro-type-building}).
\item[Expression Macros:] These are normal functions which are executed as soon as they are typed.
\paragraph{Related content}
\item See the \href{}{macro API documentation} for details about its tools, classes an methods.
\item See the \href{}{macro snippets and tutorials} section in the Haxe Code Cookbook.
\section{Macro Context}
\define{Macro Context}{define-macro-context}{The macro context is the environment in which the macro is executed. Depending on the macro type, it can be considered to be a class being built or a function being typed. Contextual information can be obtained through the \ic{haxe.macro.Context} API.}
Haxe macros have access to different contextual information depending on the macro type. Other than querying such information, the context also allows some modifications such as defining a new type or registering certain callbacks. It is important to understand that not all information is available for all macro kinds, as the following examples demonstrate:
\item Initialization macros will find that the \expr{Context.getLocal*()} methods return \expr{null}. There is no local type or method in the context of an initialization macro.
\item Only build macros get a proper return value from \expr{Context.getBuildFields()}. There are no fields being built for the other macro kinds.
\item Build macros have a local type (if incomplete), but no local method, so \expr{Context.getLocalMethod()} returns \expr{null}.
The context API is complemented by the \expr{haxe.macro.Compiler} API detailed in \Fullref{macro-initialization}. While this API is available to all macro kinds, care has to be taken for any modification outside of initialization macros. This stems from the natural limitation of undefined \tref{build order}{macro-limitations-build-order}, which could cause e.g. a flag definition through \expr{Compiler.define()} to take effect before or after a \tref{conditional compilation}{lf-condition-compilation} check against that flag.
\paragraph{Related content}
\item See the \href{}{macro Context API documentation}.
\item See the \href{}{macro snippets and tutorials} section in the Haxe Code Cookbook.
Most of the time, arguments to macros are expressions represented as an instance of enum \type{Expr}. As such, they are parsed but not typed, meaning they can be anything conforming to Haxe's syntax rules. The macro can then inspect their structure, or (try to) get their type using \expr{haxe.macro.Context.typeof()}.
It is important to understand that arguments to macros are not guaranteed to be evaluated, so any intended side-effect is not guaranteed to occur. On the other hand, it is also important to understand that an argument expression may be duplicated by a macro and used multiple times in the returned expression:
The macro \expr{add} is called with \expr{x++} as argument and thus returns \expr{x++ + x++} using \tref{expression reification}{macro-reification-expression}, causing \expr{x} to be incremented twice.
Since \type{Expr} is compatible with any possible input, Haxe provides the type \type{haxe.macro.ExprOf<T>}. For the most part, this type is identical to \type{Expr}, but it allows constraining the type of accepted expressions. This is useful when combining macros with \tref{static extensions}{lf-static-extension}:
The two direct calls to \expr{identity} are accepted, even though the argument is declared as \expr{ExprOf<String>}. It might come as a surprise that the \type{Int} \expr{1} is accepted, but it is a logical consequence of what was explained about \tref{macro arguments}{macro-arguments}: The argument expressions are never typed, so it is not possible for the compiler to check their compatibility by \tref{unifying}{type-system-unification}.
This is different for the next two lines which are using static extensions (note the \expr{using Main}): For these it is mandatory to type the left side (\expr{"foo"} and \expr{1}) first in order to make sense of the \expr{identity} field access. This makes it possible to check the types against the argument types, which causes \expr{1.identity()} to not consider \expr{Main.identity()} as a suitable field.
\subsection{Constant Expressions}
A macro can be declared to expect \tref{constant}{expression-constants} arguments:
With these it is not necessary to detour over expressions as the compiler can use the provided constants directly.
\subsection{Rest Argument}
If the final argument of a macro is of type \type{Array<Expr>}, the macro accepts an arbitrary number of extra arguments which are available from that array:
The Haxe Compiler allows \emph{reification} of expressions, types and classes to simplify working with macros. The syntax for reification is \expr{macro expr}, where \expr{expr} is any valid Haxe expression.
\subsection{Expression Reification}
Expression reification is used to create instances of \type{haxe.macro.Expr} in a convenient way. The Haxe Compiler accepts the usual Haxe syntax and translates it to an expression object. It supports several escaping mechanisms, all of which are triggered by the \expr{\$} character:
\item[\expr{\$\{\}} and \expr{\$e\{\}}:] \type{Expr -> Expr} This can be used to compose expressions. The expression within the delimiting \expr{\{ \}} is executed, with its value being used in place.
\item[\expr{\$a\{\}}:] \type{Array<Expr> -> Array<Expr>} or \type{Array<Expr> -> Expr} If used in a place where an \type{Array<Expr>} is expected (e.g. call arguments, block elements), \expr{\$a\{\}} treats its value as that array. Otherwise it generates an array declaration.
\item[\expr{\$b\{\}}:] \type{Array<Expr> -> Expr} Generates a block expression from the given expression array.
\item[\expr{\$i\{\}}:] \type{String -> Expr} Generates an identifier from the given string.
\item[\expr{\$p\{\}}:] \type{Array<String> -> Expr} Generates a field expression from the given string array.
\item[\expr{\$v\{\}}:] \type{Dynamic -> Expr} Generates an expression depending on the type of its argument. This is only guaranteed to work for \tref{basic types}{types-basic-types} and \tref{enum instances}{types-enum-instance}.
Additionally the \tref{metadata}{lf-metadata} \expr{@:pos(p)} can be used to map the position of the annotated expression to \expr{p} instead of the place it is reified at.
This kind of reification only works in places where the internal structure expects an expression. This disallows \expr{object.\$\{fieldName\}}, but \expr{object.\$fieldName} works. This is true for all places where the internal structure expects a string:
\item field access \expr{object.\$name}
\item variable name \expr{var \$name = 1;}
\item field name \expr{\{ \$name: 1\} }
\item function name \expr{function \$name() \{ \}}
\item catch variable name \expr{try e() catch(\$name:Dynamic) \{ \}}
Furthermore, a \expr{new} expression can be reified by providing \href{}{haxe.macro.TypePath} argument: \expr{new \$typePath()}
\subsection{Type Reification}
Type reification is used to create instances of \type{haxe.macro.Expr.ComplexType} in a convenient way. It is identified by a \expr{macro : Type}, where \expr{Type} can be any valid type path expression. This is similar to explicit type hints in normal code, e.g. for variables in the form of \expr{var x:Type}.
Each constructor of \type{ComplexType} has a distinct syntax:
\item[\expr{TPath}:] \expr{macro : pack.Type}
\item[\expr{TFunction}:] \expr{macro : Arg1 -> Arg2 -> Return}
\item[\expr{TAnonymous}:] \expr{macro : \{ field: Type \}}
\item[\expr{TParent}:] \expr{macro : (Type)}
\item[\expr{TExtend}:] \expr{macro : \{> Type, field: Type \}}
\item[\expr{TOptional}:] \expr{macro : ?Type}
\subsection{Class Reification}
It is also possible to use reification to obtain an instance of \type{haxe.macro.Expr.TypeDefinition}. This is indicated by the \expr{macro class} syntax as shown here:
The generated \type{TypeDefinition} instance is typically passed to \expr{haxe.macro.Context.defineType} in order to add a new type to the calling context (not the macro context itself).
This kind of reification can also be useful to obtain instances of \expr{haxe.macro.Expr.Field}, which are available from the \expr{fields} array of the generated \type{TypeDefinition}.
The Haxe Standard Library comes with a set of tool-classes to simplify working with macros. These classes work best as \tref{static extensions}{lf-static-extension} and can be brought into context either individually or as a whole through \expr{using haxe.macro.Tools}. These classes are:
\item[\type{ComplexTypeTools}:] Allows printing \type{ComplexType} instances in a human-readable way. Also allows determining the \type{Type} corresponding to a \type{ComplexType}.
\item[\type{ExprTools}:] Allows printing \type{Expr} instances in a human-readable way. Also allows iterating and mapping expressions.
\item[\type{MacroStringTools}:] Offers useful operations on strings and string expressions in macro context.
\item[\type{TypeTools}:] Allows printing \type{Type} instances in a human-readable way. Also offers several useful operations on types, such as \tref{unifying}{type-system-unification} them or getting their corresponding \type{ComplexType}.
Furthermore the \type{haxe.macro.Printer} class has public methods for printing various types as a human-readable format. This can be helpful when debugging macros.
\trivia{The tinkerbell library and why Tools.hx works}{We learned about static extensions that using a \emph{module} implies that all its types are brought into static extension context. As it turns out, such a type can also be a \tref{typedef}{type-system-typedef} to another type. The compiler then considers this type part of the module, and extends static extension accordingly.\\
This ``trick'' was first used in Juraj Kirchheim's \emph{tinkerbell}\footnote{} library for exactly the same purpose. Tinkerbell provided many useful macro tools long before they made it into the Haxe Compiler and Haxe Standard Library. It remains the primary library for additional macro tools and offers other useful functionality as well.}
\section{Type Building}
Type-building macros are different from expression macros in several ways:
\item They do not return expressions, but an array of class fields. Their return type must be set explicitly to \type{Array<haxe.macro.Expr.Field>}.
\item Their \tref{context}{macro-context} has no local method and no local variables.
\item Their context does have build fields, available from \expr{haxe.macro.Context.getBuildFields()}.
\item They are not called directly, but are argument to a \expr{@:build} or \expr{@:autoBuild} \tref{metadata}{lf-metadata} on a \tref{class}{types-class-instance} or \tref{enum}{types-enum-instance} declaration.
The following example demonstrates type building. Note that it is split up into two files for a reason: If a module contains a \expr{macro} function, it has to be typed into macro context as well. This is often a problem for type-building macros because the type to be built could only be loaded in its incomplete state, before the building macro has run. We recommend to always define type-building macros in their own module.
The \expr{build} method of \type{TypeBuildingMacro} performs three steps:
\item It obtains the build fields using \expr{Context.getBuildFields()}.
\item It declares a new \type{haxe.macro.expr.Field} field using the \expr{funcName} macro argument as field name. This field is a \type{String} \tref{variable}{class-field-variable} with a default value \expr{"my default"} (from the \expr{kind} field) and is public and static (from the \expr{access} field).
\item It adds the new field to the build field array and returns it.
This macro is argument to the \expr{@:build} metadata on the \type{Main} class. As soon as this type is required, the compiler does the following:
\item It parses the module file, including the class fields.
\item It sets up the type, including its relation to other types through \tref{inheritance}{types-class-inheritance} and \tref{interfaces}{types-interfaces}.
\item It executes the type-building macro according to the \expr{@:build} metadata.
\item It continues typing the class normally with the fields returned by the type-building macro.
This allows adding and modifying class fields at will in a type-building macro. In our example, the macro is called with a \expr{"myFunc"} argument, making \expr{Main.myFunc} a valid field access.
If a type-building macro should not modify anything, the macro can return \expr{null}. This indicates to the compiler that no changes are intended and is preferable to returning \expr{Context.getBuildFields()}.
\subsection{Enum building}
Building \tref{enums}{types-enum-instance} is analogous to building classes with a simple mapping:
\item Enum constructors without arguments are variable fields \expr{FVar}.
\item Enum constructors with arguments are method fields \expr{FFun}.
\todo{Check if we can build GADTs this way.}
Because enum \type{E} is annotated with a \expr{:build} metadata, the called macro builds two constructors \expr{A} and \expr{B} ``into'' it. The former is added with the kind being \expr{FVar(null, null)}, meaning it is a constructor without argument. For the latter, we use \tref{reification}{macro-reification-expression} to obtain an instance of \type{haxe.macro.Expr.Function} with a singular \type{Int} argument.
The \expr{main} method proves the structure of our generated enum by \tref{matching}{lf-pattern-matching} it. We can see that the generated type is equivalent to this:
enum E {
If a class has the \expr{:autoBuild} metadata, the compiler generates \expr{:build} metadata on all extending classes. If an interface has the \expr{:autoBuild} metadata, the compiler generates \expr{:build} metadata on all implementing classes and all extending interfaces. Note that \expr{:autoBuild} does not imply \expr{:build} on the class/interface itself.
This outputs during compilation:
fromInterface: TInst(I2,[])
fromInterface: TInst(Main,[])
fromBaseClass: TInst(Main,[])
It is important to keep in mind that the order of these macro executions is undefined, which is detailed in \Fullref{macro-limitations-build-order}.
\paragraph{Related content}
\item \href{}{Haxe snippets and tutorials about build macros} in the Haxe Code Cookbook.
Normal \tref{build-macros}{macro-type-building} are run per-type and are already very powerful. In some cases it is useful to run a build macro per type \emph{usage} instead, i.e. whenever it actually appears in the code. Among other things this allows accessing the concrete type parameters in the macro.
\expr{@:genericBuild} is used just like \expr{@:build} by adding it to a type with the argument being a macro call:
When running this example the compiler outputs \ic{TAbstract(Int,[])} and \ic{TInst(String,[])}, indicating that it is indeed aware of the concrete type parameters of \type{MyType}. The macro logic could use this information to generate a custom type (using \expr{haxe.macro.Context.defineType}) or refer to an existing one. For brevity we return \expr{null} here which asks the compiler to \tref{infer}{type-system-type-inference} the type.
In Haxe 3.1 the return type of a \expr{@:genericBuild} macro has to be a \type{haxe.macro.Type}. Haxe 3.2 allows (and prefers) returning a \type{haxe.macro.ComplexType} instead, which is the syntactic representation of a type. This is easier to work with in many cases because types can simply be referenced by their paths.
\paragraph{Const type parameter}
Haxe allows passing \tref{constant expression}{expression-constants} as a type parameter if the type parameter name is \expr{Const}. This can be utilized in the context of \expr{@:genericBuild} macros to pass information from the syntax directly to the macro:
Here the macro logic could load a file and use its contents to generate a custom type.
\paragraph{Related content}
\item \href{}{Haxe snippets and tutorials about build macros} in the Haxe Code Cookbook.
\subsection{Static extension}
The concepts of \tref{static extensions}{lf-static-extension} and macros are somewhat conflicting: While the former requires a known type in order to determine used functions, macros execute before typing on plain syntax. It is thus not surprising that combining these two features can lead to issues. Haxe 3.0 would try to convert the typed expression back to a syntax expression, which is not always possible and may lose important information. We recommend that it is used with caution.
The combination of static extensions and macros was reworked for the 3.1.0 release. The Haxe Compiler does not even try to find the original expression for the macro argument and instead passes a special \expr{@:this this} expression. While the structure of this expression conveys no information, the expression can still be typed correctly:
\subsection{Build Order}
The build order of types is unspecified and this extends to the execution order of \tref{build-macros}{macro-type-building}. While certain rules can be determined, we strongly recommend to not rely on the execution order of build-macros. If type building requires multiple passes, this should be reflected directly in the macro code. In order to avoid multiple build-macro execution on the same type, state can be stored in static variables or added as \tref{metadata}{lf-metadata} to the type in question:
With both interfaces \type{I1} and \type{I2} having \expr{:autoBuild} metadata, the build macro is executed twice for class \type{C}. We guard against duplicate processing by adding a custom \expr{:processed} metadata to the class, which can be checked during the second macro execution.
\subsection{Type Parameters}
\section{Initialization Macros}
Initialization macros are invoked from command line by using the \expr{--macro callExpr(args)} command. This registers a callback which the compiler invokes after creating its context, but before typing what was argument to \expr{-main}. This then allows configuring the compiler in some ways.
If the argument to \expr{--macro} is a call to a plain identifier, that identifier is looked up in the class \type{haxe.macro.Compiler} which is part of the Haxe Standard Library. It comes with several useful initialization macros which are detailed in its \href{}{API}.
As an example, the \expr{include} macro allows inclusion of an entire package for compilation, recursively if necessary. The command line argument for this would then be \expr{--macro include('some.pack', true)}.
Of course it is also possible to define custom initialization macros to perform various tasks before the real compilation. A macro like this would then be invoked via \expr{--macro some.Class.theMacro(args)}. For instance, as all macros share the same \tref{context}{macro-context}, an initialization macro could set the value of a static field which other macros use as configuration.