04 Macros

Sibilisp Macros

If you struggle with using macros in your programs, this guide tries to shed some light onto the topic. It gives answers to question like the following ones:

• What are macros?
• In which situations should macros be used?
• How can macros be written?

We'll examine all questions and try to give appropriate solutions to exemplary code fragments.

Sibilisp's macros are based on Sibilant's macro system, and need the .sibilant file extension. This is due to the fact, that you have to use Sibilant's (include) for macro files and (include) either finds .sibilant at the end of the include path given to it, or it automatically attaches it. However, macros give you a certain opportunity to clarify and abstract your program code in a way, other languages simply don't provide. And of course, on top of Sibilant's macros, you can use Sibilisp's macros as well.

Let's begin!

Macro basics

Before we deep dive into the world of macros, here is a quick and general overview.

What are macros?

Simply speaking, macros are code that writes more code for you. Macros are indiscernable from regular programm code and are deeply embedded into the language - so much, that almost every part of Sibilisp's language is also part of a macro layer on top of pure Sibilant. And since Sibilant is also based on macros, you can see that macros are literally everywhere. In general macros look exactly like function calls and are defined exactly like functions, but instead of the (defun) macro, the (macro) macro is used. 🙃

A simple, minimal example

Let's consider for a moment that you have a code base in that you often need to swap the values of variables in place. Usually, this is tedious and boring work and it clutters your code with alot of stuff like this:

;; swap the values of variables a and b
(let ((tmp b)
      (b a)
      (a tmp))
  ...rest-of-code)

As said, think this repeatet several times throughout a larger codebase (say, > 100 LOC), keeping track of what goes on gets harder as it should. There's a reason why it turns out to be that way: Too much is going on. Instead of focusing on the main thing, we start to focus on the (let), interpret variable names and so on. This (contrived) example only deals with two variables, however in larger code bases variable counts tend to be (usually) alot higher.

TL;DR: 🤯

Let's fix that!

Here's a little macro called (swap-values!), that is suitable to solve our problem:

(macro swap-values! (var-a var-b)
  (let ((tmp (gensym "tmp")))
    `(let ((@tmp @var-b))
      (assign @var-b @var-a)
      (assign @var-a @tmp))))

Don't worry about the @ sign in front of some values or what a (gensym) does, we'll come to that in a minute. What's important now is to notice, that it more or less simply copies the problematic code and wraps it up into a macro that accepts two arguments.

With that, every instance of the aforementioned variable swappings can be replaced by a simple expression (swap-values!). Consider the example below:

(defvar a 10 
        b 20)
(swap-values! a b)  ; swap!
(.log console a b) ; 20 10
(swap-values! a b)  ; swap!
(.log console a b) ; 10 20

How does this work?

The answer to that question can be found by inspecting the resulting JavaScript code. When the above code is transpiled, Sibilisp's transpiler injects all of Sibilisp's macros into the file that is about to be transpiled and passes it to Sibilant's compiler. The compiler first expands all macro expressions into valid Sibilant code, and after that compiles the code into valid JavaScript. This is why macros allow you to write code that literally writes code.

Let's see what that means. Open the resulting JavaScript file, it's contents should look something like this, (without annotations):

let a = 10,
  b = 20;

(function (tmp$2) {    // <-- (swap-values! a b)
  b = a;
  return a = tmp$2;
}).call(this, b);

console.log(a, b);

(function (tmp$3) {    // <-- (swap-values! a b)
  b = a;
  return a = tmp$3;
}).call(this, b);

console.log(a, b);

As you can see, all occurences of (swap-values!) are gone and what remains is the inner part of the macro. Because the inner parts are themself macros, those are transpiled as well, resulting in the above calls to anonymous functions.

When should one use macros?

Despite the fact that macros seem to have unlimited powers, they are not suitable to every problem. Instead, macros solve a different set of problems than functions do. In general speaking, most of the time a function can do what a macro can do and is easier to write.

However, macros are unmatched when you want to abstract away code patterns (like we did above) rather than functionality. In that sense, macros allow you to expand the expressiveness of a program by abstracting away boilerplate code. This is possible because macros have (if needed) access to the abstract syntax tree (AST) the compiler generates and they can transform that tree while it is being compiled. 😲

We start with a simple macro that doesn't explicitly access the AST, but that is complex enough so you can clearly see the benefits gained by macros. Let's suppose you have a function rotate-around that takes a DOM node, an axis argument and an angle, it's implementation could be written like this:

;;; formulas are from: 
;;; https://www.geeksforgeeks.org/computer-graphics-3d-rotation-transformations/

(defun rotate-around (obj axis angle)
  ;; Takes a DOM node, an axis and an angle and rotates the node around the axis
  ;; in 3D space. Returns the node
  (let* ((c (cosine angle))
         (s (sine angle))
         (rotm (cond ((eql? axis 'x) (list 1 0 0 0 0 c s 0 0 (- s) c 0 0 0 0 1))
                     ((eql? axis 'y) (list c 0 (- s) 0 0 1 0 0 s 0 c 0 0 0 0 1))
                     :else (list c s 0 0 (- s) c 0 0 0 0 1 0 0 0 0 1))))
    (setf obj 'style 'transform (+ "matrix3d(" + (.join rotm ",") + ")"))
    obj))

There's a lot of clutter here. First off, let's get rid of all the lists of numbers first, because they really move our focus away from the main purpose of rotate-around into the nitty-gritty details.

Here's how you could write macros for calculating rotation matrices in 3D space:

(macro rotate-x (angle)
  `(let ((c (cosine @angle))
         (s (sine @angle)))
    (list 1 0 0 0 0 c s 0 0 (- s) c 0 0 0 0 1)))

(macro rotate-y (angle)
  `(let ((c (cosine @angle))
         (s (sine @angle)))
    (list c 0 (- s) 0 0 1 0 0 s 0 c 0 0 0 0 1)))

(macro rotate-z (angle)
  `(let ((c (cosine @angle))
         (s (sine @angle)))
    (list c s 0 0 (- s) c 0 0 0 0 1 0 0 0 0 1)))

From now on, we can clarify the rotate-around function by replacing the calculation parts with our macros. And we can "lower" the (let* ) expression into a (let ) expression, because we no longer have to keep those sine/cosine values around!

Just keep in mind that, although we can reduce the code a programmer has to read and understand, the resulting JavaScript isn't reduced in size - in fact, it is a bit more code than before. However, we are not trying to improve performance now, what we improve is readability. For most applications, having to pay a single hour more to the programmer is simply more expensive than the cost that's created by a performance penalty. Plus that you can always improve the macros later to gain more performance if needed. And a decent JavaScript bundler can futher optimize the code before it is shipped to production as well.

The clarified version looks like this:

(defun rotate-around (obj axis angle)
  ;; Takes a DOM node, an axis and an angle and rotates the node around the axis
  ;; in 3D space. Returns the node
  (let ((rotm (cond ((eql? axis 'x) (rotate-x angle))
                    ((eql? axis 'y) (rotate-y angle))
                    :else (rotate-z angle))))
    (setf obj 'style 'transform (+ "matrix3d(" + (.join rotm ",") + ")"))
    obj))

That's much clearer and therefor easier to read now.

We have removed the main distraction now, but some minor improvements can be made. See that setting of a CSS transformation? It burdens us the details of accessing the correct style property and how to construct a valid 3D transformation matrix value for it. This is what we are going to put into a macro as well:

(macro set-css-transform! (el matrix)
  `(setf @el 'style 'transform (+ "matrix3d(" (.join @matrix ", ") ")")))

This is what the final version of rotate-around looks like:

(defun rotate-around (obj axis angle)
  ;; Takes a DOM node, an axis and an angle and rotates the node around an axis 
  ;; in 3D space. Returns the node
  (let ((rotm (cond ((eql? axis 'x) (rotate-x angle))
                    ((eql? axis 'y) (rotate-y angle))
                    :else (rotate-z angle))))
    (set-css-transform! obj rotm)
    obj))

Macro syntax explained

Finally, we have to talk about the syntax of macros. 💬

You know by now, that a macro is something in between a function and a template. A macro returns a so called quoted piece of code that it stands in for. When the macro is executed on compile time, it receives it's arguments as nodes of the AST and is free to manipulate them before incorporating them into it's own return value.

To quote pieces of code, you usually use the backtick (`) in front of whatever you want to quote. Alternatively, you can also use the (quote) macro.

Here's an intentionally simple example of so called "quoting":

(macro five ()
  `5)

Whenever you'd need the value 5, you could instead now write (five), and the compiler will insert the number. What would happen, if the quote is missing? 🤔 In that case, the compiler will generate an error, because it receives an invalid return value when the macro is called during compile time (💥).

The @ sign

The @ sign is used to insert (or "splice in") a macro's argument(s) into quoted code and to gain access to the AST nodes of them. We will discuss the AST in greater detail in the next sections.

Here is a simple example of splicing in a value. It also uses the (five) macro we created:

(macro five-plus (n)
  `(+ (five) @n))

Now suppose you've written (five-plus 5) in your code. When this is seen by the compiler, it is expanded into this JavaScript:

(5 + 5)

The interesting part is, how this code came to be. This first 5 is created by the (five) macro, whereas the second one comes from the value 5 given as an argument and then spliced in.

This example also proofs that you can use macros inside other macros, because the (five) macro is fully evaluated, so the compiler works recursively. This is great, because it means that it doesn't matter how deep we nest macros: All of them will be expanded at some point into valid JavaScript code. 🤩

By now you've seen how to splice in single values via the @ sign. However, what if you need to splice in multiple values at once, for example when passing arguments into a function call?

You can splice in multiple values at once via spreading with ...@. Here's how that looks like.

Imagine the (when) macro wouldn't exist. Well, if you have an idea for a macro that is not included in Sibilisp, there's a good change you can invent it yourself. Here's a possible implementation of (when) you could write:

(macro when (is-truth ...body)
  `(if @is-truth
       (do ...@body)
       (nil)))

AST nodes 🌳

As said, macros receive on compile time their arguments as AST nodes. Such a node is - at compile time - modeled with a (hash) object. It has various useful fields or properties that you can inspect, but the most commonly needed ones are:

• type
• token
• contents

What do these three mean?

Obviously, type specifies the type of expression that was entered.

The token property simply holds a String of the actual code. If - for example - you need to have access to a given value to pre-calculate things, the token property is the way to go.

contents, as the name implies, grants access to contained AST nodes, or in other terms, the sub-expressions of the currently handled expression. The property is modeled with a (list).

What is a (gensym)?

Sibilisp ships with two macros, that are especially designed for writing other macros, named gensym and with-gensyms. You can use the gensym macro to generate unique variable names, so called "generic symbols". They are used if the code that a macro generates contains variable names that might shadow names of variables or arguments the user defined. Consider this macro for a moment:

(macro greet (name has-special-greeting?)
  `(scoped
    (defvar a (ternary @has-special-greeting?
                       "Welcome, "
                       "Hello, "))
    (+ a @name)))

This seems fine, right? When we call it with the parameters "John Doe" and true, it works as intended, generating the following JavaScript:

(function () {
  let a = (true ? "Welcome, " : "Hello, ");
  return a + "John Doe";
})();

But what would happen if, for some reason, the name argument is a variable named a? The compiler would generate this (obviously wrong) JavaScript:

(function () {
  let a = (true ? "Welcome, " : "Hello, ");
  return a + a;
})();

Although this is a completely contrived example, you get the point: Whenever you write macros that generate variables or functions in their generated code, you should use gensym or with-gensyms to generate unique variable names, so you don't shadow already existing variables accidentally.
Here's how the greet macro looks like when written using with-gensyms:

(macro greet (name has-special-greeting?)
  (with-gensyms (a)
    `(scoped
      (defvar @a (ternary @has-special-greeting?
                          "Welcome, "
                          "Hello, "))
      (+ @a @name))))

This generates a different JavaScript when compiled, where it doesn't matter if a variable a is passed in:

(function () {
  let a$123 = (true ? "Welcome, " : "Hello, ");
  return a$123 + a;
})();

Just for completeness, here's how greet would be written with gensym instead of with-gensyms:

(macro greet (name has-special-greeting?)
  (let ((a (gensym 'a)))
    `(scoped
      (defvar @a (ternary @has-special-greeting?
                          "Welcome, "
                          "Hello, "))
      (+ @a @name))))

---