How do we extend GML?

Before proceeding, you might find it useful to read How does coroutine execution work?

A lot of the ideas that are used for this library are inspired by the inimitable Katsaii and her macro hacking research.

 

At first glance, adding new language features to GML seems like magic. When writing a library, or really any interchangeable code, we usually prioritise interacting with systems via functions. Some libraries encourage you to edit variables directly, others use shaders or surfaces to achieve the desired effect, and many libraries use macros and enums to represent constants specific to that library.

Macros are very powerful, however. Macros effectively insert whatever their value is directly into the code. Typically, we'd set a macro to a string or a number, and occasionally we'll use a macro as an expression that we want to use in multiple places. For example,

#macro ON_DIRECTX  ((os_type == os_windows) or (os_type == os_xboxone) or (os_type == os_xboxseriesxs) or (os_type == os_uwp))

We can then use ON_DIRECTX elsewhere in the codebase to execute different behaviour depending on whether the game is running on a DirectX platform (this is useful for fixing issues with UV coordinates). Here, the macro is being used an enclosed expression - it's entirely self-contained and can be dropped in wherever it's needed without further consideration.

This library uses many macros, but most of the these macro are not self-contained. The macros in this library insert incomplete expressions into code. In order to produce valid, complete expressions the macros must be used together. This sounds like a problem initially, but it allows us to create a grammar that can express complicated structures.

Let's look at how CO_BEGIN and CO_END work. These two macros are conceptually the most complicated but getting to grips with how they work explains how the rest of the custom syntax macros fit together.

#macro CO_BEGIN  ((function(){__CoroutineBegin(function(){
#macro CO_END    });return __CoroutineEnd();})());

This is pretty complicated code, and uses a lot of nesting which is hard to read, so let's use this in context to make things easier.

function Example()
{
    return CO_BEGIN
        show_debug_message("What an incredible function.");
    CO_END
}

This expands out to:

function Example()
{
    // CO_BEGIN
    return ((function(){__CoroutineBegin(function(){
    
    //The single line of code we put in the coroutine
    show_debug_message("What an incredible function.");
    
    //CO_END
    });return __CoroutineEnd();})());
}

You can see that we're using macros to insert legitimate GML around the content of the coroutine. Let's take another step to tease apart the actual operations that are going on. I'll rewrite things a little bit so that it's easier to see how the generator function is able to return a new coroutine root struct.

function Example()
{
    var _generatorfunction = function()
    {
        //Add some code for execution to a global coroutine root struct (global.__coroutineNext)
        //A global empty coroutine root struct is created on boot and is always available
        //We generate a new struct at the end of this generator function for use in the next coroutine
        __CoroutineBegin(function()
        {
            show_debug_message("What an incredible function.");
        });
        
        //Return the coroutine struct we just added the code to
        return __CoroutineEnd();
    });
    
    //Run the generator function and return its value (the coroutine root struct)
    return _generatorFunction();
}

It's kinda incredible how much we can do with macros. With two tiny statements, CO_BEGIN and CO_END, we can unpack an entire function inside another function! You'll notice that CO_BEGIN ends with __CoroutineFunction(function() { and CO_END starts with });. This enabled the two macros to be used to encapsulate code inside a function. This is a repeating pattern with macros that this library uses: a macro will either open up a new function or close a function.

Let's make our example a bit more complex. First up, let's introduce three more macros.

#macro REPEAT  });__CoroutineRepeat(function(){return 
#macro THEN    });__CoroutineThen(function(){
#macro END     });__CoroutineEndLoop(function(){

...and now the coroutine itself...

Please note that the use of REPEAT 5 THEN ... END here is entirely unnecessary, repeat(5) {} would do fine. REPEAT (and WHILE / IF / FOREACH) are only required if the loop contains a YIELD command. We're using the coroutine version of the loop here as an example.

function Example()
{
    return CO_BEGIN
        show_debug_message("What an incredible function.");
        REPEAT 5 THEN
            show_debug_message("Wow!");
        END
    CO_END
}

...which unpacks to the following generator function:

function Example()
{
    //CO_BEGIN
    var _generatorfunction = function()
    {
        __CoroutineBegin(function()
        {
             //First line of code
             show_debug_message("What an incredible function.");
        
        //REPEAT
        });
        __CoroutineRepeat(function()
        {
            return 5 // 5 comes from the number between REPEAT and THEN
        
        //THEN
        });
        __CoroutineThen(function()
        {
            //Second line of code
            show_debug_message("Wow!");
        
        //END
        });
        __CoroutineEndLoop(function()
        {
        
        //CO_END
        });
        
        //Return the coroutine struct we just added the code to
        return __CoroutineEnd();
    });
    
    //Run the generator function and return its value (the coroutine root struct)
    return _generatorFunction();
}

Note that the last function is empty. One of the quirks of this macro-based system is that this happens a lot!

Now it is hopefully clear how these macros, defined as incomplete expressions, link together to generate valid GML. In reality, these macros aren't doing anything more than calling functions like any other library. Their strength is in how the functional components are hidden from view which makes it a lot easier to write code without getting caught up on function calls that clutter the screen.