Parser

The Steppable Parser

Steppable has now got its own language. It is designed to be minimal but powerful. It is also designed for non-programmers to quickly grasp and master. For more information, see Syntax.

Parser generation

The language parsing is handled by Tree-sitter, a fast parsing tool used by editors to colorize code. We aim to compensate the lower performance of Steppable itself with its great efficiency at parsing.

The grammar file of the parser is located in parser/grammar.js. See the Tree-sitter documentation for more information on editing the grammar.js file.

Parsing process

Steppable code is executed while parsing, i.e., no bytecode or immediate products are produced.

The parser, written in C++, relies on the Tree-sitter C bindings. It recursively descends down the code tree and executes instructions in order.

Adding new features to the language

Before trying to expand Steppable based on the current feature set, please note the following.

1. Do not add small features as new syntaxes. Instead, try adding them as new functions.
2. Do not change anything in the current syntax. Make sure the updated grammar is still backwards-compatible.
3. Do not add or remove operators.
4. Do make sure that the new syntax does not collide with any of the existing syntaxes.

Make sure that your feature conforms to all the above-mentioned rules before proceeding.

---

To add a new syntactical feature...

1. Decide

Decide if you want to add a new expression type, or a new statement type. If your feature is a new statement type, name it <my_feature>_statement or <my_feature>_stmt, where <my_feature> is the name of your new feature. Otherwise, name it <my_feature>_expression or <my_feature>_expr.

2. Add a new grammar rule

In grammar.js, add a new rule. See the Tree-sitter documentation for more details.

// ...
module.exports = grammar({
    name: "stp",
    conflicts: $ => [
        [$._expression, $.function_call],
        // add if needed / prompted
    ],

    rules: {
        // ...
        my_syntax_statement: $ => seq(
            "my_syntax",
            $._expression
        ),
    }
});

You will also need to register your syntax feature to either $.statement or $._expression.

// ...
module.exports = grammar({
    // ...
    rules: {
        statement: $ => choice(
            // ...
            $.my_syntax_statement, // <- Add this
        ),

        // ...

        _expression: $ => choice(
            // ...
            $.my_expr, // <- Add this
        )
    }
});

3. Update parser.c

The parser runtime C library needs to be regenerated and recompiled every time the grammar file is updated. To do this, run the following in the parser directory.

npm run gen

4. Add C++ handling

To add handling for your new grammar...

If your feature is a new expression

1. Add a new part in the if-else statement handleExpr function in parser/src/stpExprHandler.cpp.

   STP_LocalValue handleExpr(const TSNode* exprNode,
                             const STP_InterpState& state,
                             const bool printResult = false,
                             const std::string& exprName = "")
   {
       assert(exprNode != nullptr);
       if (ts_node_type(*exprNode) == "ERROR"s or ts_node_is_missing(*exprNode))
           STP_throwSyntaxError(*exprNode, state);
   
       std::string exprType = ts_node_type(*exprNode);
   
       STP_LocalValue retVal(STP_TypeID_NULL);
   
       if (exprType == "number")
       {
           // ...
       }
       // ...
       if (exprType == "my_expression")
       {
           // handle your expression
       }
   }

   You may choose to add the handling function in the if statement, or write a separate function in another C++ file. However, all C++ files created need to be added to the CMakeLists.txt file.