Rules

Rules in CDTk

CDTk’s grammar rules are the foundation for parsing source text into logical, tree-structured representations (ASTs). Rules dictate how tokens and sub-rules assemble into language constructs like expressions, assignments, or function calls. CDTk v8 introduces a robust, field-based identity system for rules and processes, which improves reliability and expressiveness in parsing.

---

What Is a Grammar Rule?

A grammar rule in CDTk does two things:

1. Describes a Pattern: Specifies the order and types of tokens or sub-rules it matches using a declarative syntax.
2. Constructs Output: Maps matched elements to named fields—usually for AST node construction.

Example:

public Rule Assignment = new Rule("variable:@Identifier '=' value:@Number")
    .Returns("variable", "value");

• variable:@Identifier binds a token (from the lexer) to the field variable.
• '=' matches a literal character.
• value:@Number maps another token to value.

---

Defining Rules in CDTk

Rules are defined as public Rule fields inside a class deriving from RuleSet. The field name is the unique rule identity.

Typical Usage:

class MyRules : RuleSet
{
    public Rule Expression = new Rule("left:@Term '+' right:@Term")
        .Returns("left", "right");

    public Rule Assignment = new Rule("variable:@Identifier '=' value:@Number")
        .Returns("variable", "value");
}

• Identity: The field name (Expression, Assignment) is the rule’s ID.
• Declarative Patterns: Patterns use tokens and other rules.
• Returns Clause: Maps rule outputs to fields used for AST construction.

---

Rule Syntax Cheat Sheet

Syntax Element	Meaning
@TokenName	Matches a token defined in your TokenSet
Field:RuleName	Binds nested rule output to a field name
'literal'	Matches a literal string or operator
`	`
*	Zero or more (repetition)
+	One or more (repetition)
?	Optional (zero or one)

Example Patterns:

1. Binary Expression:

   public Rule Binary = new Rule("left:@Number op:'+' right:@Number")
       .Returns("left", "op", "right");

2. Alternation:

   public Rule Factor = new Rule("@Number | '(' expr:Expression ')'")
       .Returns("expr");

3. Repetition:

   public Rule Args = new Rule("args:@Identifier+").Returns("args");

4. Optional:

   public Rule SemiExpr = new Rule("expr:Expression ';'?").Returns("expr");

---

Rule Validation

CDTk v8 includes validation utilities to check grammar and rule correctness before compilation.

How to Validate:

var rules = new MyRules();
var diagnostics = rules.Validate();

if (diagnostics.HasErrors)
{
    foreach (var diagnostic in diagnostics.Items)
        Console.WriteLine($"{diagnostic.Level}: {diagnostic.Message}");
}

Validation catches:

• Undefined tokens/rules
• Ambiguous or recursive structures
• Pattern sequence errors

---

AST Construction

Rule outputs are mapped to object fields for AST node creation:

public Rule AssignmentNode = new Rule("variable:@Identifier '=' value:@Number")
    .Returns("variable", "value");

• Maps matched @Identifier and @Number to the resulting node’s variable and value fields.

Nested rules build hierarchical ASTs:

public Rule Expression = new Rule("left:@Term '+' right:@Term")
    .Returns("left", "right");

---

Using Semantic Models

Advanced users may attach semantic models for deeper analysis or custom logic:

public MyModel Model => new MyModel(__AllRules, __Ast);

• Models process parsed AST nodes and enhance grammar capabilities.
• Models require explicit construction and type-safe logic—no magic injection.

---

Order & Rule Priority

Rules are matched in field definition order. Place the most specific or higher-priority rules first:

public Rule Keyword = new Rule("keyword:'if'");
public Rule Identifier = new Rule("id:@Identifier");

---

Best Practices

1. Descriptive Field Names: Use meaningful rule and field names for clarity.
2. Validate Often: Use .Validate() during development for correctness.
3. Correct Ordering: Place specific rules before general ones.
4. Extend with Models: Use semantic models to add custom AST transforms.

---

Debugging Rules

• Check token definitions (make sure TokenSet matches usage).
• Watch for conflicting alternation/repetition.
• Use .Validate() for early error detection.

---

Next Steps

Explore:

• Code Generation / Mapping
• Implementing Models
• Grammar Validation

For further details, refer to your source documentation or core CDTk parser files.