Learn

Learn CDTk: Getting Started

Welcome to CDTk! This guide introduces the basics of building a compiler in 3 steps:

1. TokenSet: Recognize lexical patterns (tokens).
2. RuleSet: Structure grammar rules.
3. MapSet: Define how parsed input transforms into output.

---

Start Here: The Basics

CDTk simplifies compiler design with its unified and fully-typed pipeline:

• TokenSet: Identify and categorize input patterns (e.g., numbers, keywords, operators).
• RuleSet: Combine tokens into meaningful structures using grammar rules.
• MapSet: Convert parsed structures into your desired output (e.g., code, JSON, config).

class Number { }
class Identifier { }
class AssignmentNode { }

var tokens = new TokenSet
{
    new Token<Number>(@"\d+"),
    new Token<Identifier>(@"[A-Za-z_][A-Za-z0-9_]*")
};

var rules = new RuleSet
{
    new Rule<AssignmentNode>("variable:@Identifier '=' value:@Number")
        .Returns<AssignmentNode>("variable", "value")
};

var mappings = new MapSet
{
    new Map<AssignmentNode>("int {variable} = {value};")
};

var compiler = new Compiler()
    .WithTokens(tokens)
    .WithRules(rules)
    .WithTarget(mappings)
    .Build();

var result = compiler.Compile("x = 42;");
Console.WriteLine(result.Output);  // Outputs: int x = 42;

---

The Pipeline

When you call compiler.Compile(input), CDTk processes the input systematically:

1. Lexing (TokenSet): Converts input text into tokens (e.g., Identifier("x"), Equals("="), Number("42")).
2. Parsing (RuleSet): Organizes tokens into an AST (Abstract Syntax Tree).
3. Code Generation (MapSet): Transforms the AST into output (e.g., int x = 42;).

Learn More: Check out the Pipeline Overview.

---

Defining Your Compiler

TokenSet

Tokens define basic patterns in your language. For example:

var tokens = new TokenSet
{
    new Token<Identifier>(@"[A-Za-z_][A-Za-z0-9_]*"),  // Variables
    new Token<Number>(@"\d+"),                        // Numbers
    new Token<Equals>("="),                           // Assignment operator
    new Token<Whitespace>(@"\s+").Ignore()            // Ignored whitespace
};

Learn More: See Tokens.

RuleSet

Rules describe your grammar in a declarative syntax:

var rules = new RuleSet
{
    new Rule<AssignmentNode>("variable:@Identifier '=' value:@Number")
        .Returns<AssignmentNode>("variable", "value")
};

Learn More: See Rules.

MapSet

Define templates for code generation or output transformation:

var mappings = new MapSet
{
    new Map<AssignmentNode>("int {variable} = {value};")  // Outputs C syntax
};

Learn More: See Mapping.

---

Compiling Your Input

Combine the TokenSet, RuleSet, and MapSet into Compiler:

var compiler = new Compiler()
    .WithTokens(tokens)
    .WithRules(rules)
    .WithTarget(mappings)
    .Build();

var result = compiler.Compile("x = 42;");
Console.WriteLine(result.Output);  // Outputs: int x = 42;

---

Debugging and Next Steps

Debugging

If you encounter issues:

1. Unexpected Tokens: Adjust your TokenSet definitions.
2. Grammar Errors: Refine your RuleSet for clarity.
3. Missing Outputs: Confirm MapSet covers all node types.

Learn More: See the Debugging Guide.

---

Explore Further

CDTk offers advanced features and infinite customization:

• Tokens: Dive deeper into token definitions.
• Rules: Learn about recursive structures and alternations.
• Mapping: Explore multi-target generation.
• Examples: Study real-world compiler setups.