FAQ

General Questions

What is CDTk?

CDTk (Compiler Description Toolkit) is a .NET library for building compilers, transpilers, and code generators. It provides a unified API centered around three components:

1. TokenSet (for tokens)
2. RuleSet (for grammar rules)
3. MapSet (for code generation)

Starting with Version 7, CDTk introduces a strongly-typed API, enhancing safety, clarity, and IDE support.

---

Who should use CDTk?

CDTk is ideal for:

• Building domain-specific languages (DSLs)
• Creating code generators and transpilers
• Prototyping new programming languages
• Teaching compiler construction principles
• Transforming data between formats (e.g., JSON to XML, SQL to NoSQL)

---

What .NET versions are supported?

CDTk targets .NET 10.0 and above.

---

Getting Started

How do I create my first compiler?

Here’s a minimal example using the fully-typed API:

// Define token types
class Number { }
class Identifier { }

// Define tokens
var tokens = new TokenSet
{
    new Token<Number>(@"\d+"),
    new Token<Identifier>(@"[A-Za-z_][A-Za-z0-9_]*")
};

// Define grammar rules
class AssignmentNode { }
var rules = new RuleSet
{
    new Rule<AssignmentNode>("variable:@Identifier '=' value:@Number")
        .Returns<AssignmentNode>("variable", "value")
};

// Define code generation
var mapping = new MapSet
{
    new Map<AssignmentNode>("let {variable} = {value};")
};

// Build and compile
var compiler = new Compiler()
    .WithTokens(tokens)
    .WithRules(rules)
    .WithTarget(mapping)
    .Build();

var result = compiler.Compile("x = 42;");
Console.WriteLine(result.Output);  // Output: let x = 42;

For more detailed examples, see Examples.md.

---

Where can I find examples?

See Examples.md for:

• Calculators
• Mini languages
• JSON parsers
• Multi-target transpilers
• Domain-specific languages (DSLs)

---

Tokens

My tokens aren't matching correctly

Check token order. Tokens are processed in the order they are defined. The first matching token is selected:

// Wrong: Identifier can match keywords
new Token<Identifier>(@"[A-Za-z_][A-Za-z0-9_]*"),    // Too generic
new Token<Keyword>(@"if|else|while"),                // Never gets matched

// Correct: Define keywords first
new Token<Keyword>(@"if|else|while"),
new Token<Identifier>(@"[A-Za-z_][A-Za-z0-9_]*")

---

How do I ignore whitespace?

Use .Ignore() on tokens:

class Whitespace { }
new Token<Whitespace>(@"\s+").Ignore();

---

How do I handle multi-line comments?

Use RegexOptions.Singleline for patterns like /* comments */:

class BlockComment { }
new Token<BlockComment>(@"/\*.*?\*/", RegexOptions.Singleline).Ignore();

---

How do I match escaped characters in strings?

class StringLiteral { }
new Token<StringLiteral>(@"""([^""\\]|\\.)*""");

This supports escape sequences like \n, \t, and \".

---

Rules

My rule isn't matching

1. Token references should use @:

   // Wrong
   new Rule<Assignment>("Identifier '=' Number")
   
   // Correct
   new Rule<Assignment>("variable:@Identifier '=' value:@Number")

2. Verify literal formatting: Use '=' for exact matches.

3. Simplify debugging: Start with minimal rules and incrementally add complexity.

---

How do I handle operator precedence?

Use separate rules for each precedence level:

class ExprNode { }
class TermNode { }
class FactorNode { }

new Rule<ExprNode>("Term (('+' | '-') Term)*")
new Rule<TermNode>("Factor (('*' | '/') Factor)*")
new Rule<FactorNode>("@Number | '(' ExprNode ')'")

---

What is left recursion and why is it bad?

Left recursion causes infinite loops:

// BAD: Left recursive
new Rule<Expression>("Expression '+' Term")

// GOOD: Iterative
new Rule<Expression>("Term ('+' Term)*")

---

How do I make optional elements?

Use ? for optional components:

class VarDeclNode { }
new Rule<VarDeclNode>("@Type @Identifier ('=' Expression)?")
    .Returns<VarDeclNode>("type", "name", "init");

---

How do I parse lists?

Use * (zero or more) or + (one or more):

class BlockNode { }
new Rule<BlockNode>("'{' Statement* '}'");

class ArgList { }
new Rule<ArgList>("Expression (',' Expression)*");

---

Code Generation

My mappings aren't generating output

1. Check node type names: Node types in your Map must match .Returns() declarations.

2. Verify placeholder names: Ensure placeholders in templates ({fieldName}) match field names defined in .Returns().

---

How do I transform data types?

Use specific node types in your rules and mappings:

class IntNode { }
class StringNode { }

new Map<IntNode>("int32_t {variable}")
new Map<StringNode>("char* {variable}")

---

Can I use conditional logic in mappings?

Yes, use Validate or advanced custom rules:

class ValidatedNode { }
new Map<ValidatedNode>("Validated Output");

Alternatively, map different nodes using specific cases like:

Class ComplexRule future context handling wanceled {"QUICK WALKTHRU"}