tree-sitter-codon

A tree-sitter grammar for Codon, the high-performance Python compiler from Exaloop.

Overview

Codon is a Python compiler that produces native code as fast as C/C++. This grammar enables syntax highlighting, code navigation, and analysis for Codon source files.

Codon-Specific Features Supported

Beyond standard Python syntax, this grammar supports Codon-specific extensions:

• Pipe operators: |> and ||> for data flow pipelines
• LLVM/Python extern blocks: @llvm and @python decorated functions with inline code
• Static type syntax: Int[64], Ptr[float], List[T]
• Range expressions: 1...10 (inclusive range)
• Directives: ## codon: key = value
• C imports: from C import foo: int
• Generic type parameters: def foo[T](x: T) -> T
• Match statements: Full Python 3.10+ pattern matching

Installation

npm (Node.js)

npm install tree-sitter-codon

pip (Python)

pip install tree-sitter-codon

Building from Source

git clone https://github.com/phdyex/tree-sitter-codon.git
cd tree-sitter-codon
npm install
npm run build

Usage

Python

import tree_sitter_codon
from tree_sitter import Language, Parser

# Get the Codon language
lang = Language(tree_sitter_codon.language())

# Create a parser
parser = Parser(lang)

# Parse some Codon code
source = b'''
@par
for i in range(1000):
    total += process(i)
'''

tree = parser.parse(source)
print(tree.root_node.sexp())

JavaScript/Node.js

const Parser = require('tree-sitter');
const Codon = require('tree-sitter-codon');

const parser = new Parser();
parser.setLanguage(Codon);

const source = `
@llvm
def add(a: int, b: int) -> int:
    %tmp = add i64 %a, %b
    ret i64 %tmp
`;

const tree = parser.parse(source);
console.log(tree.rootNode.toString());

CLI

# Parse a file
tree-sitter parse example.codon

# Generate syntax tree
tree-sitter parse --stat example.codon

# Run in playground
npm run start

Grammar Highlights

Pipe Expressions

Codon's pipe operators enable functional data flow:

# Forward pipe
data |> process |> filter |> output

# Parallel pipe
items ||> expensive_computation

Extern Blocks

Inline LLVM IR or Python code:

@llvm
def fast_add(a: int, b: int) -> int:
    %result = add i64 %a, %b
    ret i64 %result

@python
def use_numpy(arr):
    import numpy as np
    return np.sum(arr)

Generic Types

def identity[T](x: T) -> T:
    return x

class Container[T]:
    value: T

Range Expressions

# Inclusive range
for i in 1...10:
    print(i)

Development

Generate Parser

npm run generate

Run Tests

npm test

Build WASM

npm run build-wasm

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

Adding Tests

Test cases go in test/corpus/. Each file contains test cases in this format:

================================================================================
Test Name
================================================================================

source code here

--------------------------------------------------------------------------------

(expected_syntax_tree)

================================================================================

License

Apache License 2.0

Related Projects

• Codon - The Codon compiler
• tree-sitter - Parser generator
• tree-sitter-python - Python grammar

References

• Codon Documentation
• Codon Paper (CC'23)
• Tree-sitter Documentation