lambda-graphs: Multi-View Code Representation Tools

lambda-graphs aims to generate combined multi-code view graphs that can be used with various types of machine learning models (sequence models, graph neural networks, etc).

Overview

lambda-graphs is a CLI tool and Python library for generating code representations (AST, CFG, DFG) from source code. It supports C, C++, Java, and JavaScript, at both method-level and file-level granularity.

• AST (Abstract Syntax Tree)
• CFG (Control Flow Graph)
• DFG (Data Flow Graph)
• Combined graphs (any combination of the above)

lambda-graphs provides both a CLI and a Python API so you can generate graphs programmatically without shelling out. It is designed to be easily extendable to various programming languages. This is primarily because we use tree-sitter, a highly efficient incremental parser that supports over 40 languages.

---

Setup

1. Create a conda environment:

conda create -n lambda-graphs python=3.11

2. Activate the environment:

conda activate lambda-graphs

3. Install the package in development mode:

pip install -e .

4. Install GraphViz (Optional - for visualization):

GraphViz is only required if you want to generate DOT, PNG, or SVG output files.

Ubuntu/Debian:

sudo apt install graphviz

MacOS:

brew install graphviz

Windows: Download from graphviz.org

---

Generating Graphs

Using CLI

After setup, use the lambda-graphs command directly.

The attributes and options supported by the CLI are well documented and can be viewed by running:

lambda-graphs --help

Single File Analysis:

Generate a combined CFG and DFG graph for a C++ file:

lambda-graphs --lang "cpp" --code-file ./test.cpp --graphs "cfg,dfg"

Generate an AST for a C file with output in JSON format:

lambda-graphs --lang "c" --code-file ./example.c --graphs "ast" --output "json"

Folder Analysis (Multi-file Projects):

lambda-graphs can analyze entire projects by combining multiple source files from a folder:

lambda-graphs --lang "c" --code-folder ./project/src --graphs "cfg,dfg" --output "json"

This will:

1. Recursively scan the folder for all .c and .h files
2. Combine them into a single temporary file (preserving includes, declarations, definitions)
3. Generate the requested codeviews from the combined source
4. Output results to the output/ directory

You can customize the combined output file name:

lambda-graphs --lang "cpp" --code-folder ./mylib --combined-name "myproject" --graphs "ast,cfg"

Inline Code Analysis:

You can also analyze code snippets directly without a file:

lambda-graphs --lang "c" --code "int main() { int x = 5; return x; }" --graphs "ast,cfg"

Additional CLI Options:

Option	Description
--output	Output format: json, dot, svg, or all (dot generates PNG; svg generates SVG). Default: dot
--collapsed	Collapse duplicate variable nodes into a single node in AST
--last-def	Add last definition information to DFG edges (shows where variables were last defined)
--blacklisted	Comma-separated list of AST node types to exclude from the graph

Flag-Codeview Compatibility:

Flag	AST	CFG	DFG
--collapsed	✓	✗	✗
--blacklisted	✓	✗	✗
--last-def	✗	✗	✓
--last-use	✗	✗	✓

Examples:

# Generate all output formats (DOT, JSON, PNG)
lambda-graphs --lang "c" --code-file test.c --graphs "cfg" --output "all"

# Collapse duplicate variable nodes in DFG
lambda-graphs --lang "cpp" --code-file test.cpp --graphs "ast" --collapsed

# Add last definition tracking to DFG
lambda-graphs --lang "c" --code-file test.c --graphs "dfg" --last-def

# Exclude specific AST node types
lambda-graphs --lang "c" --code-file test.c --graphs "ast,cfg" --blacklisted "comment,string_literal"

Using the Python API

You can also use lambda-graphs as a library to get graph objects directly (no file I/O):

from lambda_graphs import generate

# -- Generate from a code string -------------------------------------------
result = generate(
    "cpp",
    code="int main() { int x = 5; return x; }",
    graphs=["ast", "cfg", "dfg"],
)

# Each graph is a networkx.MultiDiGraph
print(result.ast.nodes(data=True))
print(result.cfg.nodes(data=True))
print(result.dfg.nodes(data=True))
print(result.combined.nodes(data=True))

# Graph-level metadata (the "graph" key in JSON output)
print(result.combined.graph)  # {"language": "cpp", "views": ["ast", "cfg", "dfg"]}

# -- Generate from a file or folder ------------------------------------------
result = generate("cpp", code_file="./test.cpp", graphs=["cfg", "dfg"])
result = generate("c", code_folder="./project/src", graphs=["cfg", "dfg"])

# -- With extra options ------------------------------------------------------
result = generate(
    "cpp",
    code="...",
    graphs=["ast", "dfg"],
    collapsed=True,                              # collapse duplicate variable nodes
    last_def=True,                               # add last-def info to DFG edges
    blacklisted=["comment", "number_literal"],   # exclude AST node types
)

# -- Export to disk ----------------------------------------------------------
result.to_json("output.json")   # JSON
result.to_dot("output.dot")     # DOT
result.to_png("output.png")     # PNG image
result.to_svg("output.svg")     # SVG image

generate() returns a GraphsResult object with the following attributes:

Attribute	Type	Description
.ast	nx.MultiDiGraph | None	AST graph
.cfg	nx.MultiDiGraph | None	CFG graph
.dfg	nx.MultiDiGraph | None	DFG graph
.combined	nx.MultiDiGraph	Combined multi-view graph (always present)
.language	str	Source language

generate() parameters:

Parameter	Type	Required	Description
language	str	✓	Source language: "c" / "cpp" / "java" / "javascript"
code	str	pick one	Source code as a string
code_file	str|Path	pick one	Path to a source file
code_folder	str|Path	pick one	Path to a source folder (auto-merges multi-file projects)
graphs	list[str]		Which graphs to generate, default ["ast", "cfg", "dfg"]
collapsed	bool		Collapse duplicate variable nodes, default False
last_def	bool		Add last-def annotation to DFG edges, default False
last_use	bool		Add last-use annotation to DFG edges, default False
blacklisted	list[str]		AST node types to exclude
combined_name	str		Custom name for merged file (code_folder mode only)

See docs/json-output-format.md for details on the JSON output format.

---

Limitations

While lambda-graphs provides method-level and file-level support, it's important to note the following limitations and known issues:

General Limitations

• Syntax Errors in Code: To ensure accurate codeviews, the input code must be free of syntax errors. Code with syntax errors may not be correctly parsed and displayed in the generated codeviews. Note that the code does not need to be compilable, only syntactically valid.

C++ Specific Limitations

In addition to the general limitations, the tool has the following limitations specific to C++:

• Limited Template Metaprogramming Support: Complex template metaprogramming patterns may not be fully captured in the generated codeviews.

• Partial Preprocessor Directive Support: Preprocessor directives (e.g., #define, #ifdef) are parsed but not fully processed. Conditional compilation may not be accurately reflected in the codeviews.

• Limited Support for Advanced C++ Features: Some advanced C++ features such as:

  • Complex inheritance hierarchies
  • Multiple inheritance with virtual functions
  • Template specializations
  • SFINAE patterns
  • Concepts (C++20)

  may not be fully represented in the generated codeviews.

Acknowledgments

This tool builds upon the tree-sitter parsing framework and is inspired by research on source code representation learning for AI-driven software engineering tasks.