Python / Flask Mini-Compiler

A mini-compiler front-end built with ANTLR 4.13.2 and Java that parses a single-file Flask web application: Python code, embedded HTML templates (inside """...""" strings), Jinja2 template syntax ({{ ... }} and {% ... %}), and CSS (inside <style> blocks).

The compiler runs all phases: lexing → parsing → two ASTs (Python + Jinja2) → symbol table → semantic analysis (10 checks) → code generation to a pure HTML/CSS/JS/Python website (no Flask, no Jinja2).

---

Pipeline

 tests/*.txt  (Flask app source)
      │
      ▼
 ┌─────────────────┐   tokens   ┌─────────────────┐   parse tree
 │  PythonLexer    │ ─────────► │  PythonParser   │ ─────────────┬──────────────────┐
 │  (5 modes)      │            │  (ANTLR)        │              │                  │
 └─────────────────┘            └─────────────────┘              ▼                  ▼
                                                     ┌──────────────────────┐ ┌───────────────────────┐
                                                     │  PythonBaseVisitor   │ │  SymbolTableVisitor   │
                                                     │  → AST 1 (Python)    │ │  → SymbolTable rows   │
                                                     │  → AST 2 (Jinja2)    │ └───────────┬───────────┘
                                                     └──────────┬───────────┘             │
                                                                │                         ▼
                                                                │             ┌───────────────────────┐
                                                                │             │  SemanticAnalyzer     │
                                                                │             │  10 checks →          │
                                                                │             │  src/error/semantic.txt│
                                                                │             └───────────────────────┘
                                                                ▼
                                                     ┌──────────────────────┐
                                                     │  Generator (phase 6) │
                                                     │  AST1 data → AST2 →  │
                                                     │  generated/ website  │
                                                     └──────────────────────┘

Project Structure

Path	Purpose
src/gen/PythonLexer.g4	Lexer grammar — 5 lexical modes (Python, HTML, Jinja2 expr, Jinja2 stmt, CSS)
src/gen/PythonParser.g4	Parser grammar — depth-checked indentation (block[int d] + indents() predicate), Python, HTML, Jinja2, CSS
src/gen/*.java	ANTLR-generated lexer, parser, visitor base classes
src/classes/Node.java	Abstract base of ALL AST nodes — stores node name + line number, polymorphic print()
src/classes/Jinja2Tree.java	Root of AST 2 + the bindings map the Generator fills with AST-1 data
src/classes/	~150 AST node classes (one class per grammar alternative), all extending Node
src/PythonBaseVisitor.java	Visitor that builds BOTH ASTs from the parse tree
src/SymbolTableVisitor.java	Second visitor: symbol table rows + the facts the semantic checks read
src/symbolTable/SymbolTable.java, Row.java	Symbol table structure + lookup helpers + printing
src/error/	Semantic analysis: Error interface → abstract BaseError → 10 checks + SemanticAnalyzer
src/generator/	Code generation: Generator, TemplateCompiler, CssGenerator
src/Main.java	Entry point: runs every phase for each test program
tests/	6 test programs (see Test Programs below)
generated/	Code-generation output: the pure HTML/CSS/JS/Python website

Lexer Modes

The lexer switches modes so the same characters mean different things in different contexts:

Mode	Entered by	Exited by	Handles
DEFAULT	—	—	Python/Flask keywords, operators, literals, INDENT (4 spaces)
HTML	"""	"""	HTML tags, attributes, text content
JINJA2_EXPR	{{	}}	Jinja2 expressions, filters (|), comparisons
JINJA2_STMT	{%	%}	if/elif/else/endif, for/endfor, set, block, extends, include
CSS	<style>	</style>	Selectors, declarations, units (px, em, %…), colors

AST — Two Trees

The compiler builds two separate abstract syntax trees, as required:

1. AST 1 — Python (root: Program): all Python statements. Where a """...""" template appears, the Python tree holds only a lightweight reference node (ExprHtmlString / ExprTemplate) that prints as -> Jinja2Tree 'TEMPLATE'.
2. AST 2 — Jinja2 (root: src/classes/Jinja2Tree.java): the full template content (HTML tags, Jinja2 {{ }} / {% %} nodes, CSS rules), extracted from the Python tree. Each tree is named after the Python variable it was assigned to (e.g. TEMPLATE), or template#N for inline templates. Program.printTree() prints AST 1 followed by every Jinja2 tree.

Both trees are produced by PythonBaseVisitor (extends the generated PythonParserBaseVisitor). Every grammar alternative has a labeled name (#exprAddSub, #j2ForBlock, …) and a matching Java class in src/classes/.

All ~150 node classes inherit from the abstract base src/classes/Node.java, which stores the node name (derived from the class name) and the source line number in every node — both passed in by the visitor via ctx.start.getLine(). Node provides:

• getNodeName() / getLine() — node metadata accessors
• header() — "NodeName [line N]" prefix used by every node's toString()
• print() — polymorphic per-node print method
• abstract toString() — each subclass overrides it (polymorphism)

Program.printTree() prints the entire tree by dispatching these polymorphic methods, so every printed node shows its name, line number, and children with readable indentation (shared Indent helper).

Python indentation is enforced by the parser: block[int d] carries the nesting depth and a semantic predicate (indents(n) in PythonParser.g4) checks the exact INDENT count of each line, so dedented statements correctly close their block instead of being swallowed by it.

High-level shape for the product-page test:

AST 1: Program                                    AST 2: Jinja2Tree 'TEMPLATE'
├── ImportFrom (flask → Flask, request, …)        └── HtmlContent
├── AssignStatement (app = Flask(__name__))           └── HtmlRegularTag <html>
├── AssignStatement (products = [ {...}, {...} ])         ├── <head> → <title>,
├── FunctionDef get_next_id                               │    HtmlStyleTag → CssContent → CssRule*
│   └── Block → Assign / For / If / Return                └── <body>
├── AssignStatement (TEMPLATE =                               ├── <div class="top-bar"> …
│      ExprHtmlString → Jinja2Tree 'TEMPLATE')  ──────►       ├── J2IfBlock (page == 'home')
├── FunctionDef home / add_product /                          │   └── J2ForBlock (product in products)
│      product_details / delete_product                       │       └── tags, Jinja2Expr {{ product.name }}
│   └── Decorator (@app.route(...)), Params, Block            ├── J2IfBlock (page == 'add') → form
└── IfStatement (__name__ == "__main__")                      └── J2IfBlock (page == 'details') → card

Symbol Table

SymbolTableVisitor walks the parse tree with a namespace stack — entering a function/class/template/CSS block pushes a scope; leaving pops it. SymbolTable provides helper operations: addRow, lookup(key), lookup(key, namespace), contains, rowsInNamespace, removeRow, size, and print. Each entry is a Row:

Column	Example
LINE	179
NAMESPACE	home::template
KEY	products
KIND	import, function, class, parameter, variable, global-ref, jinja2-var, jinja2-loop-var, jinja2-set-var, css-class, css-id, css-element

Semantic Analysis

After the symbol table is built, error.SemanticAnalyzer runs 10 semantic checks against it. The design demonstrates the required abstraction + inheritance + polymorphism:

interface Error                (src/error/Error.java)
        ▲
abstract class BaseError       (src/error/BaseError.java)
  — template method run(), record(), scope-aware lookup helpers,
    shared constants (PYTHON_BUILTINS, FLASK_BUILTINS, JINJA2_GLOBALS)
        ▲
10 concrete check classes      (one class per error kind)

Code	Class	Detects
P1	DuplicateFunctionError	Two functions with the same name in the same scope
P2	DuplicateParameterError	Two parameters with the same name in one function
P3	UndefinedVariableError	Identifier used but never declared anywhere
S1	ScopeError	Identifier declared only in a non-enclosing scope (e.g. another function's local)
T1	TypeError	Invalid operand types in arithmetic, e.g. "price: " + 100
T2	TypeMismatchError	Return value contradicts the -> type annotation, e.g. def f() -> int: return "free"
F4	MissingFlaskAppError	@app.route / app.run() used but app = Flask(__name__) never created
F2	DuplicateEndpointError	Two route functions share the same endpoint name
F3	NoReturnInRouteError	Route function has no return statement
J1	UndefinedTemplateVarError	{{ var }} never passed to render_template_string()

The checks are driven entirely by the symbol table: SymbolTableVisitor records extra fact kinds (usage, route-function, template-kwarg, binop, return-annotation, return-literal-type, flask-app, route-app-ref, …) and each check scans the rows.

Results go to src/error/semantic.txt (one section per check, per compiled file) and a summary is printed to stdout. tests/semantic-errors.txt intentionally triggers all 10 checks; the four clean test programs report none.

Code Generation

Phase 6 (src/generator/) compiles the Flask + Jinja2 source into a website made of pure HTML + CSS + JavaScript + Python — no Flask, no Jinja2:

AST 1 (Python)  ── products data array ──────────────►  data.js
                ── routes + render kwargs (page=...) ─►  which pages exist, url_for targets
                        │ bindData() — the Generator passes the data
                        ▼ from the Python tree into the second tree
AST 2 (Jinja2)  ── compiled once per page ───────────►  index.html / add.html / details.html
                     {% if page == .. %}  evaluated at compile time (page pruning)
                     {% for %} / {{ }}    compiled to JS render functions →  app.js
                ── <style> subtree ──────────────────►  style.css
stdlib-only web server ──────────────────────────────►  server.py

File	Role
generator/Generator.java	Orchestrator: extracts the products array + route/page map from AST 1, binds the data into the Jinja2 tree, writes all files
generator/TemplateCompiler.java	Compiles AST 2 per page: static HTML for compile-time parts, JS template-literal render functions for dynamic parts; rewrites Flask forms to addProduct(this) / deleteProduct(id)
generator/CssGenerator.java	Emits style.css from the CSS subtree

Output (in generated/): index.html, add.html, details.html, style.css, data.js, app.js, server.py. Add/delete changes persist via localStorage; url_for(...) becomes real links (details.html?id=1), so all four pages navigate into each other. Run it with:

python generated\server.py     # then open http://localhost:8000

Generation runs automatically when Main compiles tests/product-page.txt.

Generator scope

The generator supports the language subset defined by the grammar, demonstrated on the required product-store application (as is standard for a course compiler). Its assumptions:

• The data array is named products and is a list of dict literals.
• One template per program, organized as top-level {% if page == '...' %} sections; pages are discovered from render_template_string(TEMPLATE, page="...") kwargs in the routes.
• url_for(...) takes at most one keyword argument (becomes ?id=...); a route with no render call (e.g. delete_product) is compiled to the JS deleteProduct() action.
• The generated addProduct() runtime reads the form fields name/price/image/details; the details page looks products up by ?id=.
• Jinja2 filters (|), {% set %}, {% block %} / {% extends %} / {% include %} are parsed but rejected by the generator with a clear "unsupported" message.

A different app that follows the same patterns (other data values, pages, CSS, more routes) generates correctly — the data extraction, page pruning, loops, {{ }} interpolation, and link resolution are all generic.

Build & Run

Requires JDK and antlr-4.13.2-complete.jar (expected one directory above the project root).

# from the project root
javac -cp "..\antlr-4.13.2-complete.jar" -d out\production\PythonCompiler (Get-ChildItem src -Recurse -Filter *.java).FullName
java  -cp "out\production\PythonCompiler;..\antlr-4.13.2-complete.jar" Main

Or simply run Main from IntelliJ IDEA (the module already references the ANTLR jar).

With no arguments, Main compiles all six test programs in order, printing both ASTs, the symbol table, and the semantic-error summary for each (code generation runs on product-page.txt). Passing a file path as the first argument compiles only that file.

If the grammar files change, regenerate the parser/lexer:

cd src\gen
java -jar ..\..\..\antlr-4.13.2-complete.jar -visitor -package gen PythonParser.g4 PythonLexer.g4

Test Programs

Three standalone test programs (one per required scenario) plus the full combined app:

File	Scenario
tests/1-view-products.txt	View products — home() route, Jinja2 {% for %} product grid
tests/2-add-product.txt	Add product — HTML form, request.form, get_next_id(), redirect
tests/3-product-details.txt	Product details — find-by-id loop, abort(404), details card
tests/4-delete-product.txt	Delete product — per-product delete form, filter loop, redirect
tests/product-page.txt	Full app combining all pages including delete
tests/semantic-errors.txt	Intentionally broken app that triggers all 10 semantic checks

Each product has: image, name, price, details. Every test file is real runnable Flask code — you can also execute it with Python to demo the web interfaces.

Project Status (vs. the project-2 brief)

#	Requirement	Status
1	Lexer/Parser for Python, Jinja2, HTML, CSS	✅ Done
2	Two ASTs (Python + Jinja2) with the Generator passing the data array into the second tree	✅ Done (Jinja2Tree + Generator.bindData())
3	Nodes with OOP / inheritance / polymorphism + name & line per node	✅ Done (Node base class)
4	Semantic analysis, ≥ 5 errors, both parts	✅ Done (10 checks: Python, Flask, Jinja2)
5	Code generation — generated parts work together	✅ Done (generated/ website, verified end-to-end)
6	Interfaces: view / add / details / delete + smooth navigation	✅ Done (source apps + generated site)
7	Per-node printing + full tree + symbol table	✅ Done
—	Written report + AST diagram + group-members file for submission	❌ Still to produce

Known Issues / Limitations

1. Index printing: IdentifierAccess.toString() prints subscripts without brackets — p["id"] appears as pExprLiteral{ "id" } instead of p["id"].
2. Only 4-space indentation is supported; tabs/2-space indents are skipped silently.
3. Generator scope limits — see “Generator scope” above.