Architecture and Pipeline

Architecture & Pipeline

Spec reference: MASTER_SPEC §8.

The compiler is a sequence of independent, deterministic transformations. Each stage receives immutable input and produces new immutable output; no stage modifies a previous representation.

Pipeline overview

Natural Language
  → Scene Analyzer      → Scene JSON
  → Scene Validator     → Validated Scene JSON
  → Resolver            → Prompt

The Resolver resolves concepts to Knowledge Base tags and translates them into a single flat prompt string, in resolution order. There are no categories and no separate Category Splitter or Prompt Builder stage.

Each stage produces a new representation of the same scene. Where a stage boundary matters, "Raw Scene JSON" (Analyzer output) and "Validated Scene JSON" (Validator output) are distinguished; otherwise both are just "Scene JSON".

Reference algorithm

Behaviour is defined by this reference; implementations may differ internally.

function Compile(input):
    scene    = Analyze(input)          # Natural Language → Scene JSON
    scene    = ValidateScene(scene)    # schema + normalization
    resolved = Resolve(scene)          # concepts → Illustrious tags (+ dedup)
    prompt   = TagsToPrompt(resolved)  # tags → one flat prompt string
    return prompt

Stages at a glance

#	Stage	Input	Output	Allowed	Forbidden
1	Scene Analyzer	Natural language	Scene JSON	Extract explicit concepts, preserve ambiguity	Generate tags, optimize, guess
2	Scene Validator	Scene JSON	Validated Scene JSON	Validate schema, normalize, remove invalid fields, warn	Create/guess concepts, generate tags
3	Resolver	Validated Scene JSON	Prompt	Normalize, resolve aliases, look up, expand, generate tags, dedup, join into a flat prompt, warn	Analyze language, invent, optimize, call an LLM

The Analyzer's reference extraction order is deterministic: characters → environment → objects → interactions → camera → lighting.

The pipeline and its stages are unchanged in V2; the V2 additions live inside the Resolver (an opt-in semantic fallback, modifier preservation, optional parallel resolution) and in Knowledge Base tooling. The Knowledge Base now loads once per process and reuses a compiled lookup table, so output is identical but faster.

Pipeline invariants

• Each representation exists only within its window: natural language before the Analyzer; Scene JSON before the Resolver; the flat prompt after it.
• Each stage has exactly one responsibility.
• No stage may bypass another stage.
• Every stage must be independently testable.

Immutability & data ownership

Each stage owns only its output and treats every transformation as immutable:

Input → Transform → Output   (Input remains unchanged)

Stages must not modify the outputs of previous stages.

Compiler Result wrapper

Every stage returns the same wrapper object, exposing debugging information behind a uniform interface:

{
  "data": {},
  "warnings": [],
  "errors": [],
  "metadata": {}
}

Traceability

Every output element is traceable back through:

Natural Language → Scene JSON → Knowledge Base Entry → Resolved Tag → Prompt

Failure policy

• Compilation should continue whenever possible.
• Warnings never stop compilation.
• Errors stop the affected compilation.
• Fatal conditions stop the compiler entirely.

See Configuration, Errors & Logging for the severity model and message codes.

Related pages

• Data Model
• Scene JSON & Schemas
• ComfyUI Nodes

---

A note on AI assistance

This project — its code, tests, and documentation — is built with heavy help from AI coding assistants and maintained by a solo author. AI makes mistakes: despite testing and review, parts of this may be inaccurate, out of date, or wrong. Treat it as a good-faith best effort, not authoritative truth — if the docs and the code disagree, the code wins. Please open an issue when something looks off.