Overview and Vision

Overview & Vision

Spec reference: MASTER_SPEC §1–3.

Purpose

Scene Compiler is a deterministic prompt compiler for ComfyUI. It transforms a natural-language scene description into a structured prompt compatible with Danbooru-format image-generation models. It targets the Illustrious model family exclusively, while the architecture is designed to support other model families in future versions without structural change.

The problem

Current prompt-generation workflows rely on a Large Language Model to generate Danbooru tags directly:

Natural Language → LLM → Prompt

This is unsuitable for deterministic generation. LLMs routinely:

• invent scene elements that were never described (clothing, poses, lighting, environments);
• emit invalid tags;
• produce inconsistent prompts; and
• return different outputs for identical inputs.

Tools such as TIPO reduce but do not eliminate this, because they still delegate tag generation to the language model.

The vision

Scene Compiler takes a fundamentally different position:

The language model understands the scene; the compiler generates the prompt.

It treats prompt generation as a compilation problem rather than a text-generation problem. It receives a natural-language description, extracts explicit semantic information, converts it into an intermediate representation, resolves every concept against a deterministic Knowledge Base, and only then produces Illustrious tags.

Two problems are kept completely independent:

• Language understanding — performed once, by the Scene Analyzer.
• Prompt generation — performed deterministically, by the rest of the pipeline.

Version 1 is a foundation. Future versions extend the compiler; they do not replace it.

How Version 2 evolved this

Version 1 proved the position above. Version 2 kept every guarantee and changed two things: the compiler knows more and asks less.

Know more. V1 was honest but narrow — a concept missing from the Knowledge Base was simply dropped, and the Knowledge Base only grew by hand.

• The Knowledge Base grows itself. A fully automated pipeline regenerates it from committed Danbooru snapshots. There is deliberately no approval gate: an automation that stops to ask a human is an automation that does not run. Generated entries are accepted by rules, heuristics, and confidence — never by manual review.
• Near-misses are caught, not dropped. An opt-in semantic fallback resolves concepts that miss exact lookup. It is off by default, deterministic when enabled, and can only return an entry that already exists in the Knowledge Base — so principle 3.4 (no hallucinations) still holds exactly. Deterministic lookup always wins.
• Detail stops leaking. Silently discarding part of what the user wrote is now treated as a bug, not as tidiness: modifiers survive, every list item is transcribed, and relationships are kept rather than flattened to a bare noun. The same fidelity applies to explicit content.

Ask less. V1 exposed nearly every internal as a node or a widget. That is honest, but it pushes the compiler's internal shape onto the user.

• One place for settings. Every option lives on the Configuration node; stage nodes take a single optional config connection instead of their own copies.
• Fewer nodes. Stages that carried no real decisions were folded away — the Category Splitter and Prompt Builder into the Resolver (V1.1), the Knowledge Base Loader into the Configuration node (V2). Eight nodes became five.
• Do not ask what the system knows. The Knowledge Base ships in the package, so its path is no longer a question. Options that are reserved or not genuinely user-facing are not exposed; they keep working defaults.

This is a refinement of principle 3.10 (Nodes Are Interfaces): a node exists to move data, not to display a setting. Nothing here weakens determinism — every V2 addition is opt-in, bounded to the Knowledge Base, and reproducible.

Scope

In scope: natural-language analysis and scene extraction; Scene JSON generation; deterministic tag resolution against the Knowledge Base; scene validation; category assignment (as traceability metadata); flat prompt translation; automatic Knowledge Base growth; the opt-in semantic fallback; EasyIllustrious integration (via connectable outputs); Ollama integration (default Analyzer backend).

Out of scope (must NOT be added):

• Automatic creativity, prompt optimization, enhancement, or beautification.
• Style invention or artist selection/recommendation.
• Automatic generation of camera, lighting, environment, composition, or quality information.
• Automatic character completion (inventing hair, eyes, clothing, accessories, expressions, poses, or relationships).

Governing rule: if information is not explicitly present in the scene description, the compiler MUST NOT invent it.

Non-goals

These are intentional design boundaries, not missing features:

• The compiler is not a prompt generator — prompt optimization belongs to downstream workflows.
• The compiler is not an AI artist — it makes no artistic decisions.
• The compiler does not hallucinate — missing information stays missing; unknown concepts stay unknown.
• The compiler does not replace EasyIllustrious — it provides structured semantic outputs only.
• The compiler does not replace the user — an incomplete description yields an incomplete output.
• No automatic prompt engineering — it never injects masterpiece, best quality, detailed, etc.
• No style decisions and no image-model knowledge beyond the Resolver.

Design principles

These principles define the identity of the project and are mandatory.

#	Principle	Meaning
3.1	Determinism First	The compiler core is deterministic: a given Scene JSON, Knowledge Base, and config → identical output, with no random seeds. The one non-deterministic step is the LLM understanding the language up front; everything downstream is reproducible.
3.2	Analyze Before Resolving	Language → concepts (Scene JSON) → tags. The intermediate representation is never skipped.
3.3	Explicit Over Implicit	Only explicitly described information may appear in Scene JSON.
3.4	No Hallucinations	Preserve uncertainty rather than invent. Fewer correct tags beat more wrong tags.
3.5	Traceability	Every tag is explainable along the full chain.
3.6	Separation of Concerns	Every stage has exactly one responsibility.
3.7	Immutable Pipeline	Each stage produces new immutable output; nothing mutates a previous representation.
3.8	Knowledge Is Data	Knowledge lives in the Knowledge Base, never in code.
3.9	The Resolver Owns Model Knowledge	Only the Resolver turns concepts into Illustrious tags.
3.10	Nodes Are Interfaces	ComfyUI nodes expose functionality; they contain no compiler logic.
3.11	Human Readability	Knowledge Base entries and IRs are understandable without reading source.
3.12	Simplicity & Long-Term Stability	Simple deterministic rules over complex heuristics; Scene JSON is a stable public interface.
3.13	Fidelity (V2)	Described detail must reach the prompt or be reported. Silently dropping a modifier, a list item, or a relationship is a bug — including in explicit content.
3.14	Automation Without Gates (V2)	Anything the compiler can do for itself, it does unattended. Approval gates and interactive prompts are anti-patterns; the Knowledge Base Editor is the only manual surface and is off the critical path.
3.15	Ask Only What Matters (V2)	Settings live in one place, and the user is not asked for anything the system already knows or is not ready to expose. A node moves data; it is not a settings panel.

Related pages

• Requirements & Success Criteria
• Architecture & Pipeline
• Roadmap

---

A note on AI assistance

This project — its code, tests, and documentation — is built with heavy help from AI coding assistants and maintained by a solo author. AI makes mistakes: despite testing and review, parts of this may be inaccurate, out of date, or wrong. Treat it as a good-faith best effort, not authoritative truth — if the docs and the code disagree, the code wins. Please open an issue when something looks off.