Repository navigation
Roadmap
Spec reference: MASTER_SPEC §29–30.
Following this order minimizes refactoring — dependencies always point downward, and no step depends on unfinished future work. Every completed phase must leave the repository in a working state. A smaller, stable Version 1 is preferred over an incomplete Version 2.
Historical record. This is the original Version 1 build plan, kept as-is. Some stages it lists no longer exist as separate components: V1.1 folded the Category Splitter and Prompt Builder into the Resolver, and V2 folded the Knowledge Base Loader node into the Configuration node. For what ships today, see ComfyUI Nodes.
| Phase | Name | Deliverables |
|---|---|---|
| 0 | Repository Initialization | Repo structure; linting, formatting, testing framework, CI; documentation skeleton. No compiler logic. |
| 1 | Foundation | JSON Schemas, data models, Compiler Result wrapper, configuration system, logging, Knowledge Base Loader. Every structure exists; nothing compiles yet. |
| 2 | Knowledge Layer | Knowledge Base format, alias system, expansion rules, category system, validation tools; first Knowledge Base dataset. |
| 3 | Compiler Core (no LLM) | Scene Validator, Resolver, Category Splitter, Prompt Builder + tests. Scene JSON → prompt outputs, without any LLM. |
| 4 | Scene Analyzer | Ollama interface, official system prompt, Scene JSON parser, response validation, retry logic + tests. Natural language now becomes Scene JSON. |
| 5 | ComfyUI Integration | Pipeline nodes + support nodes (Debug Viewer, Configuration). All nodes are thin wrappers. (V2 folded Knowledge Base loading into the Configuration node.) |
| 6 | Testing | Unit, integration, regression, Knowledge Base validation, and example-scene tests. |
| 7 | Integration & Examples | Example workflows, prompts, and Scene JSON; complete docs; benchmark dataset and performance measurements. |
| 8 | Release Preparation | Finalize regression suite; performance optimization; documentation and Knowledge Base review; package and release Version 1. |
Development rules: the compiler must remain functional after every phase; no phase may introduce unfinished architecture; refactoring is minimized; behavioural compatibility always has priority.
Release criteria for V1: the compiler converts natural language into categorized Illustrious tags; all stages are tested; the Knowledge Base validates; official example workflows execute without modification; documentation is complete; the compiler remains deterministic; and no mandatory feature relies on prompt generation.
Each version has a clearly defined scope. Features scheduled for future versions should not be implemented early unless explicitly required.
Objective: a complete deterministic compiler that transforms natural language into Illustrious-compatible prompts. The objective is reliability, not intelligence.
Core features: Scene Analyzer, Scene JSON, Knowledge Base, Scene Validator, Resolver (resolves and translates concepts into one flat prompt), EasyIllustrious integration, Ollama integration, Debug Viewer, configuration system, JSON Schemas, regression tests. (V1.1 removed the separate Category Splitter and Prompt Builder stages.)
Supported model family: Illustrious only. The architecture is model-independent, but V1 intentionally ships a single Resolver.
Explicitly excluded: semantic search, embeddings, Knowledge Base editor, automatic prompt optimization/beautification, graph representation, plugin system, multiple model families, automatic style/camera/lighting/composition/ quality generation.
Objective: improve concept resolution without sacrificing determinism. Semantic assistance is added while the compiler architecture stays unchanged.
Zero required user input. The whole pipeline keeps running end-to-end automatically. Every V2 feature — including automatic Knowledge Base growth — works without a human in the loop. There are no approval gates or interactive prompts. The Knowledge Base Editor is the only manual surface, and it is optional and off the critical path.
All of the following shipped:
-
Concept fidelity (done) — the compiler no longer silently loses parts of a concept:
-
Modifier preservation. Head-noun reduction (
open white shirt→white shirt) now retries each dropped modifier as a<modifier> <head-noun>compound through normal resolution, emitting it as its own tag in discovery order (open white shirt→white shirt,open shirt).SC0019(Concept Reduced) marks the reduction. KB-only and deterministic — goldens unchanged. -
Full list transcription. The Analyzer system prompt requires transcribing every list item; a deterministic post-parse check warns
SC0021(List Under-Transcription) when a tag-list input yields fewer concepts than it has items.
-
Modifier preservation. Head-noun reduction (
-
Semantic & embedding fallback (done, opt-in) — an opt-in nearest-neighbour fallback (
semantic.enabled, off by default). When enabled, a concept that misses both deterministic lookup and head-noun reduction falls back to its nearest entry in a committed embedding index (data/kb_embedding_index.json), accepted only abovesemantic.min_similarityand emittingSC0020(Semantic Fallback). The default backend is a deterministic, offline, dependency-free character-n-gram model. Deterministic lookup always wins; the fallback can only return an existing KB entry, never invents a tag; identical input yields identical output. Build the index withscripts/build_embedding_index.py. -
Automatic Knowledge Base Builder (done) — a single, fully-automated pipeline (
scripts/build_knowledge_base.py, zero human input) regenerates the generated KB deterministically: generate candidates → ingest Danbooru aliases/implications → auto-validate (structural rules + confidence, no approval gate) → conflict-scan → write.SC0022(Candidate Rejected) marks entries auto-validation drops. Seedocs/knowledge_base_build.md. -
Knowledge Base Editor (done) — an optional web editor served on the ComfyUI sub-route
http://127.0.0.1:8188/scene-compiler/kb. CRUD + avalidateendpoint reuse the authoritativekb_validationrules (single source of truth) against the candidate and the resulting whole-KB state, so invalid entries cannot be saved. Saves are atomic (temp + fsync + rename) and format-safe. The compiler never depends on the editor. -
Alias & validation tooling (done) —
scripts/detect_kb_conflicts.py(CI-gateable conflict scan) andscripts/coverage_benchmark.py(KB coverage report). -
Knowledge Base Versioning (done) — a Knowledge Base directory can ship a
manifest.json(newknowledge_base_manifestschema). Projects pin a version viaresolver.knowledge_base_version; an in-memory migration layer adapts older datasets to the current entry schema at load time without rewriting pinned files. - Performance (done) — load-once KB cache (reused across node runs), a compiled normalized lookup table (single-scene resolution ~57.6µs → ~17.8µs), optional incremental per-category lazy loading, optional threshold-gated parallel per-concept resolution that merges back in exact discovery order (deterministic, sequential by default), and a non-flaky performance regression guard. No change to deterministic output.
-
Configuration cleanup (done) —
prompt_builder.remove_duplicate_tagsis now wired to real behaviour (gates the Resolver'sSC0007deduplication; defaultstrue).prompt_builder.trim_empty_outputswas removed (inert since V1.1 dropped categories). - Analyzer localization (dropped) — multi-language system prompts were planned but dropped as unnecessary.
Entity Graph: Scene JSON may evolve internally into an Entity Graph, but Scene JSON remains the public interface for backward compatibility. Version 2 stays compatible with Version 1 Scene JSON; no workflow changes required.
Objective: transform Scene Compiler into a general-purpose prompt compiler.
- Multi-model support: Illustrious, Pony, Flux, NoobAI, future Danbooru and (where applicable) non-Danbooru models. Each model uses its own Resolver sharing the same Scene JSON.
- Plugin system: Resolvers, Analyzers, Prompt Builders, Validators, and Knowledge Providers become plugins.
- Multiple Analyzer backends: Ollama, Claude, GPT, Gemini, Mistral, custom NLP engines.
- Interactive Knowledge Base: visual editor, search, conflict resolution, validation, statistics, version management.
- Workflow Optimizer: automatic workflow generation, node recommendations, EasyIllustrious helpers, templates.
- Compiler SDK: the compiler as a reusable Python package usable independently of ComfyUI.
- Benchmark Suite: public dataset, reference prompts, performance/regression/Knowledge Base benchmarks.
Long-term vision: Scene Compiler becomes a compiler framework in which Illustrious is only one supported backend, while the compiler core stays unchanged.
Objective: tidy everything up and reduce friction for everyday use. The multi-node pipeline is ideal for inspection and for replacing individual stages, but most stage nodes carry little or no configuration, so a full workflow is more wiring than most users need.
- Unified compiler node — a single node that runs the whole pipeline (Analyzer → Validator → Resolver) and exposes the prompt directly, with configuration inlined or taken from a Configuration node. (V1.1 already collapsed the back half by removing the Category Splitter and Prompt Builder.)
- Individual nodes stay available — the granular nodes remain for debugging and for replacing a single stage; consolidation adds a convenience node, it does not remove the toolbox.
- General tidy-up — fold the mostly-config-less nodes together, simplify the default workflow, and align defaults across nodes.
The deterministic compiler core is unchanged; V4 is about packaging and ergonomics, not new compiler behaviour.
This project — its code, tests, and documentation — is built with heavy help from AI coding assistants and maintained by a solo author. AI makes mistakes: despite testing and review, parts of this may be inaccurate, out of date, or wrong. Treat it as a good-faith best effort, not authoritative truth — if the docs and the code disagree, the code wins. Please open an issue when something looks off.