Development Testing and Contributing

Development, Testing & Contributing

Spec reference: MASTER_SPEC §24–28.

Repository structure

The repository is organized around compiler stages; every directory has a clear responsibility.

ComfyUI-SceneCompiler/
├── __init__.py
├── nodes/            # ComfyUI nodes only — no compiler logic
├── compiler/         # pure compiler logic — never imports ComfyUI
│   ├── analyzer/
│   ├── validator/
│   ├── resolver/     # resolution + flat prompt translation
│   └── common/       # config, logging, result, knowledge base, embeddings
├── knowledge_base/   # Knowledge Base JSON files only — no code
├── schemas/          # JSON Schemas and data models
├── prompts/          # official LLM system prompts and templates
├── config/           # configuration files
├── tests/
│   ├── analyzer/
│   ├── resolver/
│   ├── validator/
│   ├── integration/
│   └── regression/
├── examples/         # workflows, example Scene JSON, example prompts
├── scripts/          # dev utilities, KB generators, validation, migration
├── docs/
├── LICENSE
├── README.md
└── pyproject.toml

Key rule: compiler/ must never import ComfyUI, and nodes/ must contain no compiler logic.

Coding guidelines

This project is developed primarily with AI-assisted programming; readability and maintainability are valued over cleverness and premature optimization.

• Python 3.11+; explicit type hints on every public function; typed data models over anonymous dictionaries.
• Small, single-responsibility functions with explicit names; minimal hidden state; no magic values.
• Separation of concerns — logic in compiler/; nodes are interfaces; avoid mutable global state.
• No hardcoded tag mappings — every mapping belongs in the Knowledge Base.
• Comments explain why, not what. Every public class/function has a docstring.
• Structured logging, not print(); every error message explains what failed, why, and a possible fix.
• Tests accompany new functionality; every bug fix adds a regression test.
• Minimal dependencies — prefer the standard library.
• Refactors preserve behaviour — behavioural compatibility outweighs implementation convenience. Every PR includes a description, reason, tests, and (if needed) a docs update.

Testing strategy

Testing is a core feature: every stage is independently verifiable, and every bug becomes a permanent regression test.

Test type	Purpose
Unit	Each stage in isolation (Analyzer, Validator, Resolver).
Integration	The complete pipeline; output must be deterministic.
Regression	Every fixed bug gets a permanent test; a resolved bug must never reappear unnoticed.
Knowledge Base	Duplicate IDs/aliases, missing categories, unknown/missing tags, circular aliases/expansions, schema validity.
Schema validation	Every JSON file validates against its schema.
Performance	Compilation time, Resolver speed, KB loading, memory.
Golden	Generated prompts compared against reference outputs (identical input → identical output).

Knowledge Base tooling (V2)

The generated Knowledge Base is rebuilt by a fully-automated, zero-input pipeline — scripts/build_knowledge_base.py — with supporting tools scripts/detect_kb_conflicts.py (CI-gateable conflict scan) and scripts/coverage_benchmark.py (coverage report). The opt-in semantic fallback's index is built with scripts/build_embedding_index.py. See Knowledge Base → Automatic builder and docs/knowledge_base_build.md.

Reference test dataset

Version 1 ships an official benchmark set that becomes part of the long-term validation suite: single character, two characters, character interaction, indoor scene, outdoor scene, fantasy scene, modern scene, unknown concepts, ambiguous concepts, and a complex multi-character scene.

Guidelines for AI engineering agents

Compiler stages must not depend directly on each other — only public interfaces are shared, and dependencies point downward only:

Allowed:   Scene Analyzer → Scene Validator → Resolver
Forbidden: Resolver → Scene Validator

Stages stay stateless where possible; configuration is injected, not globally imported; the Knowledge Base is loaded once and reloaded only explicitly. Exceptions (unexpected failures) and warnings (expected, recoverable failures) stay independent.

Mandatory rules

• When ambiguous, prefer deterministic behaviour.
• When architecture conflicts with convenience, prefer architecture.
• When optimization conflicts with readability, prefer readability.
• When intelligence conflicts with explainability, prefer explainability.
• When creativity conflicts with correctness, prefer correctness.
• When uncertain, generate warnings instead of assumptions.
• Never hardcode knowledge. Never bypass compiler stages. Never merge responsibilities.
• Always write tests before a feature is complete; every bug fix adds a regression test.
• Every tag must be explainable; every output must be reproducible.

The compiler is not a prompt generator; it is a deterministic compiler. Protect this principle above all others.

Versioning strategy

Scene Compiler follows Semantic Versioning MAJOR.MINOR.PATCH:

• MAJOR — breaking changes (schema changes, pipeline changes).
• MINOR — new features (new nodes, new Resolvers, new Knowledge Base capabilities).
• PATCH — bug fixes, performance, documentation, regression fixes.

Version 1 provides a stable API, stable Scene JSON, and a stable Knowledge Base format. Deprecated features should remain available for at least one major version, and warnings must clearly indicate deprecated behaviour.

Related pages

• Knowledge Base
• Roadmap
• Requirements & Success Criteria

---

A note on AI assistance

This project — its code, tests, and documentation — is built with heavy help from AI coding assistants and maintained by a solo author. AI makes mistakes: despite testing and review, parts of this may be inaccurate, out of date, or wrong. Treat it as a good-faith best effort, not authoritative truth — if the docs and the code disagree, the code wins. Please open an issue when something looks off.