Text to BIM, natively.
A plain-English building brief in, a real, standards-compliant IFC building model out: authored and coordinated across architecture, structure, and MEP, and verified end to end by open-source tools, with no proprietary authoring tool anywhere in the loop.
IFC is the open substrate, so no vendor tool is required. Verification is the reward.
Why · Quickstart · How it works · What the verifier guarantees · Features · Status · Docs
Note
Private, early-stage research project (v0.0.1). Milestones M0-M7 are built, tested, and passing, and the deterministic pipeline is proven live. The LLM-driven generative path runs on open weights but has not yet been demonstrated at frontier-model quality, and the real-scale training run is GPU-gated. The Status & honesty section is precise about what is proven versus scaffolded. Nothing here requires a commercial licence.
Turning a brief into a BIM model today means one of two bad options:
- Prompt an LLM for IFC directly - you get something plausible but not valid: it ignores the schema, clashes, and building code, and there is no way to trust it.
- Author it by hand in a proprietary tool (Revit, Vectorworks) and check it in another proprietary tool (Solibri) - correct, but slow, closed, and impossible to reproduce, deploy headless, fork, or automate without commercial licences.
Every prior working Text-to-BIM system takes the second road: Text2BIM, BIM Copilot, and BIMgent drive Vectorworks; revit-assistant, Generative-BIM, and Text2MBL drive Revit; all lean on Solibri to check. You cannot reproduce, deploy, or verify any of them without a licence.
HarnessBIM takes a third road. IFC4 is the ISO-standard data model every BIM tool already reads and writes, so we make IFC the native substrate and the vendor tool disappears. Three commitments follow:
- Backend-agnostic, IFC-canonical. Agents never talk to a specific tool. They author
canonical IFC4 through a
BimBackendinterface; concrete engines (IfcOpenShell headless, Bonsai/Blender GUI, and future Elements/FreeCAD) are pluggable adapters held to a semantic-equivalence contract so the abstraction cannot quietly leak. - Verification is the reward. Schema validity, IDS conformance, zero clashes, code compliance, and structural sanity are machine-checkable. A fully open-source checker suite (replacing Solibri) drives an evaluator-optimizer refinement loop, and is reserved as the reinforcement-learning reward. This is the design's most novel piece: the "pytest of BIM." Independent 2026 results now prove this design in print - Mila's floor-plan RLVR (arXiv:2605.14117) trains with verifiable connectivity and area rewards and zero reward on invalid geometry, which is exactly our reward shape.
- No proprietary tool, open weights first-class. The whole pipeline runs on open standards and open-source engines. Local open-weight models (Qwen/Llama via Ollama/vLLM) are a first-class path, not a fallback.
One natural-language brief flows through an orchestrator to discipline specialist agents, each of which writes Python against the backend-agnostic API in a sandbox; the result is canonical IFC4, which a coordinator drives through the 9-checker suite until it converges.
flowchart TD
A[Natural-language brief] --> B[Orchestrator + Brief agent<br/>RAG-grounded structured program]
B --> C[Plan: architectural -> structural / MEP -> coordinate -> refine]
C --> D[Discipline agents<br/>LLM writes Python vs BimBackend -> sandbox exec -> IR]
D --> E[Backends: IfcOpenShell headless / Bonsai GUI<br/>canonical IFC4]
E --> F[Coordinator: 9-checker OSS suite<br/>schema - IDS - clash - code - structural - mep - egress - proofs - relations]
F -->|BCF issues| D
F -->|converged| G[Valid IFC4 + validation report + BCF]
The full layered stack (every layer is swappable)
┌──────────────────────────────────────────────────────────────────────┐
│ INTERFACES CLI · Python SDK · MCP server · A2A · web viewer │
├──────────────────────────────────────────────────────────────────────┤
│ HARNESS (the "OS") LangGraph graph · checkpointed state · HITL │
│ gates · sandboxed code interpreter · tracing │
├──────────────────────────────────────────────────────────────────────┤
│ AGENTS Orchestrator -> Brief / Architectural / Structural │
│ / MEP / Coordinator (clash + checks) │
├───────────────┬───────────────┬───────────────┬──────────────────────┤
│ KNOWLEDGE │ MEMORY │ TOOLS/MCP │ VERIFICATION │
│ hybrid RAG + │ tiered + │ BIM ops · │ schema · IDS · │
│ Graph-RAG │ decisions + │ checkers · │ clash · code · │
│ (codes+IFC) │ skill lib │ retrieval │ structural … -> BCF │
├──────────────────────────────────────────────────────────────────────┤
│ LLM PROVIDER LAYER LiteLLM: Anthropic · OpenAI · Google · Ollama │
│ · vLLM · capability routing · structured out │
├──────────────────────────────────────────────────────────────────────┤
│ BIM BACKEND ABSTRACTION BimBackend interface -> canonical IR │
│ IfcOpenShell (headless) · Bonsai (Blender) │
│ canonical interchange: IFC4 │
└──────────────────────────────────────────────────────────────────────┘
Invariant: agents know only the BimBackend interface and the IR; the harness knows only
agents and tools; nothing above the LLM layer names a model vendor; nothing above the backend
layer names an engine. See docs/01-architecture.md.
Runs headless with no API key on the deterministic path. Requires Python 3.11+.
git clone https://github.com/ReverseZoom2151/harnessbim && cd harnessbim
python -m pip install -e ".[ifc,verify]" # headless authoring + the OSS checker suiteGenerate a coordinated multi-discipline model and watch every checker run:
harnessbim project "a 3-storey steel-frame office, ~1200 m2, open-plan, central core, VAV HVAC" --out office.ifcwrote office.ifc (disciplines: architectural, structural, mep)
PASS schema (0 issues)
PASS ids (1 issues)
PASS clash (0 issues)
PASS code (1 issues)
PASS structural (2 issues)
PASS mep (0 issues)
PASS egress (0 issues)
PASS proofs (0 issues)
PASS relations (578 issues)
That is a real, valid IFC4 file (~1,400 entities) you can open in any IFC viewer - produced with no Revit, no Vectorworks, no Solibri, and no API key. More commands:
harnessbim generate "a single-storey 3-bedroom house" --out house.ifc # single-discipline, 8 checkers
harnessbim verify office.ifc # run the OSS suite standalone
harnessbim view office.ifc --out office.html # self-contained 3D web viewer
harnessbim query office.ifc "how many spaces are there?" # structured question over the model
harnessbim sketch plan.png --out plan.ifc --storeys 2 # floorplan image -> IFC (deterministic CV)Add --llm to project/generate to use LLM-driven discipline agents (see
LLM setup); --backend bonsai targets the Blender adapter.
Install extras (pick what you need)
| Extra | Pulls in | For |
|---|---|---|
ifc |
ifcopenshell | headless IFC authoring (required) |
verify |
ifctester, ifcclash, PyNiteFEA | the OSS checker suite |
llm |
litellm | pluggable LLM providers (API or local) |
rag |
chromadb, rank-bm25, sentence-transformers | knowledge / RAG |
harness |
langgraph, mcp | orchestration graph + MCP server |
serve |
fastapi, uvicorn | A2A server + web viewer host |
train |
torch, transformers, peft, trl, datasets | the LoRA/RLVR training path |
scan-las |
laspy | LAS/LAZ scan ingestion |
scan-e57 |
pye57 | E57 scan ingestion |
scan-ml |
torch | local TorchScript scan perception (weights supplied separately) |
scan |
scan-las+scan-e57+scan-ml | all local scan adapter dependencies |
all |
ifc+llm+rag+harness+verify+serve | the complete Text-to-BIM runtime |
research |
all+scan+train+dev | local research workstation profile |
python -m pip install -e ".[all]" # complete Text-to-BIM runtime
python -m pip install -e ".[research]" # local scan/training/developer workstationall intentionally excludes Blender/Bonsai, Revit, official dataset payloads, point-cloud
captures, and learned model weights: they are external runtime boundaries, not package
dependencies. research installs the local Python adapters, but still requires those assets.
Importing the package pulls in torch/tensorflow transitively, so runs emit some harmless TensorFlow/oneDNN warnings on stderr; filter them for clean output.
LLM setup (optional, for the generative path)
HarnessBIM is model-agnostic via LiteLLM. Point it at any provider:
# a hosted model
export ANTHROPIC_API_KEY=... # or OPENAI_API_KEY / GEMINI_API_KEY
harnessbim project "..." --llm
# or a fully-local open-weight model, no key
ollama pull qwen2.5-coder:7b
export HARNESSBIM_MODEL=ollama_chat/qwen2.5-coder:7b
harnessbim project "..." --llmSmall local models (1.5B/3B) run the loop end to end but often fall back to the deterministic path; a 7B+/frontier model is recommended for genuine LLM authoring (see Status & honesty).
Every model HarnessBIM accepts has passed a fully open-source suite of 9 checkers - this is the trust boundary and the reward signal. The LLM proposes; the verifier disposes.
| # | Checker | Guarantees | Built on |
|---|---|---|---|
| 1 | schema | well-formed IFC4 (hard gate) | ifcopenshell.validate |
| 2 | ids | conforms to the brief's requirements | buildingSMART IDS / IfcTester |
| 3 | clash | no hard geometric interferences | ifcclash / BVH |
| 4 | code | code-compliance rules, clause-cited | rule engine (IBC/NFPA-seeded) |
| 5 | structural | member/deflection sanity | PyNite FEM (+ geometric fallback) |
| 6 | mep | system connectivity, slope, velocity | routing checks |
| 7 | egress | travel distance + exit capacity | walkable-graph shortest path |
| 8 | proofs | numeric geometry proofs (envelope, area, bands) | IR-level proof bank |
| 9 | relations | declared vs geometry-inferred relationships | relationship inference |
Findings are emitted as tool-neutral BCF; a failed schema check zeros the reward; the
refinement loop routes issues back to the owning discipline and re-verifies until convergence
(sanity, not certification - it flags likely issues for human review, it does not claim
legal sign-off). These 9 are the default suite; opt-in simulation surrogates (energy,
egress) can be registered as extra checkers that double as RLVR reward terms, but they are
not auto-wired into the default 9. Details in docs/08-verification-and-eval.md.
| One-shot LLM -> IFC | Proprietary authoring (Revit/VW + Solibri) | HarnessBIM | |
|---|---|---|---|
| Standards-valid output | ✗ (plausible, not valid) | ✓ | ✓ (9-checker gated) |
| Reproducible / headless | ✓ | ✗ | ✓ |
| No commercial licence | ✓ | ✗ | ✓ |
| Auditable verification | ✗ | partial (closed) | ✓ (open, cited, BCF) |
| Runs on open weights | ✓ | ✗ | ✓ |
| Multi-discipline coordination | ✗ | ✓ | ✓ (arch + struct + MEP) |
| Forkable / extensible core | ✗ | ✗ | ✓ (Apache-2.0) |
- Backend-agnostic IFC authoring -
BimBackendABC +IfcOpenShellBackend(headless, default) +BonsaiBackend(Blender subprocess), held to a semantic-equivalence contract; IFC4 canonical interchange.docs/02 - Unified 9-checker OSS verification loop - schema, IDS, clash, code, structural,
MEP, egress, numeric proofs, relations; BCF emission; the same suite is the in-loop verifier,
the offline benchmark, and the RL reward.
docs/08 - Multi-agent orchestrator - Orchestrator + Brief + Architectural + Structural + MEP
- Coordinator;
harnessbim project= one brief to a coordinated model.docs/03
- Coordinator;
- Pluggable LLM layer (LiteLLM) - Anthropic/OpenAI/Gemini + local open weights via
Ollama/vLLM; capability-aware router (api/local profiles + budget); constrained decoding +
prompt cache.
docs/04 - Knowledge / RAG - hybrid retrieval (BM25 + dense + RRF + rerank, zero-dep fallback)
- Graph-RAG over the IR + a starter cited codes corpus. Powers
query.docs/05
- Graph-RAG over the IR + a starter cited codes corpus. Powers
- Memory & skills - tiered project memory, decision log + contradiction detection,
Voyager-style skill library, reflection.
docs/06 - Tools, MCP & A2A - BIM ops as callable tools; an MCP server (7 tools + resources +
prompts) so any MCP host can drive it; an A2A server for discipline plug-ins.
docs/07 - Vision & scan input - deterministic CV floorplan vectorizer (
sketch) with a VLM correction hook; an evidence-first point-cloud Scan-to-BIM workflow with review, BCF, and approval-gated materialisation. - Viewers - a self-contained web-ifc + three.js 3D viewer; a SQLite range-streamed sidecar and an IFC to 3D-Tiles exporter for large models.
- Quantities, cost & carbon - deterministic QTO/BOM, 5D cost, embodied carbon, space-program conformance.
- Training path - trajectory-to-dataset pipeline, LoRA/QLoRA (peft+trl), and RLVR/GRPO using the verifier as the reward.
The 9-checker core is the product; everything here is built on top of it and routes its output
back through the same verifier. A market/theory-of-change analysis of the built environment was
turned into a scoped build program (docs/13) of 20 items
across six epics (A-F), and all 20 now have a real, tested, deterministic-first v1 shipped as a
Python API layer (16 new packages), a standards-native HTTP gateway, and new CLI verbs (report,
setout, carbon, energy, typology, roi, serve, store). SOTA research (2024-2026)
validating each epic is in docs/research-sota/.
- Interop & platform - a persistent, versioned project store; a standards-native gateway
(
gateway.create_app: FastAPI REST + a BCF-3.0-style issues endpoint + IDS on/verify, behind theserveextra); a public plugin registry (entry-point groupharnessbim.plugins) so third parties can register backends, checkers, rules, and connectors that show up indefault_checkers(). - Pre-construction - permit / design-review report, constructability + RFI predictor, 4D schedule + procurement planner, as-built diff, robotics/setout export.
- Performance, cost & carbon - EN 15978 (A1-A3) embodied carbon in
quantities/, a material-substitution carbon optimizer (the verifier as the constraint oracle), a steady-state envelope-energy estimate, and a performance-metrics pack. - Design value & ROI - a pluggable-provider design-ROI score, edit-time trade-off deltas, parameterized typology generators, and an authorable DesignSpec (rule-pack + IDS + skill defaults) a firm can encode and provably check its models against.
- Knowledge & copilot - an IFC-corpus architecture repository and a session-scoped,
transparent design copilot (
POST /copilot/turn): parameterized strategies are realized deterministically and returned with the verifier evidence. A browser client remains optional. - Twin & simulation (v1 scaffolds / opt-in) - a digital-twin binding scaffold (simulated telemetry + drift detection; real telemetry is external), opt-in energy/egress simulation surrogates that register as extra checkers and RLVR reward terms (not auto-wired into the default 9), and multi-view spatial input.
Honesty holds here too: the twin and external-engine integrations are real-but-scoped v1
boundaries in the same spirit as scan/; external DB/engine enrichment
(EC3/openEPD, Speckle, SpatialLM) stays behind the deterministic no-key default; and every
generative or optimizing module must pass its output through the same 9-checker suite. Full spec
and per-epic status: docs/13.
Claims here are backed by verifiable runs; where something is scaffolded or externally gated, it says so plainly.
Proven (live / tested):
- Deterministic multi-discipline
projectrun produces a valid IFC4 model (~1,400 entities) with all 9 checkers passing, headless, no API key. - Two-backend equivalence: IfcOpenShell ≡ Bonsai on Blender (semantic-equivalence gate).
- Refinement loop drives a real model from 3 blocking issues to 0 (keep-best + iteration cap).
- The generative loop runs live end to end on local Ollama models (1.5B/3B) with graceful fallback; LiteLLM to Ollama proven; MCP server, A2A server, and web viewer built and tested.
- LoRA and GRPO/RLVR smokes completed end to end on Qwen2.5-Coder-0.5B (
verifier_reward= 0.92 for a valid model, 0.0 for a broken one). - The 20 buildout modules (docs/13) all ship a tested, deterministic-first v1; the copilot is now exposed as a transparent HTTP session while twin/engine enrichments remain optional.
- Current focused release evidence, including optional-runtime skip accounting, is maintained in
docs/19-capability-ledger.mdrather than a stale fixed count.
Scaffolded / externally gated (not yet demonstrated):
- Frontier-quality generative authoring. Small local models degrade to the deterministic path; a >=90% generative schema-valid matrix against a 7B+/frontier model has not been run. This is the single biggest caveat.
- Full-scale training. Only smoke-scale LoRA/GRPO have run; the real run is GPU-gated behind
HARNESSBIM_RUN_TRAINING. "Fine-tuned beats base" is unproven. - .NET Elements / FreeCAD backends. Python adapters over a documented protocol exist and degrade gracefully; the external service binaries are future work.
- Hosted benchmark dashboard. Metrics run locally; nothing is published.
Milestones M0-M7 are built (see docs/09-roadmap-milestones.md),
and the next-wave buildout (all 20 items, v1) is built on top of them - scoped, with honest scaffold
caveats, in docs/13-built-environment-buildout.md.
HarnessBIM is a synthesis, not an invention from scratch. It is grounded in a research corpus
under resources/ - ~76 papers, 17 reference repositories, and 2 books - mined in
docs/mining/ with a line-by-line traceability matrix in
docs/10-resource-traceability.md. It takes the proven
paradigm from the TU-Munich line (Text2BIM / BIM Copilot: multi-agent, text-to-code against a
BIM API, rule-checker feedback), frees it from proprietary tooling, generalizes it to
multiple disciplines, and hardens it with patterns from Agentic Design Patterns (Gulli)
and The Hitchhiker's Guide to Agentic AI (Roitman).
Clean-room discipline: GPL reference repos (e.g. BIMgent) are read for ideas only, never copied. Copyleft engines are kept at arm's length - IfcOpenShell (LGPL) is used as a library; Bonsai/Blender (GPL) is driven as a subprocess, never linked or vendored.
Start at docs/SPEC.md. Key documents:
| 00 Vision & Principles | problem, thesis, non-goals |
| 01 Architecture | layered stack, request lifecycle |
| 02 Backend Abstraction | BimBackend, IR ↔ IFC4, equivalence |
| 03 Agent Topology | orchestrator + discipline agents |
| 08 Verification & Eval | the 9-checker suite, reward |
| 09 Roadmap | M0-M7 status |
| 13 Buildout Spec | scoped next-wave opportunities |
| 11 Risks & Licensing | copyleft-containment strategy |
harnessbim/
├── src/harnessbim/
│ ├── ir/ backends/ agents/ harness/ core: IR, adapters, agents, runtime
│ ├── verify/ rules/ quantities/ the OSS checking + QTO layer
│ ├── llm/ knowledge/ memory/ models, RAG, memory/skills
│ ├── tools/ eval/ training/ MCP/A2A, benchmarks, LoRA/RLVR
│ ├── vision/ scan/ viewer/ layout/ mep/ input modalities, viewers, discipline logic
│ ├── project/ gateway/ plugins/ docs/13 buildout: store, HTTP gateway, plugins
│ ├── reports/ analysis/ planning/ export/ permit/perf, constructability, 4D, setout
│ └── optimize/ energy/ roi/ typologies/ carbon opt, energy, ROI, typologies,
│ authoring/ repository/ copilot/ twin/ sim/ DesignSpec, knowledge, copilot, twin, sim
├── docs/ the full specification + mining + audits + research-sota
├── tests/ 81 test files
└── resources/ the research corpus (papers, repos, books) that grounds the design
Core code is Apache-2.0. Copyleft engines are kept at arm's length: IfcOpenShell (LGPL-3.0)
is used as a library; Bonsai/Blender (GPL-3.0) is driven as a subprocess, never linked or
vendored; GPL reference repos in resources/ are read-only references - do not copy code
from them. See docs/11-risks-and-licensing.md.
Text to BIM, natively. IFC is the substrate; verification is the reward.