Crucible

Cook Houdini HDAs at Karma render time — as a native Hydra scene index plugin.

Author a CrucibleProcedural prim in your USD stage. When Karma renders, Crucible cooks the referenced HDA on Houdini's main thread, injects the output mesh into the Hydra scene, and hides the input geometry — all at the scene index level with no pre-cooking, no LOPs recook, and no HAPI session to manage. Saving the HDA file re-cooks live without restarting Karma.

By Plattipus Research and Production Lab

At a glance

flowchart LR
    subgraph stage["USD Stage"]
        SRC["Mesh<br/>sourceGrid"]
        PROC["GenerativeProcedural<br/>primvars:Recipe = effect.hda<br/>primvars:InputPrim = sourceGrid"]
    end

    PROC -->|"cook at render time"| CRU{{"Crucible<br/>scene index"}}
    SRC -.->|"input geo (hidden in render)"| CRU
    HDA["effect.hda<br/>(SOP procedural)"] -->|"cooked on Houdini main thread"| CRU
    CRU -->|"injects mesh_0, mesh_1, …"| KARMA["Karma render"]

No pre-cooking, no LOPs recook, no HAPI session — the HDA is evaluated lazily as Karma pulls on the scene, and saving the .hda re-cooks live.

What it looks like

A CrucibleProcedural prim cooking in Solaris — editing the HDA and its params re-cooks the mesh live in the Karma viewport, with no LOPs recook or render restart.

assets/karma.exr is a reference render produced by the testKnit HDA in examples/ — a knit-weave SOP procedural cooking at Karma render time with no manually authored mesh.

To render the minimal example headlessly after installing:

husk --renderer Karma examples/procedural.usda
# output: examples/render_out.exr

For the knit lookdev scene, open examples/lookdev_knit.usda in a Houdini LOP network, set the two placeholder paths at the top of the file, and start Karma IPR.

Requirements

	Minimum
Houdini	21.0.559 — provides Karma, USD, Python 3.11, `libHoudiniHAPIL`
CMake	3.21
Xcode CLT	15 (macOS) — `xcode-select --install`
rez	Optional; recommended for managed pipelines

Install

Three commands turn the source tree into a loadable Houdini package. $HFS from Step 1 is what every later step depends on.

flowchart LR
    H["Source Houdini env<br/><code>$HFS</code> is set"] --> C["<b>cmake -S . -B build</b><br/>finds Houdini 21.0.559<br/>+ bundled Python 3.11"]
    C --> B["<b>cmake --build build</b><br/>compiles 5 .cpp →<br/>CrucibleHydra.dylib"]
    B --> I["<b>cmake --install build</b><br/>rpath-corrected copy<br/>into install prefix"]
    I --> P["Install prefix<br/>dylib · schema · python · 456.py"]

Step 1 — Source the Houdini environment

CMake needs HFS to locate Houdini's headers and CMake package.

The setup script derives $HFS from the current directory (HFS="$PWD"), so you must cd into the Houdini install directory before sourcing it — sourcing it by absolute path from elsewhere leaves $HFS empty.

# macOS — adjust the version to match your install
cd /Applications/Houdini/Houdini21.0.559/Frameworks/Houdini.framework/Versions/21.0/Resources
source houdini_setup_bash
cd /path/to/crucible

# Linux
cd /opt/hfs21.0.559
source houdini_setup_bash
cd /path/to/crucible

# Verify — must print the Houdini resources path; an empty line means it did NOT take
echo $HFS

Step 2 — Configure and build

cd /path/to/crucible

cmake -S . -B build \
    -DCMAKE_BUILD_TYPE=RelWithDebInfo \
    -DCMAKE_INSTALL_PREFIX=~/packages/crucible/0.1.0

cmake --build build --clean-first

--clean-first matters: stale .o files from a previous build can hide compile errors. Use it after pulling changes or switching branches.

Expected output:

-- Houdini 21.0.559 found
-- Houdini-bundled Python 3.11: .../libpython3.11.dylib
[100%] Built target CrucibleHydra

Step 3 — Install

cmake --install build

This copies the dylib, schema, Python packages, and startup scripts into the install prefix. Use cmake --install; do not cp manually — it sets the correct rpath.

Verify both C entry points are exported:

nm ~/packages/crucible/0.1.0/houdini/dso/usd/CrucibleHydra.dylib \
   | grep ' T _Crucible'

Expected:

T _CrucibleRegisterGenerativeProcedural
T _CrucibleRegisterSceneIndices

If either symbol is missing, rebuild with --clean-first and reinstall.

Activate in Houdini

With rez

rez env crucible
houdini   # or husk, hython, etc.

Rez sets HOUDINI_PATH, PXR_PLUGINPATH_NAME, PYTHONPATH, and DYLD_LIBRARY_PATH automatically from package.py.

Without rez

export CRUCIBLE_ROOT=~/packages/crucible/0.1.0

export PYTHONPATH="$CRUCIBLE_ROOT/scripts/python:$PYTHONPATH"
export PXR_PLUGINPATH_NAME="$CRUCIBLE_ROOT/schema:$PXR_PLUGINPATH_NAME"
export HOUDINI_PATH="$CRUCIBLE_ROOT/houdini:&"
export DYLD_LIBRARY_PATH="$CRUCIBLE_ROOT/houdini/dso/usd:$DYLD_LIBRARY_PATH"

houdini

Verify the plugin loaded

After Houdini starts, check the startup log written by 456.py:

cat /tmp/crucible_startup.txt

Expected output (one entry per session):

456.py: ctypes.CDLL OK  pid=12345
456.py: CrucibleRegisterSceneIndices() called OK
456.py: CrucibleRegisterGenerativeProcedural() called OK
456.py: Plug.Load: isLoaded=True

To run the full connectivity check from the Houdini Python Shell:

exec(open('/path/to/crucible/examples/check_plugin.py').read())

Usage

1. Define a CrucibleProcedural prim

def Mesh "sourceGrid" { ... }

def GenerativeProcedural "myProc" (
    prepend apiSchemas = ["HydraGenerativeProceduralAPI"]
) {
    token primvars:hdGp:proceduralType = "CrucibleProcedural"
    asset  primvars:Recipe             = @/absolute/path/to/myEffect.hda@
    string primvars:InputPrim          = "/root/sourceGrid"
}

All attributes live in the primvars: namespace so the Hydra scene index layer reads them via HdPrimvarsSchema without a custom schema translator.

See examples/procedural.usda for a complete runnable scene.

2. Override HDA parameters per prim

Add a child Scope named params under the procedural prim. Every primvars:* attribute on that Scope is passed to the HDA as a parm override before cooking. The attribute name (after stripping primvars:) must match the HDA parm token exactly:

def GenerativeProcedural "myProc" (...) {
    ...
    def Scope "params" {
        float primvars:radius          = 0.005
        int   primvars:targetquadcount = 50000
    }
}

Changing a primvar in the params Scope re-cooks the HDA and updates Karma interactively — no render restart required. Parameters absent from params cook at their HDA defaults.

3. HDA authoring contract

An HDA works with Crucible if it meets these requirements:

	Requirement
Context	SOP-context HDA with a single geometry output
Input	Accepts input geometry on port 0 (optional — falls back to a grid)
Parm names	Any parm token you want to override from USD must be a stable, user-facing name — it is used as-is as the primvar attribute name in the `params` Scope

4. Generate the params Scope automatically

Instead of authoring the params Scope by hand, use the CLI tool to introspect the HDA and write the USD layer for you:

hython tools/create_procedural_prim.py \
    --hda   /path/to/effect.hda \
    --out   examples/my_proc.usda \
    --prim  /World/MyProc \
    --input-prim /World/SourceMesh

This writes a complete .usda file with the procedural prim and a params Scope containing every HDA parameter tagged crucible = expose in the Houdini parameter editor. The tag controls which parameters are written into the USD layer by the tool — it has no effect at cook time.

Python API

The cook backend is a standalone Python module available independently of the Karma pipeline — useful for baking, batch processing, and Python Script LOPs.

`cook_hda`

from crucible_houdini.cook import cook_hda

result = cook_hda(
    hda_path        = '/path/to/effect.hda',
    overrides       = [('radius', 0.005), ('targetquadcount', 50000)],
    input_prim_path = '/root/sourceGrid',   # optional
)
# result: {'points': [...], 'face_counts': [...], 'face_indices': [...],
#          'normals': [...], 'normal_interp': 'vertex'} or None on failure

`write_mesh_to_stage`

from crucible_houdini.cook import write_mesh_to_stage

write_mesh_to_stage(result, stage, '/root/cooked')

Writes the cook result to stage as a UsdGeom.Mesh, targeting the stage's current edit layer. In a Python Script LOP, that is the editlayer Houdini injects.

Batch expansion — `lop_expand`

Bake every CrucibleProcedural prim in a stage to a real USD mesh (useful for render farm submission or USD interchange without the plugin):

from crucible_houdini.lop_expand import expand_all_procedurals

# Inside a Python Script LOP:
expanded = expand_all_procedurals(stage)
print(f'{expanded} procedural(s) baked')

Each procedural is replaced by a sibling mesh prim (<name>_expanded) and the original prim is hidden. Use expand_procedural(stage, '/path/to/proc') for single-prim control.

Debug LOP

Cook at author time (not just render time) using the built-in LOP HDA. Build it once from the project root:

hython tools/create_lop_hda.py

This writes houdini/otls/crucible_cook_lop.hda relative to the project root. Add it to a LOP network and set Recipe and Source Prim.

New Prim — writes the HDA output to a new USD path
Replace Input — overwrites the source prim in place

How it works

Scene index chain

Crucible inserts two scene indices between the USD stage and Karma. Neither modifies USD — they operate purely at the Hydra data model level. Data is pulled top-down by Karma; dirty notifications propagate bottom-up.

flowchart TD
    K["Karma<br/><i>pulls scene data</i>"]
    R["HdGpGenerativeProceduralResolvingSceneIndex<br/><i>instantiates + evaluates the procedural</i>"]
    O["CrucibleOverlaySceneIndex<br/><i>re-types prims · hides InputPrim · synthesises dirty</i>"]
    S[("inputScene<br/>USD stage")]

    K -->|"GetPrim / GetChildPrimPaths"| R
    R -->|"GetPrim"| O
    O -->|"GetPrim"| S
    S -. "PrimsDirtied" .-> O
    O -. "PrimsDirtied (+ synthesised)" .-> R
    R -. "PrimsDirtied" .-> K

First render — cook sequence

sequenceDiagram
    autonumber
    participant K as Karma<br/>(render thread)
    participant P as CrucibleGenerativeProcedural
    participant H as hapiCook.cpp
    participant M as Houdini<br/>main thread
    participant Py as crucible_houdini.cook

    K->>P: Update()
    P->>P: read Recipe / InputPrim<br/>+ params primvars
    P->>H: CookHDA()
    H->>M: dispatch_async(main_queue)
    Note over K,M: render thread blocks on a semaphore<br/>(CRUCIBLE_COOK_TIMEOUT, default 30s)
    M->>Py: cook_hda()
    Py->>Py: install HDA · wire input mesh ·<br/>set parm overrides · cook SOP
    Py-->>M: mesh dict
    M-->>H: signal semaphore
    H-->>P: CookedGeometry struct
    P-->>K: child mesh prims<br/>(mesh_0, mesh_1, …)

CrucibleOverlaySceneIndex sees a GenerativeProcedural prim with primvars:hdGp:proceduralType = "CrucibleProcedural" and does two things:
- Overlays hydraGenerativeProcedural as the Hydra prim type so the resolving scene index above it recognises the prim as something to evaluate
- Forces InputPrim to invisible so the source geometry does not appear in Karma's beauty render
HdGpGenerativeProceduralResolvingSceneIndex instantiates CrucibleGenerativeProcedural and calls Update().
Update() reads primvars:Recipe (HDA path), primvars:InputPrim (source mesh path), and all primvars:* on the child params Scope (parm overrides) from the Hydra data source.
CookHDA() in hapiCook.cpp calls crucible_houdini.cook.cook_hda() via the Python C API (PyImport_ImportModule + PyObject_CallFunction). The cook logic lives entirely in Python — no recompile required to change it.
Because HOM (hou.*) is not thread-safe, the call is dispatched to Houdini's main thread via dispatch_async(dispatch_get_main_queue(), ...) and the render thread waits on a semaphore. The timeout defaults to 30 seconds and is configurable via CRUCIBLE_COOK_TIMEOUT (seconds).
cook_hda() installs the HDA, wires the InputPrim mesh (read directly from the live USD stage), sets parm overrides, and calls sop_node.geometry() to trigger the SOP cook.
C++ deserialises the returned dict into a CookedGeometry struct and builds Hydra data sources for child mesh prims (myProc/mesh_0, myProc/mesh_1, …).
Karma renders the child mesh prims.

Live updates — two triggers, one re-cook

Both an edited params primvar and a saved .hda file converge on the same dirty path, so Karma updates in place without a render restart.

flowchart TD
    T1["params primvar edited<br/>(USD dirty on params Scope)"]
    T2["HDA file saved<br/>(mtime poll, 1×/sec)"]
    SYN["CrucibleOverlaySceneIndex<br/>synthesise dirty on procedural prim"]
    UPD["resolving scene index<br/>Update() → re-cook → diff ChildPrimTypeMap"]
    OUT["PrimsDirtied on mesh_0, mesh_1, …"]
    GPU["Karma updates vertex buffers in place<br/>no mesh teardown · no GPU realloc"]

    T1 --> SYN
    T2 --> SYN
    SYN --> UPD --> OUT --> GPU

Interactive parameter updates

When a params primvar changes:

The USD stage fires a dirty notification for the params Scope prim.
CrucibleOverlaySceneIndex._PrimsDirtied detects that the dirty prim is a direct child of a registered procedural and synthesises an additional dirty entry for the procedural prim itself.
The resolving scene index sees the procedural prim dirty, calls UpdateDependencies() and Update(), and diffs the returned ChildPrimTypeMap against the previous result.
Update() populates outputDirtied with all child mesh paths after each cook. The resolving scene index sends PrimsDirtied for those paths so Karma updates vertex buffers in place — no mesh teardown or GPU buffer recreation.

Live HDA reload

A background thread in CrucibleOverlaySceneIndex polls stat() on every watched HDA file once per second. When the mtime changes (i.e., the file was saved), it calls _SendPrimsDirtied for the procedural prim, which triggers a full re-cook through the same path as an interactive parameter update.

Package layout

crucible/
├── src/                         C++ Hydra scene index plugin
│   ├── crucibleSceneIndexPlugin Plugin registration — Karma CPU and Karma XPU
│   ├── crucibleOverlaySceneIndex Re-types prims; hides InputPrim; dirty synthesis
│   ├── crucibleProcedural       HdGpGenerativeProcedural — cook dispatch, mesh emit
│   └── hapiCook                 Python C API bridge to crucible_houdini.cook
├── schema/                      USD schema (CrucibleProcedural type registration)
├── scripts/python/
│   ├── crucible/                HDA introspection, HOU→USD type map, helpers
│   └── crucible_houdini/        cook.py — cook backend; lop_expand.py — bake utility
├── houdini/
│   ├── scripts/456.py           Auto-loaded startup; loads dylib via ctypes
│   ├── dso/usd/                 Dylib install target
│   └── otls/                    HDA install target (built locally via tools/)
├── resources/                   USD plugin discovery for usdview / non-Karma renderers
├── tools/
│   ├── create_lop_hda.py        Builds the debug LOP HDA (run with hython)
│   └── create_procedural_prim.py Generates a USD layer from HDA parameter introspection
└── examples/
    ├── procedural.usda           Minimal runnable scene (4×4 grid + CrucibleProcedural)
    ├── lookdev_knit.usda         Knit lookdev scene (sublayers Houdini material_lookdev)
    ├── test_scene.usda           Full scene with testKnit HDA and camera/lights
    ├── sop_PlattipusVFX.dev.testKnit.1.0.hdanc  Example SOP HDA
    └── check_plugin.py           Connectivity checker (run from Houdini Python Shell)

Registration path

TF_REGISTRY_FUNCTION does not fire for externally-loaded dylibs in Houdini 21. houdini/scripts/456.py is loaded automatically by Houdini via HOUDINI_PATH and explicitly calls both C entry points via ctypes:

CrucibleRegisterSceneIndices() — registers the overlay + resolving pair for Karma CPU and Karma XPU
CrucibleRegisterGenerativeProcedural() — registers the plugin factory so the resolving scene index can dispatch to CrucibleGenerativeProcedural::Update()

Troubleshooting

456.py: ctypes.CDLL FAILED — The dylib is missing or its rpath dependencies are unresolved. Run cmake --install build. If the dylib is present, inspect it:

otool -L ~/packages/crucible/0.1.0/houdini/dso/usd/CrucibleHydra.dylib

All @rpath/ entries must resolve inside the active Houdini installation.

CrucibleRegisterSceneIndices not found — The symbol was stripped. Rebuild with cmake --build build --clean-first then reinstall.

Plug.Load: isLoaded=False — A dylib dependency is missing. Confirm DYLD_LIBRARY_PATH contains the dso/usd/ directory and that Houdini's own frameworks are on the rpath.

No geometry in Karma — Confirm /tmp/crucible_startup.txt shows all four OK lines, then run examples/check_plugin.py from the Houdini Python Shell. If the check passes but Karma shows no output, confirm primvars:Recipe is an absolute path to an existing .hda / .hdanc file.

Cook timeout — For HDAs that take more than 30 seconds to cook, set CRUCIBLE_COOK_TIMEOUT=120 (seconds) before launching Houdini.

Parameter changes do not update Karma — Confirm the plugin version is ≥ 0.1.0. Earlier builds had an incomplete dirty-propagation path; the overlay now synthesises an explicit dirty for the procedural prim whenever its params child changes.

cook_hda import fails — crucible_houdini.cook is not on PYTHONPATH. Confirm CRUCIBLE_ROOT/scripts/python is in PYTHONPATH before launching Houdini.

Contributing

See CONTRIBUTING.md.

License

MIT — see LICENSE.

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
.github		.github
assets		assets
examples		examples
houdini		houdini
resources		resources
schema		schema
scripts/python		scripts/python
src		src
tools		tools
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
CMakeLists.txt		CMakeLists.txt
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
VERSION		VERSION
package.py		package.py

Folders and files

Latest commit

History

Repository files navigation

Crucible

At a glance

Contents

What it looks like

Requirements

Install

Step 1 — Source the Houdini environment

Step 2 — Configure and build

Step 3 — Install

Activate in Houdini

With rez

Without rez

Verify the plugin loaded

Usage

1. Define a CrucibleProcedural prim

2. Override HDA parameters per prim

3. HDA authoring contract

4. Generate the params Scope automatically

Python API

cook_hda

write_mesh_to_stage

Batch expansion — lop_expand

Debug LOP

How it works

Scene index chain

First render — cook sequence

Live updates — two triggers, one re-cook

Interactive parameter updates

Live HDA reload

Package layout

Registration path

Troubleshooting

Contributing

License

About

Resources

License

Contributing

Security policy

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

`cook_hda`

`write_mesh_to_stage`

Batch expansion — `lop_expand`

Packages