Asset Pipeline

Asset Pipeline

NWBLot separates authored source assets from runtime binary payloads. Authored files are parsed, validated, compiled, and packed by the staged asset pipeline (dependency_computer, asset_builder, asset_gatherer); runtime systems consume the cooked data through mounted volumes and typed asset codecs.

This page is the overview. Detailed asset contracts are split across:

• Asset Types
• Cooked Volumes
• Texture Conversion
• Shader and Material System
• Mesh Model and Skinning

Authored Source Formats

Authored sources include:

• .nwb: metadata and data declarations parsed by metascript.
• .tex: raw UASTC texture sidecars paired with texture .nwb metadata.
• .slang: Slang shader source.
• .slangi: shared Slang include source.
• .bind: typed material interface declarations.
• .surface: material surface hook fragments.
• .bxdf: material lighting BXDF fragments.

The cook step is where shader-language compilation, metadata validation, typed material layout generation, and runtime payload construction happen. Runtime graphics consumes cooked shader bytecode and cooked asset records.

Source files live under asset roots. Shared engine assets live under impl/assets; project-specific authored assets live under project trees such as CoolStuff/Testbed/assets and tests/smoke/assets.

Cook entry points supply asset roots explicitly. Core asset-path helpers do not infer impl/assets; a full graphics cook that needs both shared engine assets and project assets passes both roots.

The final graphics volume cook stages its work through a pack manifest. Direct byte payloads, such as shader archive records and generated graphics metadata, remain in-memory manifest entries. Registry-backed typed assets are serialized into content-addressed .nwbobj files under the cook cache first, and the manifest stores the cached object path plus a payload identity made from payload size, payload hash, and cook-key hash. The volume writer revalidates that identity before pushing the bytes into the staged volume.

Runtime Asset Loading

Runtime loading is typed:

• asset types derive from TypedAsset<T>
• asset codecs are registered by AssetRegistry
• AssetManager loads assets from an IAssetBinarySource
• the loader uses a mounted IFilesystem as its binary source
• cross-asset references use AssetRef<T>

This means runtime systems normally ask for typed assets by virtual path and type, not by parsing source files directly.

The most important invariant is direction of dependency:

authored source -> pipeline -> cooked volume -> typed runtime asset

Runtime systems should not compile shader source, parse material metadata, or inspect authored .nwb files directly.

At system level, that invariant passes through discovery, the three pipeline stages, staged volume publication, and typed codec loading:

flowchart TD
    Roots["Asset roots<br/>impl/assets<br/>CoolStuff/Testbed/assets<br/>tests/smoke/assets"]
    Discover["pipeline/launch.py discovers .nwb files"]
    Deps["dependency_computer<br/>optional UI skin dependencies"]
    Builder["asset_builder<br/>shader, material, texture, mesh, skeleton, model, CSG"]
    Validate["Parse and validate<br/>metadata, dependencies, interfaces"]
    CompilePack["Compile and pack<br/>SPIR-V, typed material bytes, meshlets, model data"]
    Artifacts["Standalone .nwba artifacts<br/>plus assets.list"]
    Gather["asset_gatherer publishes<br/>graphics volume"]
    LoaderMount["loader mounts res/graphics"]
    BinarySource["FilesystemAssetBinarySource"]
    AssetManager["AssetManager + AssetRegistry"]
    TypedAsset["Typed runtime asset"]

    Roots --> Discover --> Deps --> Builder --> Validate --> CompilePack --> Artifacts --> Gather
    Gather --> LoaderMount --> BinarySource --> AssetManager --> TypedAsset

Loading

Shader Assets

Shader assets are authored with .nwb metadata and Slang source. A shader asset records:

• a concrete shader declaration, such as mesh_shader asset;, pixel_shader asset;, compute_shader asset;, or a ray tracing shader type
• entry point
• include roots and define value sets
• optional numeric ray_query (0 or 1, default 0), plus optimization and mesh compute-shadow choices

The declaration determines the physical stage. Generic shader declarations and authored asset.stage fields are rejected, including redundant matching values. The cook step compiles Slang to SPIR-V bytecode and writes deterministic, internally versioned runtime payloads shared by all concrete shader types. Each payload preserves the authored entry-point spelling beside the bytecode; IShader retains that entry point and bytecode, while concrete types supply the physical stage as static constexpr s_Stage. Runtime shader assets never retain source strings, include roots, define combinations, or compiler state. Authors request RayQuery capability with asset.ray_query = 1;; the engine owns the compiler target/profile. Shader identity, paired source path, and archive role are derived, and the fixed engine shared-mesh object vertex stage is selected internally. Authored target_profile and mesh_object_vertex fields are rejected.

The cooker captures compiler stdout/stderr per variant. A nonzero compiler result is a cook failure with its captured diagnostics, and warnings are enabled as errors, so emitted diagnostics also prevent publication. It asks Slang to preserve the exact entry-point name in SPIR-V; runtime pipeline creation uses the recorded name rather than assuming a normalized main entry point.

Common metadata discovery parses the metascript document before passing it to ShaderCook; the shader front-end owns validation, variants, dependencies, and checksums. Shader volume writing calls SlangShaderCompiler::ComputeCompilerFingerprint and the single SlangShaderCompiler::CompileVariant implementation directly. The separate ShaderSourceDependencies module owns source-line splicing, comment masking, and include extraction/resolution for conservative dependency planning and BOM overlays. Compiler interfaces, factories, injection paths, forwarding methods, and production file-parsing convenience overloads are absent. All 14 physical stages remain supported by the compiler and archive mappings.

Each compiler invocation writes diagnostics and output in a unique temporary directory. Before publication, the cooker requires a readable diagnostic capture and validates the complete bounded SPIR-V instruction stream, exact requested entry point, and physical stage; a successful process exit alone is insufficient. Validated output is atomically promoted to a single <16-hex-source-checksum>.spv cache record, with no paired checksum file. Cache hits pass the same validation, and malformed or wrong-stage records are rebuilt. The source checksum includes the configured compiler executable content/version and shared compiler options, fingerprinted once per nonempty shader-volume append.

Dependency checksum reads also detect UTF-8 BOMs. Inputs without BOMs compile directly; BOM inputs use stripped temporary overlays that preserve include hierarchy and safely encode absolute-root bytes. Invocation diagnostics and overlays are cleaned on success and failure. Define names must be ASCII identifiers, and values reject semicolons, NUL/CR/LF, and surrounding whitespace that cannot round-trip the variant grammar.

Dependency discovery splices continued lines and ignores commented directives. Literal includes are tracked; unplanned macro includes, #include_next, and #import fail instead of silently leaving compiler inputs out of the checksum. The current generated CSG evaluator macro is explicitly declared by its preparation caller and its generated source remains separately tracked.

Shared CPU/GPU shader ABI constants live under impl/assets/graphics in macro-only headers usable by both C++ and Slang.

The neutral nwb_shader_bytecode target owns core/graphics/spirv_entry_point.h and .cpp and declares a public nwb_common dependency, alongside the standard internal target defaults. Shader asset runtime/cook targets use it for admission, and nwb_graphics_backend uses the same parser for module validation. Payload encoding, concrete types and codecs remain in impl/assets_shader; provider-specific headers remain behind graphics selection.

Concrete shader admission rejects an invalid payload header or version, empty entry point, non-word-aligned bytecode, missing SPIR-V magic, malformed instruction stream, or an entry point that does not match the concrete physical stage. It reads the complete bounded stream from a byte view without assuming aligned payload storage or copying into temporary words, then publishes the decoded common state only after validation. Failure preserves any previously loaded state. The selected graphics backend retains module validation before shader creation.

Archive roles do not change physical types: mesh_compute publishes ComputeShader assets, while mesh_object_vertex publishes a VertexShader. Codecs such as ComputeShaderAssetCodec share payload encoding and are selected by the concrete type name. Typed loaders and references derive their stage; heterogeneous calls use the explicit dynamic-stage loader. The type split adds no per-object stage storage, duplicated bytecode, or per-frame virtual stage dispatch.

The material cook can also synthesize shader entries. Materials that declare surface get a generated G-buffer pixel shader in the cook cache; surface-authored transparent materials also get generated AVBOIT occupancy, extinction, and accumulate pixel shaders. Generated sources are prepared and compiled like other shader entries.

Material Assets

Materials are typed and shader-driven. A material asset declares:

• interface
• surface
• bxdf
• shaders
• shader_variant
• parameters
• transparent
• two_sided
• refractive

Materials must explicitly author transparent, two_sided, and refractive, including zero values. The material interface comes from a .bind file. A .bind file declares typed blocks, fields, defaults, and block classes:

• material_constant: immutable material identity data.
• material_mutable: default values that can be overridden by a MaterialInstanceComponent.

The material-resource ABI accepts texture2d and sampler fields only in material_constant blocks. The .bind declaration contains a plain resource field with no attributes. The matching block/field entry in asset.parameters supplies an engine/ or project/ asset path. Textures and samplers are both typed assets. The cooker serializes block/field identity, resource kind/source, asset-name hash, and constant-byte offset; the renderer resolves a live global-heap slot later. Resource entries cannot be overridden through MaterialInstanceComponent.

bxdf is required during cooking. Each unique .bxdf source receives a deterministic shading-model id, and the cook emits the deferred lighting dispatch module that routes pixels to the material-authored BXDF. There is no engine default BXDF.

Each unique .surface source also receives a deterministic surfaceDispatchId. The cook emits shadow/generated/surface_dispatch.slangi, letting staged hardware transparent evaluation, software GPU-BVH traversal, and light-space shading obtain a hit material's optical parameters. Trace and resolve code integrate Fresnel transmission and Beer-Lambert absorption; the opaque RayQuery fast pass does not read that surface table.

A material declares either surface or explicit shaders, not both. The normal generated path uses surface: the cook wraps the .surface hook with the engine pixel-shader authoring header and the material .bind, then points the material mesh stage at the shared engine mesh shader. If the material is transparent, the cook also wraps the same .surface hook with the AVBOIT occupancy, extinction, and accumulate authoring headers. Explicit shaders remains the direct stage-to-shader map.

The transparent AVBOIT shader set is an atomic material contract: a transparent material must carry all three occupancy, extinction, and accumulate pixel-shader references, while an opaque material must carry none. The material builder, serializer, and runtime loader all enforce that rule, so malformed cooked data cannot activate a partial transparent pass.

refractive classifies a material as a refractive caster for shadow and caustic work. It does not store refraction values in metadata; those values are produced by the .surface hook.

Material-authored policy paths are project-rooted: interface, surface, bxdf, and explicit stage shaders use the project/ virtual root and are resolved against the supplied asset roots during graphics volume preparation. Generated engine shader pieces are assigned by the cook rather than authored through material metadata.

Typed material data is packed into deterministic block bytes. Static resource fields begin as zero slot words and are patched in a renderer-owned working copy after the referenced texture or sampler assets are resolved into the global descriptor heap. Runtime rendering binds material typed storage through generated typed accessors at binding 6.

Material metadata fields are defined by impl/assets_material/metadata.h.

Material runtime payloads contain:

• shader variant string
• material interface name
• shading-model id
• surface dispatch id
• typed layout hash
• typed layout block table
• typed layout field table
• typed block bytes
• static material resource-reference records
• per-stage shader references
• AVBOIT occupancy, extinction, and accumulate pixel shader references when and only when transparent is set
• transparent flag
• two-sided flag
• refractive flag

Texture Assets

A texture source is a texture asset; .nwb file paired with an implicit same-directory, same-basename raw linear UASTC .tex sidecar. Authors choose format (uastc_ldr_4x4 or uastc_hdr_4x4), dimension, resolution, and alpha_mode; the .tex filename is never authored. The impl/assets_texture cooker derives the complete contiguous mip chain from the resolution, checks the exact sidecar byte count without image conversion or re-encoding, and serializes a typed Texture payload. A separate HDR alpha stream must trail RGB and mirror its mip/slice layout exactly. Metadata rejects source color-space flags, filename/width/height/depth fields, fixed layout/address fields, mip records/counts, sampler fields, and version/revision fields; internal binary versions remain codec contracts.

The current source contract supports 2d, cube, and volume dimensions. Cube textures preserve six faces in +X, -X, +Y, -Y, +Z, -Z order; volume textures preserve ascending Z slices and reduce depth with each mip. See Texture Conversion for authoring commands and the complete metadata contract.

At runtime, TextureAssetLoader loads the typed asset through AssetManager, creates the corresponding device image, and registers a global sampled-image heap slot. Cached Vulkan capability probing selects LDR ASTC 4x4, BC7, or RGBA8 in that order; opaque HDR uses ASTC HDR 4x4, BC6H, or RGBA16_FLOAT, while HDR alpha uses RGBA16_FLOAT with the alpha value or mask merged into the image. Material texture2d fields consume only 2D texture assets today; cube and volume assets remain available through the typed loader for consumers that use their matching bindless descriptor classes.

Mesh Assets

Mesh assets contain geometry streams and meshlet data:

• positions
• normals
• tangents
• UVs
• colors
• meshlet descriptors
• meshlet bounds
• meshlet position reference deltas
• meshlet attribute reference deltas
• packed local vertex references
• packed primitive indices

The renderer is built around meshlet-style submission. Hardware mesh shader devices can use the mesh path directly. Without effective native support, eligible engine mesh programs use cached object-space indexed raster; other draws use an internal compute-emulation path derived from the mesh shader source.

Mesh cook code is split into meshlet build, bounds, candidates, scoring, stream reorder, reference encoding, and validation helpers. Runtime mesh payloads are validated again when loaded.

Skeleton and Skin Assets

Skeleton assets contain:

• joints
• parent indices
• bind poses
• child ranges
• child indices
• joint-name lookup maps

Skin assets bind a mesh to a skeleton and store:

• four-weight/four-index influences
• inverse bind matrices

impl/ecs_mesh/skinning owns runtime skinned mesh cache resources and GPU skinning work.

Model Assets

Model assets compose scene objects out of typed asset references. A model can contain:

• skeleton objects
• static mesh objects with material references
• skinned mesh objects with a Skin reference, optional material, and local skeleton-object selection

Skinned objects omit mesh; the field validator rejects it. Runtime source loading obtains the mesh from the loaded Skin and caches the resolved source assets. Static objects still declare their mesh. The internal MDL2 model payload omits the duplicated skinned mesh identity. The model ECS system expands model assets into the runtime entities/components required by rendering and skinning.

CSG Shape Cook Data

CSG shape metadata is cooked separately. Built-in runtime shapes include:

• plane
• box
• sphere
• capsule

Each registered shape type has a name, shader module include, parameter byte size, default parameter bytes, and a required CPU bounds callback (registration rejects a null callback). Its cooked/runtime GPU type id is derived deterministically from the canonical shape name rather than registration order; the cooker and registry reject any collision. At runtime the built-in shapes are registered under namespaced names engine/csg/plane, engine/csg/box, engine/csg/sphere, and engine/csg/capsule. CSG receiver and cutter ECS components connect those shapes to renderer work.

Built-in authored shape assets currently live under:

• impl/assets/csg/plane
• impl/assets/csg/box
• impl/assets/csg/sphere
• impl/assets/csg/capsule

CSG shape cooking can emit generated include files so renderer shaders can evaluate the same registered shape set consistently.

Asset Pipeline Stages

Cooking uses three independent tools coordinated by pipeline/launch.py:

1. dependency_computer returns its inputs unchanged by default, preserving order and duplicates. With --include-skin-dependencies, it resolves selected UI skins against the asset roots and adds their referenced texture provider files once; general transitive dependency expansion is not implemented. Its --output is a UTF-8 file containing one input per line. The pipeline launcher discovers selected .nwb files before invoking this stage and forwards the optional skin dependency flag.
2. asset_builder compiles selected .nwb assets into standalone .nwba artifacts and an assets.list manifest. It does not expand dependencies; related assets required by metadata validation must be included in the inputs. --asset-root values under impl use the engine/ virtual namespace; other roots use project/. Omitting --asset-type selects graphics; explicitly empty or unsupported values are rejected by the builder, including internal build requests.
3. asset_gatherer packages .nwba files into the existing graphics volume format through IFilesystem. Identical duplicate virtual identities and payloads merge. Library callers gathering independently built graphics assets set AssetGatherOptions::mergePayloads to NWB::Core::Assets::QueryAutoCollectedAssetGatherMerge() (declared in core/assets/gather_merge_registry.h); the graphics asset implementation self-registers its merge hook so the gather tool no longer names an impl/ symbol directly. The generic gather API also accepts a project callback for other metadata merge contracts; returning false rejects the conflict and preserves the published volume. Other payload conflicts fail. Gathering performs no source parsing or compilation.

Each .nwba artifact has an 88-byte little-endian header (0x4142574e magic, version 1, 64-byte virtual-path NameHash, payload size, FNV checksum) followed by the exact runtime codec payload. The gatherer validates that identity before packaging.

The builder uses CookEntryRegistry / CookEntryAutoRegistrar for typed asset entries and AssetVolumePrepareAutoRegistrar for graphics/shader prepare steps. Runtime uses a parallel registration concept for codecs: the loader builds an AssetRegistry and calls auto-collected codec registration before constructing the AssetManager.

NWB::Pipeline::AssetBuilder::BuildAssets is the C++ build entry point (pipeline/asset_builder/build.h). NWB::Pipeline::AssetGatherer::GatherAssets is the gather entry point (pipeline/asset_gatherer/gather.h). Shared artifact, cache, manifest, and volume-writing support remains in core/assets/volume.

Run the complete pipeline through the repository launcher:

python -m launcher pipeline --config dbg --asset-root impl/assets CoolStuff/Testbed/assets --output-directory runtime/res

See Build and Verification for flags, skip-build tool directories, and Linux cook convergence.

Cook-Side Modules

Several modules support asset cooking:

• impl/assets_shader — compiles Slang to SPIR-V and records shader archive entries.
• impl/assets_material — validates project-rooted material metadata, builds typed .bind layouts, assigns BXDF shading-model and surface dispatch ids, and emits generated material pixel shader, AVBOIT occupancy/extinction/accumulate pixel shaders, deferred BXDF, and surface dispatch sources. The implementation is split by bind parsing, typed layout building, material metadata parsing, generated shader source, and dispatch codegen so cook contracts are easier to audit independently.
• impl/assets_texture — validates and serializes UASTC texture metadata and sidecars; its runtime loader creates/caches global sampled-image resources.
• impl/assets_mesh — cooks mesh and skin payloads (there is no separate assets_skin).
• impl/assets_skeleton, impl/assets_model — cook skeleton and model payloads.
• impl/assets_csg — cooks csg_shape metadata and generated shape includes.
• core/assets/bunch — expands one authored .nwb bundle into several individual assets.
• impl/assets_graphics — shared graphics cook logic that aggregates shader, material, and CSG entries into the graphics volume metadata. This is the cook-logic module and is distinct from impl/assets/graphics, which holds the shared CPU/GPU ABI headers.
• core/assets/volume — shared cook support for standalone .nwba artifacts, the pack manifest, content-addressed object cache, staged volume writer, and final publication.
• pipeline/ — the three tool executables and launch.py orchestrator.

The concrete pipeline flags are listed in Build and Verification.

Library result contracts

Fallible path, metadata, dependency-selection and volume-write producers return values through global Expected<T, E>. DependencyProviderCatalog::selectInputs, read and resolve return the selection, parsed document and provider index after admission. The volume writer returns AssetVolumeWriteResult; built-asset reads return identity/payload-offset reports while keeping caller-owned byte storage. Registered metadata accumulation, reusable serialization buffers and project merge callbacks keep their actual mutation contracts. Check producer results before moving candidates or publishing manifests/volumes, and preserve owning arenas and detailed errors. See Diagnostics and Invariants.

Determinism Expectations

Cooked payloads are expected to be deterministic. Determinism work includes:

• stable virtual paths
• stable shader dependency checks through alias roots, including generated cook-cache sources
• deterministic typed material layout metadata
• deterministic typed block bytes
• deterministic material-generated shader, BXDF dispatch, and surface dispatch source
• deterministic meshlet and skeleton payload serialization
• staged volume publication

Determinism is verified by the Linux convergence procedure rather than by an automated in-tree byte-diff test: the asset cook tests only assert that a cook succeeds. The convergence procedure runs isolated two-root cooks over impl/assets and CoolStuff/Testbed/assets and tracks two separate outcomes:

• whether the cook succeeds
• whether independent cook outputs are byte-identical

A cook can succeed while still producing non-identical bytes, so keeping these outcomes separate makes failures easier to diagnose.

The registry object cache keeps unchanged typed asset payloads at a stable object path across repeated cooks and writes a new object file only when the serialized payload identity changes. This reduces repeated volume-pack work, but the runtime contract is unchanged: the application still loads typed assets from the published cooked volume, not from the cook cache.