Mesh Model and Skinning

Mesh Model and Skinning

NWBLot scene geometry is built around typed mesh, skin, skeleton, and model assets. Runtime systems expand model assets into ECS components, upload mesh resources, and run GPU skinning before rendering. Platform-specific mesh-path policy is documented in Platform and GPU Support.

Mesh Payloads

Mesh assets contain a MeshGeometryPayload.

Attribute streams:

• positions as Float3U
• normals as Half4U
• tangents as Half4U
• UV0 as Float2U
• colors as Half4U

Meshlet streams:

• meshlet descriptors
• meshlet bounds
• position reference deltas
• attribute reference deltas
• local vertex references
• primitive indices

The renderer uses these payloads for meshlet-style rendering. The same authored content can feed native mesh shaders, persistent indexed raster, or compute-emulated mesh processing. Windows ARM64 disables native mesh shaders by default and selects the applicable portable route; a qualified adapter must opt into native mesh shaders before graphics instance creation.

The position-reference delta, attribute-reference delta, and primitive-index streams are logical raw-byte streams. Their GPU backing buffers are zero-padded through the final 32-bit word because ByteAddressBuffer::Load reads an aligned word even for a terminal byte. The logical byte/triangle-corner count remains separate from the padded allocation size; renderer code must not infer meshlet primitive count from the physical buffer length.

Raster Transport

Native mesh shaders and compute emulation share one generated-vertex contract. The compute-emulation transport is a 64-byte vertex record: float4 position at byte 0, half4 normal at 16, half4 tangent at 24, float2 UV0 at 32, half4 color at 40, and float4 world position at 48. Its input layout reads normal, tangent, and color as RGBA16_FLOAT.

The shared compute-emulation output uses indexed output (emulation_output.slangi) shared across material passes. The compute shader stores each half4 as two packed uint words rather than relying on native-half fields in a storage buffer. That retains the 64-byte FP16 transport while avoiding a requirement for UniformAndStorageBuffer16BitAccess; the mesh-shader path keeps the same values as native halves in registers and raster varyings. The shared byte offsets and stride are defined by NWB_MESH_EMULATION_VERTEX_* macros in impl/assets/graphics/mesh/runtime_constants.h and checked against the C++ record layout.

Mesh Cook Steps

Mesh cooking is split across focused helpers:

• source metadata parsing
• meshlet candidate construction
• meshlet scoring
• meshlet frontier expansion
• meshlet bound computation
• stream reorder
• meshlet reference encoding
• payload packing
• payload validation

Runtime validation runs again when loading mesh payloads, so invalid cooked data should fail before renderer resource creation.

Source Mesh Refresh

nwb_fbx_to_nwb imports FBX scenes into this authored mesh/model data and can refresh existing mesh .nwb sources. Passing a .nwb input, or passing --refresh-nwb, reads the authored mesh metadata, canonicalizes mesh streams, rewrites vertex references/indices, preserves the source line-ending style, and reports before/after counts for positions, normals, tangents, UVs, colors, skin weights, vertex references, and indices. FBX to NWB Conversion documents mesh selection, coordinate conversion, output layouts, and the complete command-line contract.

Typical refresh usage:

python -m launcher fbx-to-nwb --working-directory . -- path/to/mesh.nwb --refresh-nwb -o path/to/mesh.nwb --force -y

Refresh mode is meant to keep large authored mesh sources deterministic after importer or canonicalization changes. It is not an FBX import mode; options such as --list-meshes are rejected for .nwb refresh.

Importer and mesh-cook hot paths keep SIMD loads at explicit helper boundaries. FBX triangle area/normal math uses AVX2 only when the build target provides it; scalar and narrower SIMD implementations preserve the same result contract on other targets, including ARM64. Meshlet scoring similarly loads precomputed triangle positions, centroids, and area normals through shared helpers before the frontier and scoring code consume SIMD vectors.

Skeleton Assets

Skeleton assets contain:

• joints
• parent indices
• local bind poses
• child ranges
• child indices
• joint name lookup map

The skeleton asset rebuilds hierarchy data after load. It can report root joint count and find joint indices by name.

Skin Assets

Skin assets bind a mesh to a skeleton.

Payload:

• mesh reference
• skeleton reference
• four-influence weights/indices
• inverse bind matrices

Skin assets do not represent an animated pose by themselves. They describe how mesh vertices are affected by skeleton joints.

The runtime GPU influence record is 24 bytes with 8-byte alignment: two packed u32 joint words at byte 0 and four FP32 weights beginning at byte 8. The Slang mirror uses uint2 packedJoint, float2 weight01, and float2 weight23 at offsets 0, 8, and 16. Each joint retains its original 16 bits, including index 65535 when the skeleton has 65536 joints; weight packing copies the original FP32 bits, including signed zero. Upload byte sizes and structure strides use the payload type's size. This transport preserves authored skin data, all four FP32 weights, and the skinning equations. Configured compiler layout and runtime checks establish the shared transport contract.

Meshlet bounds use a 20-byte raw record: the FP32 sphere occupies bytes 0 through 15 and the packed cone occupies bytes 16 through 19. CPU serialization and static/posed uploads derive sizes from MeshletBounds; raw shader readers and the posed writer share NWB_MESHLET_BOUNDS_STRIDE. The internal current mesh payload is MSH7, and older payloads are rejected and must be recooked through the launcher. Skin payloads reference their mesh separately. Asset authors do not set binary versions or strides. Main skinning pushes use five live scalar words (20 bytes); bounds, local bounds and normal repacking use two words (8 bytes). The shader derives attribute counts per meshlet, while the CPU resource cache retains its separate live counts. See GPU payload packing for compiled and runtime qualification.

ECS Skeleton Runtime

Runtime skeleton state is held in ECS components:

• SkeletonPoseComponent
• SkeletonJointPaletteComponent

SkeletonPoseComponent contains:

• parent joint vector
• local joint matrices
• skinning mode

SkeletonJointPaletteComponent contains:

• palette joint matrices
• skinning mode

Supported skinning modes:

• linear blend
• dual quaternion

Model Assets

Model assets compose multiple object types:

• skeleton objects
• static mesh objects
• skinned mesh objects

Authored model metadata uses explicit named maps: skeletons, static_meshes, and skinned_meshes. The map key is the stable model-object name, rather than an implicit list position. nwb_fbx_to_nwb emits those maps directly for both standalone model packages and asset bunches: a static export contains a named base static-mesh object, while a skinned export contains named skeleton and mesh objects. The skinned object requires skin and a local skeleton object name, with optional material and transform. It omits and rejects mesh; the loaded Skin supplies the authoritative mesh reference during source resolution. Static objects still declare their mesh.

Object transforms are optional (identity when absent) and use a finite 3x4 affine matrix with translation in the last column. The cooker rejects 4x4 source matrices rather than discarding their fourth row. The internal MDL2 model payload omits the duplicated mesh identity from skinned objects; recook older model payloads.

Static mesh objects can optionally attach to a parent model object and parent joint. Their final transform is:

model owner * parent object skeleton transform * parent joint pose * local transform

If no parent joint is provided, the joint term is omitted. If no parent object is provided, the model owner is used.

Model Expansion

The model ECS system expands a ModelComponent into runtime entities/components.

Common expansion components:

• ModelRuntimeComponent
• ModelObjectComponent
• ModelSkeletonComponent
• ModelStaticMeshAttachmentComponent
• MeshComponent
• SkinnedMeshBindingComponent
• RendererComponent
• skeleton pose/palette components

This approach keeps authored composition in assets while allowing renderer and skinning systems to operate on simple component sets.

The model system separates expansion from per-frame transform maintenance. prepare synchronizes model runtimes and creates or removes spawned model objects; update refreshes model object transforms and static mesh attachments. Projects that create model entities during startup can call syncModelRuntimes immediately to make expanded objects available before the first render.

Mesh System

MeshSystem owns runtime mesh registration and mesh resource tracking. It is also used by the skinning system so skinned runtime mesh handles can be registered consistently with static mesh handles.

Renderer mesh resources include GPU buffers and metadata derived from mesh payloads. The renderer prunes unused runtime mesh resources during prepare.

Mesh Skinning System

MeshSkinningSystem runs before RendererSystem in the Testbed render pass list.

It owns:

• runtime skinned mesh cache keyed directly by Skin identity; each loaded Skin supplies its authoritative mesh reference
• GPU skinning pipeline resources (created in validateResources via the ensure*Pipeline setup; the per-frame prepare path only consumes them through the *PipelineReady checks and fails if setup is missing)
• skinned mesh runtime instances
• skin payload upload/update
• meshlet bounds update resources

Per-frame responsibility (consume-only: pipeline creation never happens here; per-instance buffers stay demand-driven in prepare because sizes are unknown at setup):

1. inspect skinned mesh binding components
2. resolve Skin and skeleton data, loading the mesh from Skin at the source-load boundary and reusing the cached owner
3. update or reuse runtime cache entries
4. dispatch GPU skinning work
5. update skinned meshlet bounds
6. expose renderer-consumable runtime mesh handles

Submission acceptance and retryability

Recording a skinning dispatch does not immediately advance CPU-visible runtime state. Each runtime mesh records a MeshSkinningSubmissionCommit for its current edit revision. Only after the command-list submission is accepted does the system clear the dirty flags handled by that dispatch and mark its bindless resource-selector upload complete. replaceAcceptedSkinningState reuses one frame-owned live-buffer vector (m_frameLiveBuffers, cleared on scope exit) instead of allocating per call, and the accepted skinning snapshot supports in-place self-filtering (GpuPersistentResourceStateCache::replaceResourceSubset on its own state: retain-then-compact within the same arena). The unused MeshSkinningRuntimeCache::handleForEntity lookup and the hidden dirty-flag sanitize/expand helpers are removed.

If submission is rejected, those flags and the selector upload remain pending, so the next frame records a complete retry instead of treating unsubmitted GPU work as current. Applying a commit also requires the runtime mesh and resource generation to match its recorded edit revision; a newer deformable edit therefore cannot be cleared by a stale earlier submission.

Accepted Geometry Reuse

MeshSkinningDeformationState owns exact accepted and pending GPU joint palettes and compares the resolved palette, edit revision, skinning mode, and resource generation. Accepted unchanged output skips skinning, bounds, and attribute repacking; world-transform changes do not invalidate object-space geometry. The runtime mesh descriptor publishes an accepted geometry content revision (zero means unknown or pending and forces conservative work), and the ray-tracing domain keeps independently accepted revisions for the hardware BLAS and software mesh BVH. Only submission-acceptance callbacks publish reusable acceleration contents; rejection, resource replacement, and topology changes invalidate them.

The material path caches accepted object-space mesh attributes across draws (object_geometry_pass.cpp, task_graph_object_geometry_cache.*): ordinary instance/view changes reuse decoded vertices. On devices without native mesh shaders, the fixed engine program additionally uses the persistent indexed raster route (RenderPath::VertexIndexed), retaining cached object-space vertices plus a trailing u32 index region from a single decode so steady frames issue indexed draws with no decode or triangle-index work, while draws without a compatible indexed archive stage or required object-geometry sharing use compute emulation. Shared engine geometry can use indexed raster for CSG clip-only draws as well.

Meshlet Bounds Reduction

The bounds update uses one 128-thread workgroup per meshlet. Its portable default is a groupshared-tree reduction and does not assume that one hardware subgroup spans the workgroup. A wave-reduction variant exists but remains disabled; Vulkan 1.3 alone does not guarantee its required subgroup behavior, so enabling it requires an explicit capability-qualified policy.

Skinning also emits finite-valid AABB partials from per-meshlet position extrema and reduces them into a skinning-owned bounds resource tied to accepted deformation identity (local_bounds.cpp, optical_bounds_finalize_cs.slang). Unchanged accepted poses reuse local bounds. Raytracing transforms those bounds with the exact TLAS snapshot into GPU-resolved optical metadata for hardware optical traversal, without CPU readback and without substituting bind-pose bounds for arbitrary deformation.

Renderer Consumption

The renderer consumes static and skinned mesh data through gathered draw items. Each renderer-visible object needs:

• transform
• mesh or runtime skinned mesh handle
• material
• visibility
• optional CSG receiver state
• optional material instance overrides

The renderer does not need to know how a model was authored. By render time, model expansion and skinning have already produced the ECS state and runtime mesh resources it needs.