Multidata

Problems

Problem 1: Complex instantiation

XML3D currently only supports <mesh> to display geometry. In addition, instantiation is only possible on a data level. That means, that complex models consisting of N meshes can only be instantiated by declaring N <mesh> elements + additional <group> elements for transformation + shader assignment.

<group shader="house.xml#roofShader" transform="..." >
  <mesh type="triangles" src="house.xml#roof" />
</group>
<group shader="house.xml#frontShader" transform="..." >
  <mesh type="triangles" src="house.xml#frontWall" />
</group>
<group shader="house.xml#wallShader" transform="..." >
  <mesh type="triangles" src="house.xml#leftWall" />
</group>
<group shader="house.xml#wallShader" transform="..." >
  <mesh type="triangles" src="house.xml#rightWall" />
</group>
<group shader="house.xml#wallShader" transform="..." >
  <mesh type="triangles" src="house.xml#backWall" />
</group>

E.g. to include complex models, users need to understand 3 things:

1. The number and names of all meshes included in the model
2. the correct shader for each mesh
3. the correct transformation of each mesh.

Thus it is not really possible to simply put a complex model into XML3D - the user needs to copy the whole hierarchy, potentially resolving id conflicts.

Problem 2: Complex dataflow linking for instantiation

For more dynamic models, we do not only need to replicate the hierarchy with transformations and shader assignments, but must also connect dataflow nodes with input parameters to drive the animation - all this for each instance.

Here the full tag code to instantiate a TF2 animated mesh model:

  <!-- Apply skinning operator on base data of mesh -->
  <data id="sniperSkinned" proto="protos.xml#skinning">
    <data src="asset/sniper.xml#meshbase" />
    <data src="asset/sniper.xml#anim_taunt02_heelClick" />
    <float id="keySniper" name="key">0.0</float>
  </data>
  <!-- Instantiate each submesh by referring processed base data + correct index: -->
  <group shader="asset/sniper.xml#shader_sniper_head">
    <mesh type="triangles">
      <data src="#sniperSkinned" />
      <data src="asset/sniper.xml#index_sniper_head" />
    </mesh>
  </group>
  <group shader="asset/sniper.xml#shader_eyeball_l">
    <mesh type="triangles">
      <data src="#sniperSkinned" />
      <data src="asset/sniper.xml#index_eyeball_l" />
    </mesh>
  </group>
  <group shader="asset/sniper.xml#shader_eyeball_r">
    <mesh type="triangles">
      <data src="#sniperSkinned" />
      <data src="asset/sniper.xml#index_eyeball_r" />
    </mesh>
  </group>
  <group shader="asset/sniper.xml#shader_sniper_red">
    <mesh type="triangles">
      <data src="#sniperSkinned" />
      <data src="asset/sniper.xml#index_sniper_red" />
    </mesh>
  </group>
  <group shader="asset/sniper.xml#shader_sniper_lens">
    <mesh type="triangles">
      <data src="#sniperSkinned" />
      <data src="asset/sniper.xml#index_sniper_lens" />
    </mesh>
  </group>
  <group shader="asset/sniper.xml#shader_c_arrow">
    <mesh type="triangles">
      <data src="#sniperSkinned" />
      <data src="asset/sniper.xml#index_c_arrow" />
    </mesh>
  </group>

This is not only a lot of code, it also introduces a document id per instance, which makes the handling much more inconvient.

Potential Solution: Instantiate <group>

While the instantiation of whole <group> hierarchies would solve Problem #1, it does not help for Problem #2. This is because we actually need to be able to influence values deeply connected in the hierarchy, e.g. the input parameters of the base data to drive the skinning animation.

Because of that, instantiation of <group> is not a good solution as it lacks flexibility.

Suggested Solution: DataList

A suggested solution is the introduction of a new feature: datalist. A summary on what datalist is:

• datalist is orthogonal to Xflow
• It is a wrapper around Xflow nodes to simplify the composition of dataflows (e.g. without introducing new document ids)
• datalist contains mesh specific content to simplify the instantation of multipe mesh models
• New elements to declare datalists: <datalist> and <subdata>
• New element to instantiate a geometry from datalist: <multimesh> (maybe call <geometry>?)

Examples #1: House Model

<!-- Data Declaration (can be extern): -->
<datalist id="housedatalist" >
  <subdata name="roof" meshtype="triangles" shader="#roofShader" >
    <int name="index" >...</int>
    <float3 name="position" >...</float3>
    <float3 name="normal" >...</float3>
    <float4x4 name="transform">...</float4x4>
  </subdata>
  <subdata name="front" meshtype="triangles" shader="#frontShader" >
    <int name="index" >...</int>
    <float3 name="position" >...</float3>
    <float3 name="normal" >...</float3>
    <float4x4 name="transform">...</float4x4>
  </subdata>
  <subdata name="right" meshtype="triangles" shader="#wallShader" >
    ...
  </subdata>
  <subdata name="left" meshtype="triangles" shader="#wallShader" >
    ...
  </subdata>
  <subdata name="back" meshtype="triangles" shader="#wallShader" >
    ...
  </subdata>
</datalist>
<!-- Instantiation -->
<multimesh src="#housedatalist" ></multimesh>

Examples #2: TF2 Model

<!-- Data Declaration (can be extern): -->
<datalist id="sniperdatalist" >
  <!-- Declaration of shared sub data -->
  <subdata name="base">
    <float3 name="position" >...</float3>
    <float3 name="normal" >...</float3>
    <float2 name="texcoord" >...</float2>
    <!-- + All the skinning assets -->
  </subdata >
  <!-- Declaration of subdata that includes 'base' subdata via 'includes' attribute -->
  <subdata name="head" includes="base" meshtype="triangles" shader="#headShader" >
    <int name="index" >...</int> 
  </subdata>
  <subdata name="eyeball_r" includes="base" meshtype="triangles" shader="#eyeShader" >
    <int name="index" >...</int> 
  </subdata>
  <subdata name="eyeball_l" includes="base" meshtype="triangles" shader="#eyeShader" >
    <int name="index" >...</int> 
  </subdata>
  <subdata name="body" includes="base" meshtype="triangles" shader="#bodyShader" >
    <int name="index" >...</int> 
  </subdata>
  <subdata name="lens" includes="base" meshtype="triangles" shader="#lensShader" >
    <int name="index" >...</int> 
  </subdata>
  <subdata name="lens" includes="base" meshtype="triangles" shader="#lensShader" >
    <int name="index" >...</int> 
  </subdata>
</datalist>

<!-- Extension of #sniperdatalist -->
<datalist id="sniperSkinned" src="#sniperdatalist" >
  <!-- here we extend the 'base' subdata and apply a prototype via 'postproto' -->
  <subdata name="base" postproto="#skinning" >
    <data src="#someAnimationData" />
    <float name="key">0.0</float>
    <!-- original data of 'base' is implicitly added -->
  </subdata>
</datalist >

<!-- Instantiation -->
<multimesh src="#sniperSkinned" />

<!-- Just as with <data> we can also simply inline the datalist extension, avoiding the document id -->

<multimesh src="#sniperdatalist" >
  <subdata name="base" postproto="#skinning" >
    <data src="#someAnimationData" />
    <float name="key">0.0</float>
  </subdata>
</multimesh>

<!-- It is also possible to predeclare the datalist with embedded prototype: -->

<datalist id="sniperSkinTemplate" src="#sniperdatalist" >
  <subdata name="base" postproto="#skinning" >
    <data src="#someAnimationData" />
    <float name="key">0</float> <!-- Default key -->
  </subdata>
</datalist>

<!-- Instantiation only providing key, no prototype -->

<multimesh src="#sniperdatalist" >
  <subdata name="base">
    <float id="keySniper" name="key">4. 6</float>
  </subdata>
</multimesh>

How datalist works

datalist is more or less syntactical sugar for writing Xflow graph in combination with meshes.

In the following, we will compare datalist with equivalent XML3D markup without datalist.

Behaviour of includes attribute

The includes attribute will includes the Xflow data node of a <subdata> element with matching name:

<!-- Multi Data -->
<multimesh id="person" >
  <subdata name="base1" >
    <!-- BASE1 data -->
  </subdata>
  <subdata name="base2" includes="base1" >
    <!-- BASE2 data -->
  </subdata>
  <subdata name="base3" >
    <!-- BASE3 data -->
  </subdata>
  <subdata name="A" includes="base2 base3" meshtype="triangles" shader="#someShader" >
    <!-- A data -->
  </subdata>
</multimesh>

<!-- Raw Mesh Equivalent: -->

<data id="person_base1">
  <!-- BASE1 data -->
</data>

<data id="person_base2">
  <data src="#person_base1" />
  <data>
    <!-- BASE2 data -->
  </data>
</data>

<data id="person_base3">
  <!-- BASE3 data -->
</data>

<mesh type="triangles" shader="#someShader" >
  <data src="#person_base2" />
  <data src="#person_base3" />
  <data>
    <!-- A data -->
  </data>
</mesh>

Behaviour of src attribute

With src, we can refer another multimesh element. All subdata and subdata entries will be reused, but can be extended if needed:

<!-- Multi Data -->
<datalist id="basedatalist" >
  <subdata name="sub1" >
    <!-- SUB1 data -->
  </subdata>
  <subdata name="mesh1" includes="sub1" meshtype="triangles" shader="#someShader" >
    <!-- MESH1 data -->
  </subdata>
</datalist>

<multimesh src="#basedatalist" >
  <subdata name="sub1" >
    <!-- SUB1 EXTEND data -->
  </subdata>
  <subdata name="sub2" >
    <!-- SUB2 data -->
  </subdata>
  <subdata name="mesh1" includes="sub2" shader="#someOtherShader">
    <!-- MESH1 EXTEND data -->
  </subdata>
  <subdata name="mesh2" includes="sub2" shader="#someShader" >
    <!-- MESH2 data -->
  </subdata>
</multimesh>

<!-- Raw Mesh Equivalent: -->

<data id="sub1">
  <!-- SUB1 data -->
</data>

<data id="mesh1">
  <!-- MESH1 data -->
</data>

<data id="sub1_ext">
  <data src="#sub1" />
  <data>
    <!-- SUB1 EXTEND data -->
  </data>
</data>

<data id="sub2">
  <!-- SUB2 data -->
</data>

<mesh id="mesh1_ext" type="triangles" shader="#someOtherShader" >
  <!-- first we include all data connected via the 'includes' attribute -->
  <data src="#sub1_ext" />
  <data src="#sub2" />
  <!-- Then we include the overridden data -->
  <data src="#mesh1" />
  <!-- And finally the extended data -->
  <data>
    <!-- MESH1 EXTEND data -->
  </data>
</mesh>

<mesh id="mesh2" type="triangles" shader="#someShader" >
  <data src="#sub2" />
  <data>
    <!-- MESH2 data -->
  </data>
</mesh>

Behaviour of postproto, postcompute and postfilter attribute

Postproto, postcompute and postfilter are all applied after all data of a multimesh is combined. That way, it is possible to influence input parameter of all processing even with the last extension.

<!-- Multi Data -->
<datalist id="basedatalist" >
  <subdata name="sub1" >
    <float3 name="position" >...</float3>
  </subdata>
  <subdata name="mesh1" includes="sub1" meshtype="triangles" shader="#someShader" >
    <int name="index" >...</int>
  </subdata>
</datalist>

<datalist id="shapeAdd1" src="#basedatalist" >
  <subdata name="sub1" postcompute="position = xflow.morph(position, posAdd1, weight1)" >
    <float3 name="posAdd1" >...</float3>
    <float name="weight1" >0</float><!-- default weight -->
  </subdata>
</datalist>

<datalist id="shapeAdd2" src="#shapeAdd1" >
  <subdata name="sub1" postcompute="position = xflow.morph(position, posAdd2, weight2)" >
    <float3 name="posAdd2" >...</float3>
    <float name="weight2" >0</float><!-- default weight -->
  </subdata>
</datalist>

<multimesh src="#shapeAdd2" >
  <subdata name="sub1" >
    <!-- Override default weights -->
    <float name="weight1" >0.3</float>
    <float name="weight2" >0.8</float>
  </subdata>
</multimesh>

<!-- Raw Mesh Equivalent: -->

<data id="sub1">
  <float3 name="position" >...</float3>
</data>

<data id="sub1_ext">
  <data src="#sub1" />
  <data>
    <float3 name="posAdd1" >...</float3>
    <float name="weight1" >0</float><!-- default weight -->
  </data>
</data>

<data id="sub1_ext2">
  <data src="#sub1_ext" />
  <data>
    <float3 name="posAdd2" >...</float3>
    <float name="weight2" >0</float><!-- default weight -->
  </data>
</data>

<data id="sub1_ext3">
  <data src="#sub1_ext2" />
  <data>
    <float name="weight1" >0.3</float>
    <float name="weight2" >0.8</float>
  </data>
</data>

<mesh type="triangles" shader="#someShader" >
  <data compute="position = xflow.morph(position, posAdd2, weight2)" >
    <data compute="position = xflow.morph(position, posAdd1, weight1)" >
      <data src="#sub1_ext3" />
    </data>
  </data>
  <data>
    <int name="index" >...</int>
  </data>
</mesh>