1.38.x / 1.39 Token Support

[bernardkwok]

Workflow context

The desired workflow is to allow for a set of tokens to be recognized within the context of a nodegraph / node and used for string substitution. One common use case is for image filename identifier resolving but can be used to resolve geometry identifiers. Tokens allow for more greater graph reusability as the same graph can be reused with different token replacement values. e.g. a brick shader that can accept different resolution tokens applied to the same base color filename requires only one tokened image node.

As of 1.38 it is given that:

• A <token>can be a children of <node> or <nodegraph>
• "Variations" (for which one application is user "presets") on tokens can be applied via <variant>s and <variantsets.

For 1.38.2 support logic for tokens will be completed. This means tokens

• Are considered when creating nodedefs from compound nodegraphs.
• Are "transparently" applied by String Resolvers when determining a "resolved" value. (Basically be part of the "get resolved value" API interface.
• Are resolved during shader code generation by via the new resolving logic (and hence are also resolving "transparently".)

For 1.39 updates the proposal is to support a workflow where a user can create reusable "token nodes" and "publish" these are interfaces on nodegraphs.
These new nodes:

1. Contain a list of string/token replacement values.
2. Can only exist within nodegraphs.
3. A inputs on a node in a nodegraph (including those found on a "token node' can be "published" as child tokens on the associated nodegraph. (part of the 1.38 assumptions). The addition is that such inputs are "connected" to the nodegraph child tokens.

1.38.2 Status: Changes in ADSK fork. In PR for LucasFilm merge, except Runtime which is not part of main yet.

• Usage of tokens at nodegraph level and nodedef level. (confirmed as part of specification - no "work")
• Publishing <token>s to nodedefs from compound nodegraphs. Updates have been added to "publishing" utility.
• Resolving tokens inside or outside code generation
  • <token> element children of a node, nodegraph or nodedef can be applied for string resolving of sibling inputs.
• Support of <token>s in Runtime module (added -- added)
  • **To fix **: Runtime stores tokens with uniform="true" attribute. This should not be added as tokens are always uniforms.

1.39 Details : Token "node" and token interface publishing : Under review

• Add a new node definition: "".
  • This node for now is a fixed list of child <token> Elements.
  • Proposed definition is as follows (here there are 5 tokens which is the ADSK proposed number):

<nodedef name="ND_token_node" node="tokennode" nodegroup="organization" type="none">
    <token name="token1" type="string" value="" />
    <token name="token2" type="string" value="" />
    <token name="token3" type="string" value="" />
    <token name="token4" type="string" value="" />
    <token name="token5" type="string" value="" />
  </nodedef>

• Any number of instances of such nodes can be created within a nodegraph only (compound or functional).
• As with "explicit" inputs "connections" can be made between any token on a node within the graph to an "explicit" token (token child of a nodegraph).
  • Using the same logic as <input>s, , <token> would have "interfacename" attributes to form the "connection".

Example:

1. A nodegraph with child tokens. (Note: child tokens show up as "ports with a "|" beside the square).
   

<?xml version="1.0"?>
<materialx version="1.38">
   <nodegraph name="Tokenized_Image" >
      <token name="Image_Resolution" type="string" value="2k"  uiname="Image Resolution" />
      <token name="Image_Extension" type="string" value="png" uiname="Image Extension" />
      <input name="Image_Filename" type="filename" uniform="true" value="Resources/images/cloth.[Image_Resolution].[Image_Extension]" />
      <tiledimage name="tiledimage" type="color3" nodedef="ND_tiledimage_color3" >
         <input name="file" type="filename" uniform="true" interfacename="Image_Filename" value="Resources/images/cloth.[Image_Resolution].[Image_Extension]"/>
      </tiledimage>
      <output name="out" type="color3" nodename="tiledimage" />
   </nodegraph>
</materialx>

The resolved value for "Image_Filename" input would be "Resources/images/cloth.2k.png".

2. Addition of a "token node" and exposure as an "explicit tokens":
   

<?xml version="1.0"?>
<materialx version="1.38">
  <nodegraph name="Tokenized_Image">
    <token name="Image_Resolution" type="string" value="2k" uiname="Image Resolution" />
    <token name="Image_Extension" type="string" value="png" uiname="Image Extension" />
    <input name="Image_Filename" type="filename" value="Resources/images/[Image_Name].[Image_Resolution].[Image_Extension]"  />
    <token name="Image_Path" type="string" uniform="true" value="Resources/images/cloth" />
    <tiledimage name="tiledimage" type="color3" nodedef="ND_tiledimage_color3" >
      <input name="file" type="filename" uniform="true" interfacename="Image_Filename" value="[Image_Name].[Image_Resolution].[Image_Extension]"  />
    </tiledimage>
   <!-- Token node -->
    <tokennode name="tokennode" type="" nodedef="ND_token_node">
     <!-- Tokens are published as part of the interface -->
      <token name="token1" type="string" interfacename="Image_Resolution" value="2k" />
      <token name="token2" type="string" interfacename="Image_Extension" value="png"  />
      <token name="token3" type="string" interfacename="Image_Path" value="Resources/images/cloth"  />
    </tokennode>
    <output name="out" type="color3" nodename="tiledimage" />
  </nodegraph>
  </materialx>

"Image_Path" from the token node "tokennode" is "exposed" as an interface on the nodegraph.
The resolved value for "Image_Filename" input it still "Resources/images/cloth.2k.png".

3. A possible node definition (<nodedef>) and associated functional nodegraph created from the compound nodegraph:

<?xml version="1.0"?>
<materialx version="1.38">
<!-- New node definition -->
<nodedef name="ND_Tokenized_Image_color3" node="Tokenized_Image" version="1.0" isdefaultversion="true" nodegroup="texture2d" namespace="ns">
  <token name="Image_Resolution" type="string" uniform="true" value="2k" uivisible="true" uiname="Image Resolution" />
  <token name="Image_Extension" type="string" uniform="true" value="png" uivisible="true" uiname="Image Extension" />
  <input name="Image_Filename" type="filename" uniform="true" value="Resources/images/cloth.[Image_Extension]" uivisible="true" />
  <input name="Image_Path" type="string" uniform="true" value="4096" />
  <output name="out" type="color3" value="0, 0, 0" />
</nodedef>

<!-- Functional node graph -->
<nodegraph name="NG_Tokenized_Image_color3" nodedef="ND_Tokenized_Image_color3">
  <token name="Image_Resolution" type="string" uniform="true" value="2k" uivisible="true" uiname="Image Resolution" />
  <token name="Image_Extension" type="string" uniform="true" value="png" uivisible="true" uiname="Image Extension" />
  <input name="Image_Filename" type="filename" value="Resources/images/[Image_Name].[Image_Resolution].[Image_Extension]"  />
  <token name="Image_Path" type="string" uniform="true" value="Resources/images/cloth" />
  <tiledimage name="tiledimage" type="color3" nodedef="ND_tiledimage_color3" >
    <input name="file" type="filename" uniform="true" interfacename="Image_Filename" value="Resources/images/cloth.[Image_Extension]" uivisible="true" />
  </tiledimage>
  <tokennode name="tokennode" type="" nodedef="ND_token_node" >
    <token name="token1" type="string" interfacename="Image_Resolution" value="2k" />
    <token name="token2" type="string" interfacename="Image_Extension" value="png" />
    <token name="token3" type="string" interfacename="Image_Path" value="Resources/images/cloth"  />
  </tokennode>
  <output name="out" type="color3" nodename="tiledimage" />
</nodegraph>
</materialx>

When an instance of such a definition is created it will have modifiable "published" token values:
In this example, the resolve "Image_Filename" would be "Resources/images/brick.4k.exr".