declarative nodeMaterial snippet

[jbixon13]

I'm new to babylon after learning three.js and react-three-fiber, so thank you for creating a declarative version of babylon to help ease the transition! I was hoping to try out a nodeMaterial by loading it with ParseFromSnippetAsync but I'm not sure whether that's possible declaratively, or whether I need some async functional component setup that I haven't considered.

Codesandbox I'm testing on: https://codesandbox.io/s/react-babylon-testing-bzp2q?file=/src/BabylonCanvas.jsx

I'm trying to add the nodeMaterial to the sphere on line 25 but I'm not sure how to approach it. I noticed a snippetId prop but it doesn't seem to work as expected. Any advice would be helpful!

I noticed a similar question on the forum that you answered regarding the CreateFromSnippetAsync function in a useEffect but I wasn't sure if a similar approach is necessary here or how to set it up without access to their code.

https://forum.babylonjs.com/t/particlehelper-createfromsnippetasync-is-not-a-function-in-react-app/14321

[brianzinn]

@jbixon13 what about something like this (you'd maybe want to add snippetId to the dependency array):

const SnippetMaterial = (props) => {
  const scene = useBabylonScene();
  const material = useRef(null);
  useEffect(() => {
    NodeMaterial.ParseFromSnippetAsync(props.snippetId, scene).then(
      (nodeMaterial) => {
        material.current = nodeMaterial;
      },
      []
    );
  });

  return material.current === null ? null : (
    <nodeMaterial name={props.name} fromInstance={material.current} />
  );
};

In your main component:

<sphere
          name="sphere1"
          diameter={2}
          segments={16}
          position={new Vector3(0, 1, 0)}
        >
          <SnippetMaterial name="sphereMat" snippetId="#81NNDY#20" />
        </sphere>

Snippet isn't support out of the box - it's a static method (and async) - we could add it as a new material - you'll notice it loads kind of wonky, since it's waiting to download the snippet before applying.

[jbixon13]

Thank you! I've updated my codesandbox to try replicating your approach, but I can't seem to get it to work. I'm not sure if it's the dependency array like you said or something else but I'll keep playing with it.

[brianzinn]

I edited the one file BabylonCanvas.jsx to this:

import React, { useRef, useEffect } from "react";
import { Engine, Scene, useBabylonScene } from "react-babylonjs";
import { Vector3, NodeMaterial } from "@babylonjs/core";

const SnippetMaterial = (props) => {
  const scene = useBabylonScene();
  const material = useRef(null);
  useEffect(() => {
    NodeMaterial.ParseFromSnippetAsync(props.snippetId, scene).then(
      (nodeMaterial) => {
        material.current = nodeMaterial;
      },
      []
    );
  });
  //<nodeMaterial name="sphereMat" snippetId={"#81NNDY#20"} />
  return material.current === null ? null : (
    <nodeMaterial fromInstance={material.current} />
  );
};

const BabylonCanvas = () => {
  return (
    <Engine antialias>
      <Scene>
        <freeCamera
          name="camera1"
          position={new Vector3(0, 2, -10)}
          target={Vector3.Zero()}
        />
        <hemisphericLight
          name="light1"
          intensity={0.7}
          direction={Vector3.Up()}
        />
        <sphere
          name="sphere1"
          diameter={2}
          segments={16}
          position={new Vector3(0, 1, 0)}
        >
          <SnippetMaterial name="sphereMat" snippetId="#81NNDY#20" />
        </sphere>
        <ground name="ground1" width={6} height={6} subdivisions={2} />
      </Scene>
    </Engine>
  );
};

export default BabylonCanvas;

[jbixon13]

updated again and copied the above as-is, still doesn't work for some reason 😕

[jbixon13]

got it! I needed to update the first dependency array, not add a second one like I was doing before. Thanks for your quick and useful help :)

[jbixon13]

ok interestingly it works if I update the dependency array and it auto-refreshes to add the material but goes back to not working if I hard-refresh the app. It may just be a codesandbox issue, I'll try it locally

[brianzinn]

Yeah, that's a typo from me with the dependency array in the wrong place. Also, if you don't want the sphere showing up with the default material then you could put the sphere into the component as well :)

[brianzinn]

@jbixon13 sorry about that code I sent you - the reason is that useEffect doesn't trigger a re-render. There are many ways you can force a rerender once the nodeMaterial has loaded the snippet - here is one:

const SnippetMaterial = (props) => {
  const scene = useBabylonScene();
  const [material, setMaterial] = useState(null);
  const parseMaterial = useCallback(async () => {
    NodeMaterial.ParseFromSnippetAsync(props.snippetId, scene).then(
      (nodeMaterial) => {
        setMaterial(nodeMaterial);
      }
    );
  }, [props.snippetId, scene]);

  useEffect(() => {
    parseMaterial();
  }, [parseMaterial]);

  return material === null ? null : (
    <nodeMaterial name={props.name} fromInstance={material} />
  );
};

[jbixon13]

that works, both locally and in codesandbox! Thanks again for the help

[dennemark]

Hi,
I am using a hook to change nodeMaterial values. Maybe it will be of use:

import { useEffect } from 'react';
import { Color3, Color4, Vector2, Vector3 } from "@babylonjs/core/Maths/math";
import {  NodeMaterial } from "@babylonjs/core/Materials/Node/nodeMaterial";
import {  Texture } from "@babylonjs/core/Materials/Textures/texture";
import { Nullable } from "@babylonjs/core/types";

export type NodeValue = number | Color3 | Color4| Vector2 | Vector3 | Texture;


const useNodeMaterialBlock = (
    value: Nullable<NodeValue>,
    blockName: string,
    nodeMaterial: Nullable<NodeMaterial>,
    ) => {  

  useEffect(() =>{
    if(nodeMaterial !== null && nodeMaterial instanceof NodeMaterial){
        setBlockValue(blockName, value as NodeValue, nodeMaterial);
    }
  }, [value, blockName, nodeMaterial])
}

const setBlockValue = (
    name: string,
    value: NodeValue,
    material: NodeMaterial
  ) => {
      if(value instanceof Texture){
        let textureBlock = material.getTextureBlocks().find((b) => b.name === name); //does it make sense to use memo here? caching might be useful
        if (textureBlock !== undefined) {
          textureBlock.texture = value;
          textureBlock.convertToLinearSpace = true;
        }
      }else{
        let block = material.getInputBlockByPredicate((b) => b.name === name);
        if (block !== null) {
        block.value = value;
        }
    }
  };

  export default useNodeMaterialBlock;

If you want to use it with brianzinn's code, you could give the SnippetMaterial i.e. an array of objects as a prop:
[{ValueOfInputOrTextureNode, "NameOfNodeBlock"}, ...]

and iterate through all of these values within the snippet Material and assigning the hook. I even use to unfreeze and freeze the material before and after the use of the hook, so the material should be performant, if it is not changed too often.

  props.blockValues?.forEach((entry)=>{
      if(entry.value !== undefined){
        useNodeMaterialBlock(entry.value, entry.name, material) //material is the same state variable as in brianzinn's code
      }
  })

(Actually I thought I could contribute a story, but currently I am not certain if I should use the 2.3.x version or jump on 3.0.0 - also in connection to the babylonjs-loaders. I hope @brianzinn you can give us an update soon :) )

[brianzinn]

Thanks @dennemark - I would be really keen to get some new stories and recipes on materials. I would recommend the new v3-proper branch. I've created a discussion (github feature finally GA):
#107
Happy to continue there and I should have more time next week to dig in to v-next! 😄
As a side-note I am also considering moving the storybook to a separate repo to clean this one up a lot - it's pulling in a lot of dev dependencies...

[brianzinn]

@dennemark did you want to contribute a node material hook? seems really useful, especially with freezing and setting block values. Would like to see it in use or as a recipe!

[jbixon13]

I'm just now getting back into this project and had another question regarding node materials.

In the current material being passed to the sphere in my example the green color is hard-coded in the surface color node. Is there any way to pass this from a sphere object so that the same material can be used on multiple objects with different colors? This would be similar to how color is currently passed into standardMaterial through diffuseColor or specularColor.

I see an option in the NME to create a mesh.color node, but I'm not sure how I would pass that from the mesh to the Node Material.

Basically I am trying to understand the best way to create a scene that has toon properties without having to use a new material on each object

[brianzinn]

@jbixon13 I did check the docs and they mentioned that colors are not taken as uniform (and can change) by default unless you freeze them. I just took part of the hook from dennemark's comment above and applied to existing story (maybe add material to dep array...).

useEffect(() => {
  // there is a race condition here if color changes before material is loaded would lose that value
  if (material) {
    let block = material.getInputBlockByPredicate(b => b.name === 'Surface Color');
    block.value = Color3[surfaceColor]();
  }
}, [surfaceColor]);

If you wanted to work on a fully declarative syntax that would allow the react-reconciler to pass through values like Color3 then I can build that for you - would just need an imperative playground. I think it would be very similar to how 2D/3D GUI declarative syntax builds out the GUI and I suspect it would be very easy to work with using Lifecycle methods - at least a basic version where the NodeMaterial only built once to start.

[brianzinn]

@jbixon13 Hope that example helps and also @dennemark updated the story to freeze the material as well as a hook to make it easier to update the Node Material blocks (https://github.com/brianzinn/react-babylonjs/blob/master/stories/babylonjs/Basic/snippetMaterial.stories.js). If you wanted something more declarative or have any more questions please re-open. Cheers.