Create a node based system for material

[sebavan]

One of the struggle with the current default provided materials (and effects at some extend) is their monolithic build making even simple customization hard to do without building a custom version of Babylon.js.

To simplify the customization and creation of derived materials it would be great to rely on a node based system and ensure that our default materials could rely on it internally.

Not only customization would be easy, but the usual tech artist creation workflow relying on node based system should become possible.

This would work to solve issues like :

• https://forum.babylonjs.com/t/pbr-shader-name-property/1731/4
• Most of the customization available in shader builder
• Some of our partners requirements (extending the default material system)

Design will be incoming right after 4.0 to ensure it will have a long run in 4.1 alpha.

Tasks:

• Vec2 transform
• Matrix multiplication
• Vec3 transform
• Vec4 transform
• Vec4 splitter (into r, g, b, a outputs)
• Vec3 splitter (r, g, b)
• Vec4 merger (from r,g,b,a inputs)
• Vec3 merger (from r, g, b)
• Vector multiplication
• Vector addition
• Vector scale (by a float)
• MorphTarget
• Texture read (including uv transform by a matrix)
• Fog
• Instances
• Bones
• Light (can be set to a specific light)
• Light (let the system pick active lights)
• Pixel rejection
• Multiply / Scale and MatrixMultiplication blocks should be the same
• Emit well-known values with same name to reuse them
• Add support for direct connection of attribute to a fragment node
• Dot block
• Cross block
• Add RGB, R, G and B as output on texture block
• Remap (e.g. map values with range [-1, 1] to [0, 1])
• Format serialize JSON
• Shadow
• Reflection
• Trigonometry
• Make sure unconnected output do not generate code
• Add NodeMaterial.CreateDefault(scene)
• Emit comments seems broken
• build API to easily access blocks
• Fresnel
• Lerp
• subtraction node
• exponent or power node
• division node
• Expose vertex color in the vector 3 node
• step node
• Drag out a new node should auto select it
• Converter from color to vector and the opposite
• Search!!!!
• Add Round, Ceiling, and Floor nodes
• Abililty to export shader code for inclusion into project
• Block added to basic math group: one minus (1 - input)
• Auto configure should look for already added blocks before creating new ones
• Block to calculate view angle from camera.position and world matrix
• Block to calculate the light angle for directional light, or a direction from spot/point to pixel to use with dot product for mesh normal
• Add Min and Max blocks
• Add light colors to light info block
• Optimizer to reuse well-known and uniforms
• Pertub normal node
• Add an auto-turn table on the preview
• Block added to range group: invert (color)
• Add rotation support on texture block
• The ability to wire a vector1 to the RGB input of FragmentOutput. This could be wiring direct where the vector1 is wired to R, G, and B under the hood. As a backup, automatically dropping a vector/color merger with the vector1 wired to RGB/XYZ and then to the RGB input is also fine.
• RotationBlock
• Distance block
• Length block
• Create new simple shader ball mesh to be used as a preview primitive.
• Prevent float on cross or dot
• DiscardBlock (https://forum.babylonjs.com/t/pass-value-to-shader-from-gui/5646)
• Random block
• Log Block
• aTan Block
• aCos Block
• aSin
• Fract Block
• Sign Block
• aTan2 Block
• DeltaTime Block
• SmoothStep Block
• Reciprocal Block
• Radian2Degree and Degree2Radian blocks
• Replace Color block
• Posterize Block
• Sawtooth Wave Block
• Square Wave Block
• Triangle Wave Block
• Gradient Block
• WorldPosition Input Block
• WorldNormal Input Block
• Add "hinting" for direction toggle on transform node to activate when connected to vectors that will only be a direction like mesh.normal to save clicks for the user.
• Add toggle for Pre Depth Pass to the Node Material Preview Window. It can be off by default, but should operate just like the backface culling toggle.
• Add a normal direction node (1 for front face and -1 for back face returning a float)

Designer:

• uploading a texture to the texture node is broken. Try uploading a texture, then deselect the texture node, and then select it again...you'll see that the texture gets set back to "link" mode. It never uses the texture you upload.
• Add extra lights to preview window with toggles to turn each individual light on or off. Playground scene coming from @PatrickRyanMS with all of the settings you will need.
• Add a time stamp to the console window at the bottom for console messages so the user can track that a change has happened. If the same message keeps reporting in the window, it's difficult to know if the console changed or not.
• Provide input for the Reflection texture to allow dds and env
• Pulling out a Fresnel node should include the view direction input node
• Tooltips over output nodes
• Can we add an option so that when a vector block is marked as a constant that we change the background color (css) of the node so that we can easily track from the node graph which are constants and which are uniforms?
• Need to add a scalar on the gloss input for lighting before using that value in Pow so that gloss maps that fall in the 0-1 range actually run the gamut from matte to highly glossy.
• Add specular (vector3) and gloss (float) inputs to the light node to automate calculations of lighting contribution based on a spec/gloss texture
• Sometimes lose link parameter on an existing texture node when adding another texture node to the scene. This seems to be linked to the bug where UV tiling was being shared between texture nodes due to optimization. This seems to only happen when you have one texture node in the graph that is linked to a texture and you add a new texture node (the previous node will look like the texture was embedded).
• Update rounded cylinder in primitives with new glb
• Add a checkbox in parameters of user defined values (float, vector2, vector3, etc) to define the value as a constant and not a uniform.
• Add a reload button in texture node when texture is not embedded. Changes to the source do not refresh due to the cache. Even switching back to embed and back to link will pull from the cache.
• save as sur hosted document
• Add an option to not embed images in save and code generation
• Add loaders to nme
• Display message if screen width <768px
• Add range floats to float node. If they are not null, show a slider UI control in inspector
• Reduce speed of turntable on preview primitive to 35% of current speed.
• Cross product node is named "Dot" by default when placed into workspace.
• Change background behind preview primitive to match node tree background color. As a +1, the ability to switch the background color to a user chosen color (even if it's a predefined list) would be awesome.
• "opposite" node has a name that isn't clear as to what it's doing. Opposite could mean one minus, negate, or invert. Since we are doing a "one minus" operation, I think we should name the node "one minus."
• Add custom input fields in inspector for Matrix node in NME
• In NME, if you drag in a light information node, there is no light to select in the scene and so you cannot develop with that node in NME.
• Copying a sine node, pastes as a cosine node.
• Changing a cosine node to a sine node and then saving, doesn't save. The node opens back up as a cosine node.
• Dragging the input of a subtract node out does not create a float, instead it draws a red line to the center of editor.
• Being able to reload the editor by clicking edit on a different node material in the inspector
• Add User Input Properties to the Clamp node. Similar to Remap node. Clamp defaults to 0-1, but should be changeable by the user
• When node is selected and a new node is dragged onto the workspace, both nodes are selected. We should deselect the previous node.
• node groups not in alpha order. If we want to group things like math, we should use Math: Basic, Math: Trig, and Math: Vector. I'm finding it hard to find nodes even when I know what group they are in because input is at the bottom and I have to memorize a random mapping rather than knowing "I" for input should be middle top
• Focus not released from inputtext when clicking inside the editor
• Change FragmentOutput node to only accept RGBA or RGB. It cannot have both. Same logic for alpha
• Discrepancy between preview and playground rendering when wiring mesh normal to fragment output
• Cannot render world normal on mesh
• Lerp node has "Fresnel" title.
• Copy/paste for ALL selected nodes (not just the first one)
• In Edge, unable to scroll block list with mouse wheel
• Renaming of inputs do not behave correctly (+ add name (type))
• Node list scrolling does not work on Edgium
• Replace mode in inputBlock with a dropdown
• Inspector will show user defined values (when activated)
• Renaming a node does not work
• Bug with scrolling on node list
• Copy/Paste
• Check why comments are not generated
• more meshes for preview (box and torus)
• Store node position when serializing
• Find a way to preview the canvas in the node editor
• Store texture data when coming from a blob
• Let the user save / load a graph even if it does not compile
• Limit well known and mesh properties per input type
• Display constants on inputs
• General properties for material (like alpha mode)
• Property widget for fragment output
• Block dragging from output ports
• Crash when using RGBAMerger
• Remove zindex use
• Let the user to drag an port and generate an input
• Limit drag and drop to node list only
• Improve warning when dragging incompatible connections
• Save button (use material.serialize into .json file)
• Load button
• Export as shader
• Drag'n'drop for blocks
• Add float input
• Resizable colums
• Do not change type of non connected input
• Reset button (restore to a new simple material)
• Block incompatible connections
• Add support for texture block dropping (So far generates an error)
• Allow drop of inputs directly
• Update UI when texture is changed
• Make sure removing output nodes is supported
• let user change light affected to an input node
• Error messages in red
• New block should be with no input if input set to undefined

UI/UX

• Design explorations for node appearance
• Design exploration for node interface (input/output)
• Rework list of nodes to add icons
• Add hover tool tip over each node in the node toolbox

[sebavan]

Reference to the forum thread: https://forum.babylonjs.com/t/node-based-material/1739

TL/DR: I ll summarize it here.

[deltakosh]

List moved to first topic

[sebavan]

The community feedback and internal partners is a strong GO :-) I can not wait to try it.

[sebavan]

The new editor by @deltakosh is pretty cool :-) This is all getting real !!!