SkinnedMesh + ShaderMaterial

[pehrlich]

It looks like this isin't supported out-of-the-box. I've been trying to work my way through the THREE.js source, and see how to bring ShaderMaterial to SkinnedMesh.

It looks like there are a four vertex ShaderChunks and zero fragment ShaderChunks available which apparently relate to Shaders. Other chunks may relate, but these are the ones with skin in the name.

THREE.ShaderChunk[ "skinning_pars_vertex" ]
THREE.ShaderChunk[ "skinbase_vertex" ]
THREE.ShaderChunk[ "skinnormal_vertex" ]
THREE.ShaderChunk[ "skinning_vertex" ]

These each have 5-6 matches. Once to be defined, and then once again for basic, lambert, phong, normalMap, depthRGBA (not sure what this last one is).

I presume that by studying the implementation skinning for these other materials, one could bring skinning to ShaderMaterial as well.

Is this accurate? Or are there other blockers that would be encountered?

Thanks

EDIT: oops, still reading. Clearly there's more to it than this. Please hold..
EDIT2: It looks like it should be possible to add custom fragment shader logic to one of these existing materials. The trick might be uv mapping, as there isin't any by default. hm.

[mrdoob]

Is this accurate? Or are there other blockers that would be encountered?

It should. I wonder if there is a way to somehow have a list of the chunks used per material...

[pehrlich]

Setting Shader Material down for a moment, I'll post back here when I make some further progress.

---

One thing comes to mind that would be useful - if the non-compressed three.js was annotated with component source file locations. E.g., it wasn't immediately obvious that ShaderChunk was in renderers/, as opposed to materials or core.

In answer to your question: Instead of having ShaderChunks as JS arrays, it might be nice to have each chunk as an individual .glsl file within shaders/, which would then be built by grunt (or whatever you use to build) in to strings. This would allow organization, syntax highlighting, and other crunchy goodness I'm sure.

/renderers/shaders/fragments/fog_pars.glsl
/renderers/shaders/fragments/fog.glsl
...
/renderers/shaders/vertex/lights_lambert.glsl

Then materials could be broken up as well:

/renderes/shaders/materials/basic.js
/renderes/shaders/materials/lambert.js

and so on

[ddstarr009]

Is there any way to get around this issue? To make sure I properly understand the issue, is it currently not possible for something like this to work? var animmesh = new THREE.SkinnedMesh(model, material[0]);

where the material above is a ShaderMaterial.

Thanks,
David

[pehrlich]

Thats right, but such an API is not entirely infeasible.

I actually was able to make some progress with this. Pretty exciting, really, would love to finish up the work one day: leapmotion/leapjs-rigged-hand#11

[ddstarr009]

very cool...hmm, so i'm not sure how to proceed with my predicament. I purchased a max file that I'm trying to animate via ThreeJs. I converted it to JSON and apparently it has a ShaderMaterial. The model has bones so don't I have to at least use a skinned mesh? If so, is there some way I could convert the material in order for this to work? Here is the elephant model I purchased: http://protofactor.biz/PLAYER/ELEPHANT.html

Any ideas would be greatly appreciated. I'm very new to 3d modeling and new to ThreeJs as well.

Thanks,
David

[WaltzBinaire]

hey guys...what happened to this issue? im new to WebGL and so far all my previous shaders work nicely...Im trying to write custom materials for a skinnedmesh.. i do believe it is really important - to be able to access the animated mesh with further shaders...is there any other way to get shaders running on imported animations and models?

[arturitu]

I was trying to model new hand models to make them work on VR but when I tried to add a rim shader doesn't work as SkinnedMesh and it would be great to have that option.

Any idea where to start in order to find the solution?

[arturitu]

I tried to replace in this example https://github.com/mrdoob/three.js/blob/master/examples/webgl_animation_skinning_blending.html#L104 with something like this:

if ( object.isMesh) { console.log(object); object.material = new THREE.ShaderMaterial( { uniforms: { Edge: { type: 'f', value: 1.0 } }, vertexShader: [vertexShaderHere], fragmentShader: [fragmentShaderHere], skinning: true }) object.castShadow = true; } } );

Making sure to take the value of skinning = true but animations don't work.

@donmccurdy do you know if there are now with r112 a way to get around this issue?
Or maybe @brianpeiris or @robertlong they had to solve this on Hubs.

Thanks,

[donmccurdy]

@arturitu would you be able to share a demo? I'm not much good at debugging the compiled shaders unless I can enable SpectorJS for them. 😅

[arturitu]

@donmccurdy Here you have a demo > https://shader-skinning.glitch.me/ Is that ok for you? I think is not a problem with this specific shader. I guess it doesn't work with any shader.

[donmccurdy]

@arturitu ShaderMaterial provides the uniforms your shader needs for skinning, but none of the code... Starting from the example you've shared, the vertex shader could be modified as follows:

      varying vec3 N;
      varying vec3 I;

      ${THREE.ShaderChunk.skinning_pars_vertex}

      void main()
      {
          mat4 modelViewProjectionMatrix = projectionMatrix * modelViewMatrix;

          ${THREE.ShaderChunk.beginnormal_vertex}
          ${THREE.ShaderChunk.skinbase_vertex}
          ${THREE.ShaderChunk.skinnormal_vertex}

          vec3 transformed = vec3( position );

          ${THREE.ShaderChunk.skinning_vertex}

          //Transform vertex by modelview and projection matrices
          gl_Position = modelViewProjectionMatrix * vec4( transformed, 1.0 );

          // Normal transform (transposed model-view inverse)
          N = normalMatrix * objectNormal;

          // Incident vector
          I = vec3(modelViewMatrix * vec4(transformed, 1.0));

      }

That gets the vertex positions animating correctly, although I've got something still wrong with the normals in this demo... https://glitch.com/edit/#!/shader-skinning-patch?path=script.js:78:7

[arturitu]

@donmccurdy then I guess it is not a threejs issue, It's my mistake with the shader. Thanks, as always, for your precious help. I'll take a look at the normals of that model.

[arturitu]

@donmccurdy related to the normals (sorry if it is not related with the issue directly)... on the demo model I can't catch what is the problem with the normals, but with a high poly version of the hand that I have, it seems to have some issues with the normals. Here I'm using exported MeshStandardMaterial from Blender only adding it transparent = true and opacity = 0.5 >

And I can't understand why because I reviewed the normals and It seems to be ok >

And when the shader material fixed is applied, it appears these 'flipped' areas in when more than two transparent parts are overlapped >

I'm not sure if this is the problem with the normals that you figured out before.

[Mugen87]

Closing. Please use the forum if you need more help with custom shader enhancements.

For everyone else who is looking for a basic live example, this one shows a basic setup with the most important shader chunks: https://jsfiddle.net/h02s5n14/4/ (using three.js R114).