Change uniforms on a per object basis to keep the materials count low

[arose]

I would like to propose a new feature in the WebGLRenderer that allows changing uniform values on a per-object basis. Currently an object-specific uniform value requires the use of a separate material for the object. To keep the materials count low uniforms could be changed in the rendering process. This is in fact already used for the object-specific matrices including the modelViewMatrix. However, to my knowledge this part of the rendering process is private and not exposed by any kind of api.

The use-cases I have for such a feature are the following. Generally I have multiple objects that share the same geometry and material but differ in their world position and orientation (i.e. the object matrix). Such instancing quite common but leaves all instances with the same material and hence the same uniform values. So far so good. Now I have a shader that requires the inverse of the modelViewMatrix as a uniform, which depends on the on the object matrix. Adding the inverse of the modelViewMatrix to a material and calculating as well as updating it works fine as long their is a material for every object instance. The second thing I would like to do is supporting picking specific object instances in the gpu-based object picking approach (i.e. each object gets a unique "picking" color). This can be implemented using a instance-specific "picking" color via a uniform that gets updated before each object instance is rendered and thus not requiring a separate material for each instance.

Having separate materials somewhat defeats the purpose of instancing -- albeit not as much as having separate geometries. Still, constructing 1000 materials for 1000 object instances can be quite noticeable, especially in the initialization phase. This is what I want to avoid and I hope I expressed it clearly.

EDIT: see my third comment for a better approach than what I wrote in the next paragraph.

Looking at the WebGLRenderer I think changing uniforms on a per object basis can be achieved by a hook for user-defined code in renderBuffer, renderBufferDirect and renderImmediateObject, just before setProgram is called and the uniform values are send to the gpu.

function objectUniformHook( object ){

    if( object.uniformHook ){

        object.uniformHook( material );

    }

}

Ok, these are my thoughts. I am curious what you guys think about it and if you find it useful. Or if there are better ways to achieve what want. Thanks.

[arose]

Have thought some more about the implementation of this. Apparently, just changing uniform values before setProgram is called is not sufficient as the material (shared between objects) does not change and the new uniform values are never loaded onto the gpu.

However, by introducing a needsRefresh material property like this

function setProgram( camera, lights, fog, material, object ) {

    // ...

    if( material.needsRefresh ){

        refreshMaterial = true;
        material.needsRefresh = false;

    }

    // ...

}

the following sub-classing of WebGLRenderer then lets me change a material's uniform values on a per-object basis

var MyWebGLRenderer = function(){

    THREE.WebGLRenderer.apply( this, arguments );

    function updateUniforms( object, camera ){

        object.material.needsRefresh = true;

        // custom code to change uniform values on a per-object basis

    }

    var renderBuffer = this.renderBuffer;

    this.renderBuffer = function( camera, lights, fog, material, geometryGroup, object ){

        updateUniforms( object, camera );

        renderBuffer(
            camera, lights, fog, material, geometryGroup, object
        );

    }

    var renderBufferDirect = this.renderBufferDirect;

    this.renderBufferDirect = function( camera, lights, fog, material, geometry, object ){

        updateUniforms( object, camera );

        renderBufferDirect(
            camera, lights, fog, material, geometry, object
        );

    }

    var renderImmediateObject = this.renderImmediateObject;

    this.renderImmediateObject = function( camera, lights, fog, material, object ){

        updateUniforms( object, camera );

        renderImmediateObject(
            camera, lights, fog, material, object
        );

    }

};

This is the least intrusive way, requiring only one change to the library, I could come up with so far.

[arose]

Another iteration, again by sub-classing WebGLRenderer. By making setProgram and loadUniformsGeneric public instance methods the following works:

var MyWebGLRenderer = function(){

    var _this = this;

    THREE.WebGLRenderer.apply( this, arguments );

    var setProgram = this.setProgram;

    this.setProgram = function( camera, lights, fog, material, object ){

        var program = setProgram( camera, lights, fog, material, object );

        // update object.material.uniforms here

        _this.loadUniformsGeneric( material.uniformsList );

        return program;

    }

}

I guess my feature request would now be to make setProgram and loadUniformsGeneric public instance methods of WebGLRenderer. I could prepare a PR.

[WestLangley]

Without passing judgement on your proposal, which I will leave to others, here are a few comments:

I have a shader that requires the inverse of the modelViewMatrix as a uniform

If your model is unscaled (which, of course, may not be your case), you do not need to pass in the inverse of the modelViewMatrix as a separate uniform, as the upper 3 x 3 will be orthogonal. See http://en.wikibooks.org/wiki/GLSL_Programming/Applying_Matrix_Transformations.

constructing 1000 materials for 1000 object instances can be quite noticeable

You can clone a single material, instead. In THREE.ShaderMaterial.prototype.clone(), note this line:

material.uniforms = THREE.UniformsUtils.clone( this.uniforms );

This should result in only one shader program used by all object instances.

Is that not sufficiently performant for your use case?

[arose]

Thanks @WestLangley for that insightful comment! I did not knew about that spacial case to get the inverse of such a matrix. It might come in handy when, as you said, no scaling of the model is involved. Still, to make the case for my suggestion, it only works for the modelViewMatrix uniform but not for the pickingColor uniform. Also, I need I need some more matrices I haven't mentioned yet, for example the modelViewProjectionMatrix in the fragment shader where I would rather supply it directly instead of calculating it. Also, there was a recent issue requesting the addition of the modelViewProjectionMatrix (#5974). By exposing the two function (setProgram and loadUniformsGeneric) from the WebGLRenderer users of the library have the option to gain greater control of the rendering of an object.

This should result in only one shader program used by all object instances.

I had to look into this one as I did not use material.clone but did more or less the same resulting in a single program. So, initMaterial in the WebGLRenderer very conveniently ensures that shader programs with the same source code are only created once and are then reused. I did not knew that, thanks for pointing me into that direction.

Is that not sufficiently performant for your use case?

My performance problem is not with the rendering phase but with the objects creation phase. In a particular instance object creation is ~300 ms (with a single material) vs ~1000 ms (with a separate material for each object instance).

[WestLangley]

My performance problem is not with the rendering phase but with the objects creation phase.

Yes, I know. I should have phrased my question differently. "Is using .clone() not sufficiently performant for your use case?"

[arose]

"Is using .clone() not sufficiently performant for your use case?"

I see. No it isn't. I now did some performance measurement with my application and with JsPerf. Note that I am on a different machine, so the numbers have changed.

My application, creating 10,080 object instances (note that this more then just material creation):

• ~500 ms (with a single material)
• ~1600 ms (with a separate material for each object instance)

JsPerf (http://jsperf.com/three-js-material-instancing):

• ~10,000 ops/s => ~0.1 ops/ms
• 10,080 instances => ~1008 ms

The numbers indicate that material creation/cloning is the bottleneck here. More specifically uniform creation/cloning. Not surprising, but always good to see real numbers. So, when you create a lot of materials it might help performance to use only the uniforms you need in those materials.

[WestLangley]

@arose Is this still an issue for you?

[arose]

It was solved with #7048 at the time. As for the current version of three, I can't say as I am not using it. Going to close it.