Some meshes to overlap glow

[PixelsCommander]

Imagine here the stroke behind chip stack is just absent as it would happen if stroke is overlapped by the stack. This is what I d like to achieve. Currently working on PR for this but would like to have some guidance/input.

My idea is to:

• API wise have additional flag shouldExcludedOverlap: boolean
• Solution wise to render excluded meshes to one more stencil buffer and substract it from blurred glow texture.

Any better ideas on this? @deltakosh @sebavan

[deltakosh]

Hey and thank you for your interest :)
How can you have more than one stencil buffer?

[sebavan]

Api wise, I am all good with the flag.

Solution wise I am like @deltakosh curious about your solution :-) I guess the easiest to discuss it would be to create the PR and then we could help adapting it ?

[PixelsCommander]

Oh. I am still bad in buffers. Lets name it a texture which we substract from blurred one.

[deltakosh]

ok so we are on the same page :D
let's do a PR please

[deltakosh]

Up? I guess rewrite the mesh you want in the rendertarget before applying the glow should do the trick

[PixelsCommander]

TBH I tried to play around but so far cant make it work with both innerglow / outerglow.

Rewrite? Could you elaborate here please?

[deltakosh]

Just a thought but you should be able to render the object you want to overlap directly (as black objects) into the rendertarget texture just before applying the glow in the main scene (in order the remove the glow where they are)

[PixelsCommander]

@deltakosh this works for outer glow but not for inner glow. These are shots from Spector:

Outline layer drawn correctly:

Outer glow drawn correctly then inner glow which spoils the picture:

Also I observe some problems with depth buffer. So if I move camera a bit then it looks like:

The code added is:

Object.keys(this._excludedMeshes).forEach(id => {
      this._renderPhase = RenderPhase.CleaningExcluded;

      var mesh = this._excludedMeshes[id].mesh;
      var subMeshes = mesh.subMeshes;
      var engine = this._engine;
      var previousStencilFunction = engine.getStencilFunction();
      var previousStencilOperationPass = engine.getStencilOperationPass();
      var previousStencilOperationFail = engine.getStencilOperationFail();
      var previousStencilOperationDepthFail = engine.getStencilOperationDepthFail();
      engine.setStencilFunction(Engine.ALWAYS);
      engine.setStencilOperationPass(Engine.ZERO);
      engine.setStencilOperationFail(Engine.ZERO);
      engine.setStencilOperationDepthFail(Engine.ZERO);
      subMeshes.forEach(this._renderSubMesh.bind(this));
      engine.setStencilFunction(previousStencilFunction);
      engine.setStencilOperationPass(previousStencilOperationPass);
      engine.setStencilOperationFail(previousStencilOperationFail);
      engine.setStencilOperationDepthFail(previousStencilOperationDepthFail);
      this._renderPhase = RenderPhase.InnerGlow;
   });

Any suggestions?

[sebavan]

Another idea would be to render the chip after the first renderLayer to hide to border.

You could rely on a second renderGroup without cleaning the depth info ?

Let me try to do a quick PG

[sebavan]

@PixelsCommander looking more in detail in the issue, the best way to fix it would be to allow effect layer to work per rendering group.

Thinking even deeper there are a couple of other nice changes we could do in this space.

If you are not in a rush I am planing to deploy a new version by end of week. Would that fit to you ?

[sebavan]

Closed and reopen due to finger management issue ;-)

[PixelsCommander]

Yeah, sure.

[sebavan]

It happens than the changes I am doing to componentize the effectlayers are getting a bit bigger but will help componentizing other modules.

I am expecting to have this for end of next week. Basically, you would be able to associate the effect layers with a rendering group ensuring that from one group to the other you could render on top of the effect.

[PixelsCommander]

Sounds great. Looking forward for it! 2018-06-23 0:35 GMT+02:00 sebavan <notifications@github.com>:

…

It happens than the changes I am doing to componentize the effectlayers are getting a bit bigger but will help componentizing other modules. I am expecting to have this for end of next week. Basically, you would be able to associate the effect layers with a rendering group ensuring that from one group to the other you could render on top of the effect. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4463 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAxer17_9s0FU5eEgVPLTFdf7P9Asw6oks5t_XEwgaJpZM4UeLoI> .

[sebavan]

Following the PR above, you ll be able to achieve this through the use of rendering groups:
https://playground.babylonjs.com/#KTR4V2

Basically you can attach an effect layer to a chosen rendering group and push the meshes that needs to overlap in a subsequent group.

Also, please turn off the auto clear for the second group or those meshes will draw on anything present in the first one as the depth buffer would be clear.

I close the issue for now and we can open it back if needed.

[PixelsCommander]

@sebavan Glad to see there is a progress on what you were busy with.

Is this how it intended to be rendered?

Then it is not what I was trying to achieve. Green one should overlap glow under it, not having a glow around.

[sebavan]

Nope but the nightly has not been deployed yet :-) It will after deploy

[deltakosh]

Should be good now :)
https://playground.babylonjs.com/#KTR4V2#1

[PixelsCommander]

Confirm, works as expected. Thanks a lot!!

[PixelsCommander]

after applying the solution we got chips to be rendered on top of the scene so had to roll it back. @sebavan @deltakosh any ideas on how to prevent this keeping renderingGroup solution?

[sebavan]

Hello,

This is the function doing the trick:

scene.setRenderingAutoClearDepthStencil(1, false, false, false);

This prevent the depth buffer to be cleaned before rendering the group 1. You should replace 1 by the group id of your chips.

[PixelsCommander]

Once we set rendering groups on highlight layer and chips shadows are lost on all objects which do not have renderingGroup set.

I tried to change group for the rest but this is just guessing without knowing how it works under the hood in details.

@sebavan , @deltakosh guys, any ideas?

[deltakosh]

Rendering groups can be seen as layers. It is a good way to have different passes.

Can you repro in the playground a simple setup that looks like your game and we can help you define the correct rendering groups

[PixelsCommander]

Huh, now we are confused. Not able to reproduce this in the playground. And no clue what it is so far, investigating.

http://playground.babylonjs.com/#6T9SI9

[deltakosh]

This is a good news if you think about it ;)

[PixelsCommander]

Here is a playground http://playground.babylonjs.com/#6T9SI9#1

Bug(?) is caused by generator.usePercentageCloserFiltering = true;

The similar one is reproducible in a demo here http://www.babylonjs-playground.com/#R1EVD0#3 but this time with another property of shadowGenerator.useBlurExponentialShadowMap = true; and no rendering groups. Basically, the demo is broken currently.

[sebavan]

I have moved the conversation to a new issue as the issue is not related to the EffectLayer.