Multiple Lights Not Working Shadow Map Atlas - Browser / HTML5

[N8n5h]

Thank you for contributing to Armory!

Description
Shadow Map Atlas is not working with multiple lights when targeting HTML5. Under those conditions shadows will break when rendering

To Reproduce
Add multiple lights to the scene and some objects, select Runtime -> Browser, enable Shadow Map Atlasing under Armory Render Path -> Shadows, play.

Expected behavior

System
Blender: 2.83.10
Armory: latest commit
OS: Ubuntu 20.04.2 LTS
Graphics card: AMD RX 480 (mesa drivers)

[Naxela]

I've tried to get the shadows to work with the latest commit for both Armory3D and Iron (so far). I seem to have some continued problems with the shadows, although now I'm at least able to have multiple lights, so it's great progress! :) - Some of the following might help a bit on shedding some light on the shadow issue

I've tried to make a comparison between Eevee, Armory (Krom) and Browser - so on Eevee it looks like this:

On Krom it looks like this - Seems like it's inverted within the shadow cone itself:

And on Browser (AND Krom, if the following commit is not included (c64b475), which I presume is the inversion within the shadow-tile(?) itself):

I don't know if there's some additional Y-inversion that is still needed somewhere for Krom on Windows (and for Browser for that matter)?

My system is Windows with an Nvidia card if it's any help (although I think it's more a OpenGL-DirectX-WebGL related issue, in regard to this: #2102 (comment)).

Attached Blend file:

Lights-Object-Transition.zip

[N8n5h]

@Naxela could confirm too with your file, so it's clearly an oversight from my part on my tests :)... It seems I still can't fully wrap my mind over how it's supposed to convert from openGL to direct3D, so I'll probably just try stuff until something works.

I pushed a commit, can you test it out? https://github.com/N8n5h/iron/tree/light-fix-setfloats

[Naxela]

That's great! It seems to have fixed spotlights, at least on Krom:

Still has the same issue for Browser:

As for Browser, it seems that (on Windows at least) it needs to have the same things applied, not sure why since by using WebGL it should be the same as OpenGL and Linux - In any case, by having inverted Y in shadows.glsl and lights.glsl, as well as the d3d bias matrix in the Uniforms.hx, it works on browser too:

I don't know if this has something to do with Chrome and Firefox using Angle to transpile to DirectX or something?

[N8n5h]

As for Browser, it seems that (on Windows at least) it needs to have the same things applied, not sure why since by using WebGL it should be the same as OpenGL and Linux - In any case, by having inverted Y in shadows.glsl and lights.glsl, as well as the d3d bias matrix in the Uniforms.hx, it works on browser too:

Could confirm with Firefox on Linux... By manually doing the transformations calculations, Not sure why but if I'm not mistaken this is what's going on when doing the first inversion with the bias matrix and then the second inversion in the shader:

Krom openGL****

[Naxela]

That's good to know, I hadn't tested much with point lights, but I recalled having issues with point shadows even after the bias fix, although strangely when I'm testing now the issue seems to be gone when testing on Krom. Pointlights and spotlights seem to work just fine:

As for Sun and Area types of light, I suppose those are still work in progress? Not that area lights are that important at the moment, since I suppose they're probably the hardest of the bunch to implement due to LTC and PCSS and what not, but sun seems to be behaving strangely on Windows - Does it work fine on OpenGL?:

[N8n5h]

but sun seems to be behaving strangely on Windows - Does it work fine on OpenGL?

Yeah, sun probably needs the same treatment like this to work.

Edit after reading your reply again:

Not that area lights are that important at the moment, since I suppose they're probably the hardest of the bunch to implement due to LTC and PCSS and what not, but sun seems to be behaving strangely on Windows

I strove for supporting the basic light types... so area lights were out of my radar for now, but I suppose I could give it a try some time to see if I can get them to work.

[N8n5h]

The changes I pushed for armory and iron should solve the issue with sun, with browser (at least by circumventing it with the way you found with the "double inversion") and remove the need of the inversion of the shader to save some gpu cycles, please test it out and tell me if I can open a PR @Naxela 👍

Edit: I pushed another commit to iron to support sun with 1 cascade too.

[Naxela]

Hi, I tested for both point lights, spot lights and sun light, and it works great in general, fantastic work! 👍

Spotlights seems to work fine on both Krom and Browser ✔
Sunlights seem to work fine on both Krom and Browser (with cascade variations too) ✔

Spotlights seem to work fine on Krom (✔), currently point light shadows on browser seems to be the only issue left from what I could tell - I'm not sure if it's a Y-inversion thing too, or if there's some shadow clipping?

I hope this helps a little

Again, great work!

*On Windows Krom and Chrome

[N8n5h]

I'm not sure if it's a Y-inversion thing too, or if there's some shadow clipping?

I'm unable to reproduce the issue on my end with Firefox nor Chrome... can you share the test file?

[Naxela]

Hi,

I tested a bit more, and I'm not sure if it has something to do with the angle perhaps?

I've attached the example from the first one, where the light rotates at a height of about 3.7m from the plane:

Pointlight1.zip

The second example, where the lights are about 6m from the plane seems to be working fine (apart from the slight shadow overlay?):

Pointlight_2.zip

[N8n5h]

@Naxela, hopefully this should do it N8n5h@f834b4c.
Can you test again to see if it works in your end?

[Naxela]

@N8n5h

Hi, I tested it and it seems to work for Browser now, but it seems to break on Krom (Windows):

I managed to get it working by adding the direct3d11 for the flip y condition along with HTML5 (write_data.py):

https://gist.github.com/Naxela/361a35b3f604114c729ee756ba8db6fd#file-write_data-py-L537

As well as conditioning out HLSL in shadows.glsl:

https://gist.github.com/Naxela/e2544dbea31cc784ea143c8860e47c5a#file-shadows-glsl-L49

Now it works on both Browser and Krom for Windows - I hope it doesn't break anything on Linux?:

I think basic light types should work across all targets now, possibly?

[N8n5h]

Hi, I tested it and it seems to work for Browser now, but it seems to break on Krom (Windows):

Oops, yeah I forgot to check and add for DirectX too 😅 ...

I managed to get it working by adding the direct3d11 for the flip y condition along with HTML5 (write_data.py):

https://gist.github.com/Naxela/361a35b3f604114c729ee756ba8db6fd#file-write_data-py-L537

As well as conditioning out HLSL in shadows.glsl:

https://gist.github.com/Naxela/e2544dbea31cc784ea143c8860e47c5a#file-shadows-glsl-L49

Nice! If you don't mind I will copy what you did and push another commit and open a PR.

I hope it doesn't break anything on Linux?:

I've checked and HTML5 and Krom openGL works 👍

I think basic light types should work across all targets now, possibly?

Maybe... AFAIK It's still untested in MacOS with Metal, Safari (not sure if it's a supported target though), other non-chromium based browsers, Linux (C) (which I've tried but I get this #2067 even with the default cube), Windows (C), Android, and others that I've possibly forgotten.

[Naxela]

Nice! If you don't mind I will copy what you did and push another commit and open a PR.

That sounds like a good idea 👍

I hope it doesn't break anything on Linux?:
I've checked and HTML5 and Krom openGL works 👍
Maybe... AFAIK It's still untested in MacOS with Metal, Safari (not sure if it's a supported target though), other non-chromium based browsers, Linux (C) (which I've tried but I get this #2067 even with the default cube), Windows (C), Android, and others that I've possibly forgotten.

I think it's ok to consider Linux and Windows across Krom and HTML5 for now, if I'm not mistaken Android should work if it works on Linux too (if it's based on some OpenGL system)? - I think Linux for C should probably work fine as well (at least graphics wise, not sure with that error), not sure if Windows C/HL is D3D based and as for MacOS/iOS I unfortunately don't have any means to test that now (or in the foreseeable future for that matter), but I did see somewhere in the code where _InvY was being defined if HLSL or Metal was defined too, so it could be a problem there too?

In any case, shadows seem to work great across these platforms now:

• Linux (HTML5) ✔
• Linux (Krom) ✔
• Windows (HTML5) ✔
• Windows (Krom) ✔

Fantastic work with the shadows so far! 🔨

[onelsonic]

Sunlight shadows going bananas (html5)

Well, I notice a strange behavior with the current release.
When I move the camera/point of view my shadows are moving around and change positions depending on the camera.
When shadows should only move when the Sun light moves or the object moves....
While everything is working correctly with the previous build : ArmorySDK2103

[N8n5h]

Sunlight shadows going bananas (html5)

Well, I notice a strange behavior with the current release.
When I move the camera/point of view my shadows are moving around and change positions depending on the camera.
When shadows should only move when the Sun light moves or the object moves....
While everything is working correctly with the previous build : ArmorySDK2103

Yes, could confirm. Thank you for bringing this to my attention. This should fix it armory3d/iron#124

[onelsonic]

in HTML5 I got some serious shadows flickering when the camera pass in front of an object with a transparent texture (let's says a plane with tree leaves). the shadowed object in the background behind the object with the transparent texture will flicker with ita shadow appearing and disappearing .

[N8n5h]

in HTML5 in got some serious shadows flickering when the camera pass in front of an object with a transparent texture (let's says a plane with tree leaves). the shadowed object in the background behind the object with the transparent texture will flicker with is shadow appearing and disappearing .

Sorry for the late reply, is this issue reproducible with shadow map atlasing disabled?
Also, can you provide a minimal test project with the issue?

[onelsonic]

in HTML5 I got some serious shadows flickering when the camera pass in front of an object with a transparent texture (let's says a plane with tree leaves). the shadowed object in the background behind the object with the transparent texture will flicker with ita shadow appearing and disappearing .

Hi there, it seem the problem occurs as well when one single material is used by multiple objects in the scene.
I'll try to put an example together