Add support for shadow map atlasing #2102
Conversation
With this it is now possible to enable atlasing of shadow maps, which solves the existing limitation of 4 lights in a scene. This is done by grouping the rendering of shadow maps, that currently are drawn into their own images for each light, into one or several big textures. This was done because the openGL and webGL version Armory targets do not support dynamic indexing of shadowMapSamplers, meaning that the index that access an array of shadow maps has to be know by the compiler before hand so it can be unrolled into if/else branching. By instead simply using a big shadow map texture and moving the dynamic part to other types of array that are allowed dynamic indexing like vec4 and mat4, this limitation was solved. The premise was simple enough for the shader part, but for the Haxe part, managing and solving where lights shadow maps should go in a shadow map can be tricky. So to keep track and solve this, ShadowMapAtlas and ShadowMapTile were created. These classes have the minimally required logic to solve the basic features needed for this problem: defining some kind of abstraction to prevent overlapping of shadowmaps, finding available space, assigning such space efficiently, locking and freeing this space, etc. This functionality it is used by drawShadowMapAtlas(), which is a modified version of drawShadowMap(). Shadow map atlases are represented with perfectly balanced 4-ary trees, where each tree of the previous definition represents a "tile" or slice that results from dividing a square that represents the image into 4 slices or sub-images. The root of this "tile" it's a reference to the tile-slice, and this tile is divided in 4 slices, and the process is repeated depth-times. If depth is 1, slices are kept at just the initial 4 tiles of max size, which is the default size of the shadow map. #arm_shadowmap_atlas_lod allows controlling if code to support more depth levels is added or not when compiling. the tiles that populate atlases tile trees are simply a data structure that contains a reference to the light they are linked to, inner subtiles in case LOD is enabled, coordinates to where this tile starts in the atlas that go from 0 to Shadow Map Size, and a reference to a linked tile for LOD. This simple definition allows tiles having a theoretically small memory footprint, but in turn this simplicity might make some functionality that might be responsibility of tiles (for example knowing if they are overlapping) a responsibility of the ones that utilizes tiles instead. This decision may complicate maintenance so it is to be revised in future iterations of this feature.
|
This is absolutely epic! Must have been a ton of work...
Tested it and it "solves" #1772 when the new atlasing option is enabled. But, unfortunately point lights indeed seem to be problematic: This is how the scene looks in Blender:
This is how it looks in Krom on Windows (there is no visible difference when using shadow LOD or a single atlas map):
And with spot lights:
Very similar artifacts are visible in the browser (Firefox) when running with html5. Test file and RenderDoc captures (exported to two formats because I don't know if sharing a capture works): lights.zip. I hope it helps a bit :) |
Ok thanks, yeah I was afraid that directx would be problematic. I have to reinstall windows and play with it to sort it out. |
|
I don't know about rendering, but the "artifact" in the second gif reminds me a similar bug that happens to sun lamps. Looks like some wrong calculation of the camera clip start, but i am not stating nothing. It occurs on OpenGL (not tested in DirectX), here is an example file: test.zip Sun lamp (sorry for the brightness): The same bug does not occur with other types of lamps, see point lamp: Maybe it is not a bug of the atlas map but of Armory? |
If the issue can be reproduced with the |
|
Ok, it took a full evening but I managed to reinstall windows and be able to test on it 😄. The first thing I've tried is just one point light in the scene and surprise, surprise, it works... so it's the same thing that happens in HTML5. Now I've to guess and find whatever is wrong with the implementation that breaks multiple lights for not-opengl targets 😅 |
|
After some poking around, I figure that the issue might have to do with this...
So I suppose it works with one light because of the coincidence that the face was just in the perfect spot when mirrored |
See http://thedev-log.blogspot.com/2012/07/texture-coordinates-tutorial-opengl-and.html, the "opengl coordinates" where inverted for proper support of direct3d texture coordinate system.
|
Pushed a commit that seems to solve the issue with spot and point lights on Windows from my brief testing. Please test if you can to confirm if it's finally working :) |
|
Works like a charm now, thank you so much! Regarding html5: if I'm not mistaken, webgl uses a screen coordinate system in the range [-1, 1] with (0, 0) being the center. Maybe that's what causing the issues there? That would explain why the artifacts look similar but different. Edit: the issue when the camera is too close to an object still persists unfortunately (can be seen in the gif above when the camera is close above the plane), is that due to a wrong setting somewhere or is it a bug? |
Will check it out!
Yeah, I noticed it when testing it too, but it seems it's not related to atlas from what I've seen. Can you confirm this too? |
I think it's just the screen/canvas coords, texture coords should be in the usual [0, 1] range.
Oh, indeed. Maybe there is something wrong with the clustering algorithm in general? |
Yes, I tested it by simply commenting this line https://github.com/N8n5h/iron/blob/91728e210b0703822ce42c38b3a79840a9452b72/Sources/iron/object/LightObject.hx#L550 which I think should have been enough to make spot lights work, but it didn't :(
Could be, but also could be the algorithm that computes lighting for spot lights too. I will try playing around with the shaders to see if clusters have anything to do with it, but getting a file where the issue can be reliably reproduced so it's easy to quickly test for changes probably is a better priority. Also I've noticed when using windows that renderdoc allows debugging direct3d, so that probably could be worth looking too, to compare what was going on on a broken pixel versus a working one... |
|
Hi, I found a small issue with the new atlasing option: when there is a transparent material (like ArmoryPBR with opacity < 1), the following glsl compiler error is raised: When |
This should solve it #2109. Didn't spot that |
|
Thanks! |
With this it will now possible to enable atlasing of shadow maps, which solves the existing limitation of 4 lights in a scene: #1289
This is done by grouping the rendering of shadow maps, that currently are drawn into their own images for each light, into one or several big textures. This was done because the openGL and webGL version Armory targets do not support dynamic indexing of shadowMapSamplers, meaning that the index that access an array of shadow maps has to be know by the compiler before hand so it can be unrolled into if/else branching. By instead simply using a big shadow map texture and moving the dynamic part to other types of array that are allowed dynamic indexing like vec4 and mat4, this limitation was solved.
The premise was simple enough for the shader part, but for the Haxe part, managing and solving where lights shadow maps should go in a shadow map can be tricky. So to keep track and solve this, ShadowMapAtlas and ShadowMapTile were created. These classes have the minimally required logic to solve the basic features needed for this problem: defining some kind of abstraction to prevent overlapping of shadowmaps, finding available space, assigning such space efficiently, locking and freeing this space, etc. This functionality it is used by drawShadowMapAtlas(), which is a modified version of drawShadowMap().
More details about the implementation (WIP): https://github.com/N8n5h/armory/wiki/Shadow-map-atlas-for-maintainers
How to use: https://github.com/N8n5h/armory/wiki/Shadow-map-atlas-for-users
Blender files I've used for testing:
files-for-testing.tar.gz
Notes:
As explained in the detailed article for maintainers, this is completely governed by the
#arm_shadow_map_atlascompiler flag to isolate issues related to it within it until it's mature enough, so it should be relatively safe to push even if it wasn't thoroughly tested as I initially planned... but not up to me to decide.Please test it if you can.
Tried to test it on Krom C but I get this when I try to launch even the default cube [Android (C)] custom material_shaders (not working) #2067
I tried to test it on windows but unfortunately my windows installation is acting up so couldn't confirm if it works there,
I have the suspicion that point lights might bring problems because of the inverted faces, but since the mapping is done manually all platforms should have the same "cubemap standard" so maybe it's safe. Also I suspect this Only one pointlight at a time possible #1772 might bring problems too but maybe the issue it's circumvented by the fact that no array of shadow map samplers is used for atlases.
HTML5 seems to not work with multiple point lights for whatever reason. One light works but more than one breaks shadows. Don't seem to be connected to the shadow map size from my tests but could be wrong. I've tried to debug it but unfortunately couldn't make Spector nor webgl-inspector to work, so not sure what is going on in the shader side.
Tried to implement dual paraboloid but unfortunately I got stuck and couldn't make it work, here is my progress in case anyone want to check it out armory / iron. There is no doc for this but it should be pretty clear since it's an (attempt of) implementation of the things suggested by the articles and gamedev post linked in the commit message.
This depends on this from iron armory3d/iron#111