Shadow mapping render pass

[0xLiso]

This PR implements shadow mapping with PSM (Perspective Shadow Mapping), PCF filtering and "colored shadows".
The implementation it's originally based in the XEffects shadow mapping examples.

How this works:

1. To be able to make it work with Irrlicht(MT) the Nodes must be added (and removed ) to a Shadow caster list to cast shadows.
2. The draw function in RenderingCore updated to do the depth pass.
3. Render all the nodes that casts shadow in a RTT.
   3.1. Render ClientMap depth every 0.2 seconds to avoid excessive drawcalls.
   3.2. Render depth for all dynamic entities
   3.3. Mix both depth textures in the Shadow Texture.
4. execute standard DrawAll, with the Shadow Texture attached.

The textures used are (3x):
irr::video::ECOLOR_FORMAT::ECF_R32F for depth
irr::video::ECOLOR_FORMAT::ECF_G32R32F for depth+color(encoded)

Both in 16bits if configured, but the tests I did, anything below 32bits causes lot's of precision problems.

To do

This PR is Ready for Review.

• Calculate sun POV frustum
• nodes added and removed from shadow caster list
• Render depth pass
• Write node and objects shaders
• Vanish shadow on artificial light.
• "Colored shadows"
• Different quality configurations.
• PCF & Poisson disk filtering.
• Tweak PSM distortion function for every quality.
• Wield items shadows.
• Moon shadows.

Future improvements.

• Check if mesh generation is able to remove the shadows encoded in the map

• Depth shaders using standard MT shader.cpp style (I wasn't able to make it work), so we can make things like shadows for waving plants and leaves.

• Reduce drawcalls.

How to test

Go to settings tabs, enable shaders, select quality from Dynamic Shadows dropdown.

The default values are:
Ultra low:
-shadow_map_max_distance: 80 blocks
-shadow_map_texture_size: 512
-shadow_map_color: false
-filter level: 0

Low:
-shadow_map_max_distance: 120 blocks
-shadow_map_texture_size: 1024
-shadow_map_color: false
-filter level: 1

Medium:
-shadow_map_max_distance: 350 blocks
-shadow_map_texture_size: 2048
-shadow_map_color: false
-filter level: 1

High:
-shadow_map_max_distance: 350 blocks
-shadow_map_texture_size: 2048
-shadow_map_color: true
-filter level: 2

Ultra High:
-shadow_map_max_distance: 450 blocks
-shadow_map_texture_size: 4096
-shadow_map_color: true
-filter level: 2

The values Ultra Low and Low only work when the map doesn't have too big elements, and even with normal maps it could have some artifacts.

This PR is high demanding because the amount of drawcalls so it's only recommended for slightly modern hardware.

The configuration can be changed within the "All Settings" section -> shadow

Examples:

Configuration:

Examples:

[ExeVirus]

Tested with high performance Win10 64 bit CPU, extremely stable. Tested also with low end windows laptops (2), got mediocre performance and after turning quality down, was not that compelling. It should be off by default for new players in my opinion therefore in my opinion.

[kilbith]

Bringing @lhofhansl @numberZero @hecktest @sofar in

[NathanSalapat]

Seems to work well on my system. Putting shadows up to Ultra high did drop FPS to about 30, I usually get about 60. At Medium settings I didn't see any FPS drop, and the shadows still looked quite nice.

[TSafa-23]

Works well, little visual artifacts, works well on even slightly older machines. On certain maps and settings it will even run close in performance to default.

[nerzhul]

that sounds very promising. How it work in caved with torchs ?

[sfan5]

This looks like a problem.

The mapgen runs on the server, it must not change its results based on purely visual settings set on the client (never mind the fact that this can't work with multiple clients).

[0xLiso]

Ok, we can remove it without a problem. It's just we'll get double shadows (ShadowMap and propagated one) in remote servers and in local ones.

[0xLiso]

Done.

[sfan5]

Shouldn't mesh generation be able to remove the shadows encoded in the map?

[0xLiso]

I don't know. I'm going to take a look on that.

[0xLiso]

Added to future improvements.

[0xLiso]

Hi again, I tried to fix this, but as far as I know the static shadow can't be removed in client or at least I don't know how to do it.

[0xLiso]

that sounds very promising. How it work in caved with torchs ?

using the value "nightRatio" stored at vertex alpha channel that give the ratio between sunlight and artificial light.

This is also used to dilute shadows with torch even at day.

You can see that here. The green is sunlight to 1, and red is all artificial light

and same with torches:

[nerzhul]

very very nice feature PR, i added first recommandations/fixes

[nerzhul]

i'm working on removing the global Renderer singleton. Can you make this function non static & call it properly from other objects ?

[0xLiso]

I'm been working on this this and I think it's not possible without changing current MT shader implementation. We have to have access shadowRenderer data from MainShaderConstantSetter and the only way to do it is by the singleton,

[sfan5]

I gave this a very quick look and the first advice I can give is this:

• Read https://dev.minetest.net/Code_style_guidelines
  Our clang-format config isn't quite up-to-date with it but does a good job so please run it on the files that had significant changes.
• Don't include irrlicht.h directly, use irrlichttypes.h
  • that header will also provide f32, v3f + other types and set using namespace irr; for you
• class variables should not start with _

[0xLiso]

IDK why Lua_lint is giving error. The unused variable "ddhandled" is actually used, and it was in there before.

[LoneWolfHT]

Great to finally see a PR!
Would you be able to collect a https://www.bountysource.com/ bounty on this once this is merged?

Did some quick messing around:

• Weird stairs bug: Full Building:

CobbleTower.we.zip (Building I used to reproduce bug, saved with WorldEdit)

PS: Will there be/Is there a way to prevent nodes you register from casting shadows?

[0xLiso]

Great to finally see a PR!
Would you be able to collect a https://www.bountysource.com/ bounty on this once this is merged?

Thank you, I don't even know what is that XD.

Did some quick messing around:

* Weird stairs bug: ![image](https://user-images.githubusercontent.com/28972859/117208550-1b5c2000-adaa-11eb-8601-6bb5ba786dd0.png) Full Building: ![image](https://user-images.githubusercontent.com/28972859/117209137-cff64180-adaa-11eb-919c-a325d5de86ac.png)

CobbleTower.we.zip (Building I used to reproduce bug, saved with WorldEdit)

I just test it and for some reason the cobblestone stairs are slightly darker than cobblestone, even without shadows:

Without shadows:

With shadows:

PS: Will there be/Is there a way to prevent nodes you register from casting shadows?

No right now, but we can add it if it's desired.

[LoneWolfHT]

It looks like you don't re-enable default shadows when the dynamic ones are disabled

[0xLiso]

It looks like you don't re-enable default shadows when the dynamic ones are disabled

I'll take a deep look on this.

But the default shadows are not disabled. I have to find how to disable them in other part:

#11244 (comment)

[lhofhansl]

This is really awesome. Glad to see MT graphics improving. Tried it works well! Thanks for working on this!

I did notice some problems with the direction of the sunlight:

Note that even though the sun is behind the back of the player is dark and the grass became black too.
(See shadow strength to 1 to see effect better)

[Andrey2470T]

Nice! At Medium, it works enough well (FPS ranges from 40 - 50), except dense rainforests where it drops almost half. Making at High, FPS ~ 30 in plains, from 20 - 25 in forests, however, it may become even much less. At Ultra High, it ranges from 1 -5 FPS. I have GeForce GTX 1050 video card, maybe it is weak or better optimisation is needed.

Also, I see some artifacts remain still that I discovered in earlier testings like the ripple and the shadow sharpening.

Also, is artificial light emitted by torches, mese posts and etc to affect casting shadows from objects not in your plans?

[0xLiso]

This is really awesome. Glad to see MT graphics improving. Tried it works well! Thanks for working on this!

I did notice some problems with the direction of the sunlight:

Note that even though the sun is behind the back of the player is dark and the grass became black too.
(See shadow strength to 1 to see effect better)

Thank you for your feedback, the problem is that billboards don't have normals, so the shader set them to shadowed. It's fixed now.

Nice! At Medium, it works enough well (FPS ranges from 40 - 50), except dense rainforests where it drops almost half. Making at High, FPS ~ 30 in plains, from 20 - 25 in forests, however, it may become even much less. At Ultra High, it ranges from 1 -5 FPS. I have GeForce GTX 1050 video card, maybe it is weak or better optimisation is needed.

Also, I see some artifacts remain still that I discovered in earlier testings like the ripple and the shadow sharpening.

Don't worry about your GPU, the performance is limited by the number of drawcalls, and this is a CPU bottleneck. To draw the shadows for the huge elements we have in MT, we need to render from a high distance, and that generates too many drawcalls. It can be solved by changing the way MT renders, making a much better occlusion algorithm o reducing the shadow render distance that can be done in "All Settings" -> Dynamic Shadows -> Shadow map max distance. But this will generate more artifacts .

Also, is artificial light emitted by torches, mese posts and etc to affect casting shadows from objects not in your plans?

If you are talking about removing shadows when there is an artificial light, yes. If you are talking about casting shadows from an artificial light, no. To do that, we should change the renderer to do dynamic lighting and multipass render.

Here you can see the shadow disappears when is inside the torch light.

[0xLiso]

I gave this a very quick look and the first advice I can give is this:

• Read https://dev.minetest.net/Code_style_guidelines
  Our clang-format config isn't quite up-to-date with it but does a good job so please run it on the files that had significant changes.

• Don't include irrlicht.h directly, use irrlichttypes.h

  • that header will also provide f32, v3f + other types and set using namespace irr; for you

• class variables should not start with _

Suggestions added.

[lhofhansl]

Thank you for your feedback, the problem is that billboards don't have normals, so the shader set them to shadowed. It's fixed now.

Nice. Yep. That part is fixed.
There is still something wrong with the player normals, though. In the photo above also notice that the back of the player if black - even though the sun is coming directly from behind.

[lhofhansl]

There're also some artifacts:

See the round'ish bright dark effects.

Again, thanks for working on this! Just trying to find bugs.

Edit: Filtering set to 2, texture size 2048.

[0xLiso]

There're also some artifacts:

See the round'ish bright dark effects.

Again, thanks for working on this! Just trying to find bugs.

Edit: Filtering set to 2, texture size 2048.

I think is fixed now.

[lhofhansl]

Did a pass, nothing stood out as wrong.
We can probably tweak the pre-sets a bit more and get rid of the 16 bit option (way too many artifacts), but that can be done in followup if needed.

[nerzhul]

there is remaining changes, i think we are fine now. Time to bench it, but honestly i think if we have major performance issue it's clearly not especially this code, but the client code design

[lhofhansl]

honestly i think if we have major performance issue

@nerzhul What specifically? You mean the number of draw-calls? Or other general performance issues.

[nerzhul]

@lhofhansl yep general performance issue when enabling this.

[VanessaE]

Here are my thoughts on this PR. Updated today, using your repo directly rather than applying the PR over upstream. All screenshots were taken with the in-game time locked at 08:00. This is Debian Bullseye with XFCE 4.18, on a Phenom II X6 1055T processor at 2.8 GHz, with 16 GB DDR1333 RAM and an R9 280X GPU (using the amdgpu driver supplied by Debian). Maybe not the most modern system in the world, but definitely not weak either.

• There's still a bit of a "curve" to the leading and trailing (i.e. east and west) edges of the shadows, while side edges (north and south) look mostly fine:
  

• In some nodes, a transparent texture can get discolored. For example, here are basic_signs:sign_wall_glass and basic_signs:sign_wall_obsidian_glass. The shadow for the former has pink artifacts, while the latter falls over entirely. The clear one uses an indexed grayscale image, while the obsidian one is RGB mode, both with only simple on-or-off transparency in use (in the image above, you can also see the yard version of the clear one):
  

• I've mentioned this in chat previously: colored shadows work mostly okay for nodes where the color is baked into the texture (the above notwithstanding), but not on nodes which use param2 coloring, such as doyousketch2's stained_glass mod (https://github.com/minetest-mods/stained_glass):
  

• I might also suggest reducing the shadow intensity in the presets. Easily changed in the "All settings" menu, of course, but in my opinion, 0.5 makes the vertical, shaded-side of nodes and such too dark, as if there's very little scattered, ambient light. 0.35 looks better, but then for some folks, this might cause shadows cast onto horizontal surfaces i.e. flat ground to not be dark enough. Maybe a separate setting is needed for vertical surfaces?

  This is the shaded side of some signs, with the intensity set to 0.35:
  

  And on the sun-facing side:
  

• The algorithm that blurs a shadow according to its distance from the node seems to have an odd "pixelating" glitch when it comes to transparent nodes. It's a little better with the shadow filter quality set to 2, but at 1 it's... grainy. I haven't tested in any real detail but I assume the filter radius also plays into this.
  For example, here's some minetest_game default:glass with the filter quality set to 1:
  

  And with the filter quality set to 2:
  

• I am still seeing a thin "gap" between nodes and shadows they cast, even on simple stuff like default:sandstone:
  

• While overall FPS is comparable to mainline without shadows, dtime jitter is still pretty high.
  Depending on how much I'm moving around, looking around, how much stuff is visible, how far out my view range is set for, etc...... with shadows enabled, it rids at close to 100% at best, and exceeds 175% at times. Without shadows, jitter varies between 20 and 80%. This high jitter leads to a "jumpy" feel, especially when traveling in a straight line at a constant speed (with the landscape already loaded). These are my settings:
  

[0xLiso]

Here are my thoughts on this PR. Updated today, using your repo directly rather than applying the PR over upstream. All screenshots were taken with the in-game time locked at 08:00. This is Debian Bullseye with XFCE 4.18, on a Phenom II X6 1055T processor at 2.8 GHz, with 16 GB DDR1333 RAM and an R9 280X GPU (using the amdgpu driver supplied by Debian). Maybe not the most modern system in the world, but definitely not weak either.

* There's still a bit of a "curve" to the leading and trailing (i.e. east and west) edges of the shadows, while side edges (north and south) look mostly fine:

Yes to fix the curve we need to tweak the distance, but at the same time if we reduce the distance more glitches will appear from distant shadows (blocks disappearing mostly)

* In some nodes, a transparent texture can get discolored.  For example, here are `basic_signs:sign_wall_glass` and `basic_signs:sign_wall_obsidian_glass`.   The shadow for the former has pink artifacts, while the latter falls over entirely.  The clear one uses an indexed grayscale image, while the obsidian one is RGB mode, both with only simple on-or-off transparency in use (in the image above, you can also see the yard version of the clear one):

uhmmm I'll take a look on that. Tnx

* I've mentioned this in chat previously: colored shadows work mostly okay for nodes where the color is baked into the texture (the above notwithstanding), but not on nodes which use param2 coloring, such as doyousketch2's stained_glass mod (https://github.com/minetest-mods/stained_glass):

Yes, I think this is because those nodes are been render as normal blocks not as transparent ones. I think.

* I might also suggest reducing the shadow intensity in the presets.   Easily changed in the "All settings" menu, of course, but in my opinion, 0.5 makes the vertical, shaded-side of nodes and such too dark, as if there's very little scattered, ambient light.  0.35 looks better, but then for some folks, this might cause shadows cast onto horizontal surfaces i.e. flat ground to not be dark enough.  Maybe a separate setting is needed for vertical surfaces?
  This is the shaded side of some signs, with the intensity set to 0.35:
  ![image](https://user-images.githubusercontent.com/1733537/120869251-04088200-c564-11eb-9e42-4a9432cff1bc.png)
  And on the sun-facing side:

Uhm.. I think you have some settings from a previous install, the current default values are:

// Effects Shadows
settings->setDefault("enable_dynamic_shadows", "false");
settings->setDefault("shadow_strength", "0.2");
settings->setDefault("shadow_map_max_distance", "200.0");
settings->setDefault("shadow_map_texture_size", "2048");
settings->setDefault("shadow_map_texture_32bit", "true");
settings->setDefault("shadow_map_color", "false");
settings->setDefault("shadow_filters", "1");
settings->setDefault("shadow_poisson_filter", "true");
settings->setDefault("shadow_update_time", "0.2");
settings->setDefault("shadow_soft_radius", "1.0");
settings->setDefault("shadow_sky_body_orbit_tilt", "0.0");

* The algorithm that blurs a shadow according to its distance from the node seems to have an odd "pixelating" glitch when it comes to transparent nodes.  It's a little better with the shadow filter quality set to 2, but at 1 it's... grainy.  I haven't tested in any real detail but I assume the filter radius also plays into this.

I can't help with this, @x2048 '??

* While overall FPS is comparable to mainline without shadows, dtime jitter is still pretty high.
  Depending on how much I'm moving around, looking around, how much stuff is visible, how far out my view range is set for, etc...... with shadows enabled, it rids at close to 100% at best, and exceeds 175% at times.  Without shadows, jitter varies between 20 and 80%.  This high jitter leads to a "jumpy" feel, especially when traveling in a straight line at a constant speed (with the landscape already loaded).  These are my settings:

In my linux machine it about 3-14%, but I'm not sure what value is about :)

And @VanessaE , thank you very much for your review :)

Also, thank you @sfan5 for your fixes.

[VanessaE]

Yes, I think this is because those nodes are been render as normal blocks not as transparent ones. I think.

The registration looks pretty normal to me:

minetest.register_node("stained_glass:stained_glass", {
	description = "Stained Glass",
	drawtype = "glasslike_framed_optional",
	tiles = { "stained_glass.png", "stained_glass_detail.png" },
	paramtype = "light",
	paramtype2 = "color",
	palette = "unifieddyes_palette_extended.png",
	sunlight_propagates = true,
	use_texture_alpha = "blend",
	light_source = myglow,
	is_ground_content = true,
	walkable = true,
	groups = {snappy=2,cracky=3,oddly_breakable_by_hand=3, not_in_creative_inventory=1, ud_param2_colorable = 1},
	sounds = default.node_sound_glass_defaults(),
	on_construct = unifieddyes.on_construct,
	on_dig = unifieddyes.on_dig,
})

Uhm.. I think you have some settings from a previous install, the current default values are:

...something that doesn't apply in my case, because I changed them to reasonable values (as pictured at the bottom of my last comment) before commenting about the shadow darkness. 😃 Plus, in my previous tests, I had them set to 0.4.

In my linux machine it about 3-14%, but I'm not sure what value is about :)

It means how much drawtime difference there is from one frame to the next. If a frame takes say 20ms to draw, then the next one takes 40ms, then 20ms for the next one after, that's a jitter of 100%. At least that's how I interpret the number. Either way, it's much worse with shadows turned on.

[LoneWolfHT]

I get a segfault when leaving a world (exit to menu)

[x2048]

* The algorithm that blurs a shadow according to its distance from the node seems to have an odd "pixelating" glitch when it comes to transparent nodes.  It's a little better with the shadow filter quality set to 2, but at 1 it's... grainy.  I haven't tested in any real detail but I assume the filter radius also plays into this.

I can't help with this, @x2048 '??

Yes, I'll take a look. @VanessaE, thank you for a good example to test :)

• I've mentioned this in chat previously: colored shadows work mostly okay for nodes where the color is baked into the texture (the above notwithstanding), but not on nodes which use param2 coloring, such as doyousketch2's stained_glass mod (https://github.com/minetest-mods/stained_glass).

These are most likely colored with glVertexColor and need to be accounted for when we map the colored shadow. Grep for inVertexColor in the shaders to see how it's used.

[sfan5]

Any further improvements or needed fixes can be made in a future PR.

@0xLiso found another bug, with WorldEdit's region cube the shadow renders in some random spot:

@LoneWolfHT Also I tested and couldn't reproduce your crash.

[AbduSharif]

Congratulations on the merge guys.

[0xLiso]

WoW PR merged. :) Thank you all for your help. @sfan5 I'll take a look on that.

[Jipok]

Is this normal?

screenshot

Medium level of shadows. GPU: NVIDIA GeForce GTX 1050 Ti Mobile Driver: nvidia-460

[0xLiso]

Is this normal?
screenshot
Medium level of shadows. GPU: NVIDIA GeForce GTX 1050 Ti Mobile Driver: nvidia-460

Yes, that's because the shadow has exactly the same angle that the blocks and creates that kind of glitch.

2 possible solutions are, change shadow tilt by something between 15.0 and 30.0 in the advanced settings, or if you don't want hard shadows, you can increase the soft shadow range by something like 5.0

[lhofhansl]

Yeah!

Let's do further improvements in individual PRs.
(And if someone wanted to tackle reflective water I'd be quite supportive of it :) )

[Andrey2470T]

(And if someone wanted to tackle reflective water I'd be quite supportive of it :) )

I would like to see the specular lighting in MT (the reflective water is enough specific feature) with a support of setting materials for each node + yes, reflections of the environment. That would allow to make a mirror on water and on glass surfaces for example. Currently I'm learning OpenGL and GLSL, so I want to try to make that in MT, but that will rather take much time as I'm a quite newbie in it and I don't guarantee I can do that.

[Extex101]

Incredible work guys! Congrats on the merge!

[LoneWolfHT]

Looks like the segfault was due to a conflict with @SmallJoker 's free_rotation PR

[LoneWolfHT]

Just got another segfault when a mistake in some mod code caused a game crash, I'm using latest master with no code changes made

[Jipok]

2 possible solutions are, change shadow tilt by something between 15.0 and 30.0 in the advanced settings, or if you don't want hard shadows, you can increase the soft shadow range by something like 5.0

Well, the trees are better. But are animal shadows normal? Why stripes?

cringe_shadow.mp4

But in general, shadowing improves the picture very well.

[neoh4x0r]

The drawer visuals from the drawers mod https://github.com/minetest-mods/drawers are not displayed correctly.

With Shadows Disabled:

With Shadows Enabled:

[0xLiso]

The drawer visuals from the drawers mod https://github.com/minetest-mods/drawers are not displayed correctly.

First of all... LoL.

Can you create an issue for that? this PR is closed and these bugs can be easily forgotten.

[rubenwardy]

Locking this PR as it has been merged. Please report any bugs in new issues