Shadows

[AnyOldName3]

Scrawl suggested I actually make a PR for this so that it's easier to keep track of while the last few things are sorted out.

Current issues (that I can think of):

Blockers

• I've not decided which way to make the sky not receive shadows is best (https://github.com/AnyOldName3/openmw/tree/osgshadow-test-vdsm-sky-method-one versus https://github.com/AnyOldName3/openmw/tree/osgshadow-test-vdsm-sky-method-two) Scrawl likes method two and I trust his judgement above my own.
• The minimap needs shadows turning off.
• The character preview needs shadows turning off.
• Interiors need shadows to not be cast by walls etc. (as they use a 'fake' light source which is blocked everywhere)
• Settings need to be changeable by users instead of hardcoded.
• Description of settings needs adding to the documentation
• Turning on shadows should potentially somehow warn users that they might get issues with shaders off unless force shaders on, but maybe the documentation should suggest they want to do this
• Assorted parameter tuning to make things look as good as possible.
• Move as much as possible out of shadermanager.cpp
• Separate MWShadow into two classes - a shadow technique class based directly on osgShadow::ViewDependentShadowMap and a shadow manager class handling interaction with the rest of the engine.
• Work out why some Linux users are seeing issues that I'm not, and fix the problem. (This wasn't Linux-specific after all, and was just that distant land was on and broke things.)
• Ensure that shadow map projection keeps shadow map texels roughly 1:1 with screen pixels.

• • Fix this when due to perspective errors.
• • Fix this when due to a high viewing distance.
• • Fix this when due to similar light and camera angles that wasn't the issue shadow maps being set up to cover non-existent objects under the ground.
  • • Stop that fix from breaking first person meshes.
  • • Stop that fix from sometimes making shadows disappear from near to the camera when the closest thing to the camera is terrain, not an object.
• • Fix this when due to similar light and camera angles as this issue seems to have reappeared never actually been solved?
  • • See how easily this can be fixed by messing around with Cascading Shadow Maps.
  • • Implement a maintainable and configurable implementation of Cascading Shadow Maps.

• Ensure distant terrain causes no issues with the ComputeLightSpaceBounds visitor (which I'm not 100% sure is ever doing its job properly or we'd not have issues when increasing view distance but leaving distant terrain off... unless it's the same water-has-a-huge-bounding-sphere issue that means we can't do bounding volume based traversals).
• Sort out excessive light space perspective transformation effect with RenderLeafTraverser<RenderLeafBounds>.

• • Stop distant terrain screwing up the ratio calculation.
• • Work out what's going on with this screenshot. The nearest shadow map (red) is excessively distorted, removing detail from the tree on the left. I didn't work out what was going wrong, but I fixed this by accident.

• Resolve this issue Shadows #1547 (comment)
• Fix this (yet) again Shadows #1547 (comment) but for OSG 3.6 maybe not OSG 3.6 @akortunov's mad computer which has issues no one else has reported. Not just akortunov's computer absolutely ridiculous viewing distances. Fixed the viewing distance issue yet another reason. the CLSB visitor giving erroneous results & the ConvexHull thing not considering shadow casters outside of the viewing frustum.
• Fix disappearing first-person meshes. Video in Shadows #1547 (comment)
• Fix wavy Balmora banners breaking things. https://youtu.be/oMpo6RYrFwg
• Investigate issue 1 in this post: Shadows #1547 (comment) Solve issue where OSG's computed far plane is nearer than the most distant geometry. The computed far plane was fine all along. Solve distant (relatively speaking) shadows disappearing due to erroneous clipping.
• Fix stateset weirdness.

• • Work out if there's a reason @akortunov had minimap and character preview issues after I fixed them for other people and if the magic fix that shouldn't have worked with CSM is reliable and identifiable. No one else has reported anything, so hopefully this is fine.
• • Investigate and fix this unholy abomination: Shadows #1547 (comment) The person who reported this can no logner replicate it, so I'm going to assume I've fixed it by fixing something else.
• • Work out the cause of and then get rid of messages about an invalid OpenGL state.

• Fix depth precision issues.

• • Reduce Peter Panning without increasing flicker on thin meshes like leaves and ferns.
• • Reduce flicker on thin meshes like leaves and ferns without increasing Peter Panning.
• • Fix evil flicker on a bunch of things (especially with CSM and uniform/log ratio at near 1.0 and a ridiculous viewing distance). Video in Shadows #1547 (comment). The root cause is all shadow maps sharing the same near/far planes so the near shadow map has basically no depth resolution. The actual root cause is that the terrain paging system regards all shadow cameras as maximally distant, so serves up the same single whole-of-Morrowind TerrainDrawable for each of them, and that forces all shadow maps to use the same near/far planes so the near shadow map has basically no depth resolution. This issue can be avoided by disabling terrain shadows, which are basically garbage anyway and will be until the issue is resolved. Attempts to do so have lead to weird crashes, so this is being backburnered for now. The real actual cause was that the shadow map near plane was too far from the things receiving shadows. Using depth clamping to move it closer without preventing things outside the viewing frustum casting shadows has sorted this out.
• • Use normal-offset shadow mapping to try and reduce flickering with much less peter panning.

• Adjust the alpha test such that invisible actors cannot cast a shadow.
• Fix animated things being culled for no reason Shadows #1547 (comment). Specifically, it seems that the direct parents of RigGeometries are getting falsely culled sometimes, but other times it's a more complex issue that I won't go into unless someone asked. Copying the bounds from a RigGeometry to its children (which contain the actual mesh) fixed the issue, though.
• Solve yet another issue with indoor shadows not being everywhere they should be. Investigation suggests this is another LiSPSM issue. shows this is my ConvexHull::extendTowardsNegativeZ() function failing in common edge cases and the provided ConvexHull::clip(osg::Plane) function failing in edge cases made common by the extendTowardsNegativeZ function's introduction.
• More shadows being gone: Shadows #1547 (comment) When I went to test this, I encountered an issue and fixed it, but I'm not 100% sure if it was the same one. Either way, shadow maps were being used exclusively even for regions that hadn't been considered their responsibility during their cull traversal, so that was fixed by telling the shader about it.
• Degarbagify terrain shadows. The terrain paging system regards shadow cameras as maximally distant, so serves up a single whole-of-Morrowind TerrainDrawable, which is far too low quality to let all but the largest mountains cast any shadows.
• Fix colorMode issue (erroneously described as full object shadow pop-ins here: Shadows #1547 (comment))
• Fix shadows being gone from a set of steps: Shadows #1547 (comment) Technically, this is just really bad Peter Panning caused by a huge polygon offset, so it might not be fixable, per se.
• Surprisingly, another missing shadows bug:

  Some strange behavior at extreme sunset angles. Note the area roughly where the soldiers are as the camera moves around.
  https://streamable.com/fc7dw

  That doesn't look like it's supposed to happen. Can I get a save game right there along with your settings (not just shadow settings as stuff like view distance makes a difference, too).

  Save File
  Settings.cfg
  I'll add that I could not reproduce this on older build artifacts.

• Incompatibility with the water shader and disabled shadows. (Note: I can't reproduce this).

  Turning off shadows appears to break the water shader.
  https://streamable.com/0s382 (shadows off)
  https://streamable.com/l5i4w (shadows on)

  That's bad. Can you get me an APITrace of it being broken, please? There should be a thread somewhere explaining how. Maybe testing with an artefact from before normal offset shadows got merged, too, as that's the only thing I've done recently that might have broken it, and I'd have expected someone to have reported this already if it's been broken for a long time.

  I'll see if I can figure out APITrace. For now I can say it doesn't break like this on my pre-normal offset artifacts.
  Apparently I'd already fixed this by the time someone else reported it, which is why no one else could reproduce it. Good job, me, on wasting a bunch of time hunting something that was already dead.

• Weird glow on the bottom of things at night. https://forum.openmw.org/viewtopic.php?f=6&t=4417&p=61173#p61165
• Threading issue with debug HUD frustum geometry. Either double-buffer it or, as it's only there during debugging, make it dynamic so OSG handles the threading (and potentially makes the whole next frame stall while waiting for it).

Non-blockers

• Improve the culling polytope used when allow shadow map overlap is enabled so performance doesn't fall linearly as more shadow maps are added.
• Investigate this: https://forum.openmw.org/viewtopic.php?p=57721#p57721 User unable to reproduce
• At night shadows need to be tweaked in some way if anyone other than me starts complaining about them (as another fake light source is used at night, too).

• • Maybe the position of the light source should follow one of the moons.
• • Maybe the diffuse component of the light source should be reduced and the ambient one increased.

• Investigate disabling lighting calculations when rendering the shadow map for a speed boost.
• Investigate disabling the light manager for shadow map rendering as there's no point working out which lights have the most effect on which objects or sending lights to the GPU when we're ignoring them in the shader.
• Consider using a big PCF filter as an option to soften shadows and/or make undersampling look less bad.

• • PCSS is probably the only good option for soft shadows as everything else involves blurring the parts of shadows that need to be hard.

• Consider potential systems to avoid wasting shadow map space on distant stuff.

• • Maybe a heuristic in QuadTreeWorld based on maximum steepness in a cell and the angle of the sun in the sky which affects the ComputeLightSpaceBounds visitor.
• • Maybe just a boring maximum shadow map distance setting.
• • As distant statics don't exist yet, there's no good way to theorise about the ideal solution for these and if it's just going to be as simple as having a distant statics shadows setting.
• • Maybe see if there's a way to make terrain shadows separate for ECLD cells and terrain paging (distant) cells.

• Sort out excessive light space perspective transformation effect with RenderLeafTraverser<RenderLeafBounds>.

• • Suppress unsolvable issues by finding the highest value for the minimum lispsm near far ratio setting which doesn't produce artefacts. Users can help with this, and may find playing with the values at the bottom of shadow.hpp in this branch useful. Remember that before telling me that a value is safe, it needs to work (meaning not look worse than a lower value) both with and without the CLSB applied (the thing that automatically toggles), with and without a high view distance, with and without distant terrain, with and without split shadow maps (look at the end bit of the terrain fragment shader if not using exactly three), in first and third-person view, and in various combinations of these settings. MOVED FROM BLOCKERS TO NON-BLOCKERS

• Move common shadow shader code into a shader header.
• Fix 360° screenshots.
• Fix shadows being in the wrong place in the screenshots embedded in saves. https://user-images.githubusercontent.com/17745136/43905549-f5b81e40-9bf9-11e8-8c2f-6f62c87fea98.png
• See if MGE XE does something more interesting with interior shadows that I can copy.
• Investigate the performance cost of doing a different thing for one light on a range of systems to work out if Mesa's optimiser is bad or if I just expect too much from shader optimisers. Patch here: AnyOldName3@1098bd2

Maybe unrelated to shadows

• Intermittent OpenGL errors (which don't show up in APITrace etc. but give the message glCallList failed because (depending on the operation) a referenced binding point is empty; a referenced name is not the name of an object; or the given name is otherwise not valid to this operation (GL_INVALID_VALUE) when caught by KHR_DEBUG). Note: These errors may be because of a regression in OSG 3.6.x and may also happen on master-branch OpenMW - JDGBOLT's issue reports are from a build with a bunch of ongoing PRs and unreleased OSG builds.

Want to tell me how well this works on your machine? I'll need screenshots (ideally with the debug hud turned on), a description of your shadow settings and maybe a coc target and definitely to know whether distant terrain is on.

Testing both with and without this patch (including with and without shadows enabled) on a range of systems might be helpful to me. I know it makes a difference on Mesa, and no difference with AMD's and Nvidia's proprietary Windows drivers, but don't know for other situations.

Want to give me free money because you really like shadows? I have a Patreon at https://www.patreon.com/AnyOldName3

[psi29a]

@terabyte25 we're still in 0.46 development... asap / time is relative :)

[AnyOldName3]

Guys, I think all the ticks are ticked!

[akortunov]

So any objections left, or we can merge this PR at last?

[kurnevsky]

@akortunov this patch for mesa users is not merged yet, @AnyOldName3 said it would be (it seems he just forgot?).

[AnyOldName3]

Yeah, I should merge that...

[AnyOldName3]

I merged that.

[psi29a]

Last call...

[terabyte25]

Did the shadow map resolution stuff breaking due to insane resolutions thing get fixed? Does it even count as a blocker to this PR?

[akortunov]

Did the shadow map resolution stuff breaking due to insane resolutions thing get fixed?

IIRC, we consider it as a "not an issue".

[akortunov]

Any progress here? I see no blockers in the PR description.

[Capostrophic]

AnyOldName3 is stuck trying to debug some OpenGL errorrs that appear when looking at the sky under Mesa and not Nvidia.

[AnyOldName3]

It actually seems to not happen under Mesa, either, just AMD's proprietary Windows driver. It's been a nightmare to debug, as it won't happen when an OpenGL debugger is attached, and while it does trigger khr_debug messages, all OpenGL state gets reported as default, which can't be the case as otherwise there'd be no error.

The next step is to build OpenMW against a build of OSG with debug symbols, as at least then I'll be able to see what OSG thinks is going on. The issue with that is that OSG and OpenMW don't like linking on Windows, so I'm at the mercy of ananace. He finished the OSG build on Friday night, but I didn't have any time yesterday to look at it.

[psi29a]

@AnyOldName3 do you consider this a blocker? Would merging this in and getting it to the masses have any value in helping to resolve this issue? We could put out a call for feeback when people use amd gpu drivers for windows.

[AnyOldName3]

The hard part is being able to reproduce it in the first place when it's vendor-specific, and I can do that. More testers wouldn't help.

At least in the short term, I'm going to consider it a blocker. An OpenGL error shouldn't happen unless someone's made a mistake, so it's not impossible that there are other symptoms that just haven't been found yet. If I still can't fix it when I have everything set up with debug symbols, I might demote it to non-blocker and merge anyway.

[AnyOldName3]

That should be all the blockers dealt with now.

[psi29a]

Still in [Review time] or good to merge?

[psi29a]

Hellllll yesss

[terabyte25]

Finally. I should probably post it on r/Morrowind.

[rhtucker]

I would be hesitant of advertising this too much until the actual release that includes it is out. Otherwise we get a bunch of people asking why they can’t see the option. Also in our news post we should make it obvious how to enable.

But hooray! They’re merged!

[lysol90]

I could write another post on how to enable them tomorrow I guess...

[terabyte25]

@lysol90 is it possible for you to add that on to the current one?

[clort81]

This is great stuff. Probably too heavy for my little ARM SBC.
Wondering if a 'trivial' alpha-blended texture below foot would be easy-ish to implement?

The opacity could be reduced during stepping with something like:

opacity_percent = floor(100-100*(zero_shadow_height-elevation_of_foot),0)

[AnyOldName3]

Something like that is probably best achieved by adding two planes to some part of the body mesh and making them not have vertex weights so they stay stuck to the floor. It's probably less work to do it like that, and it definitely leaves the engine in a more maintainable state.

The way you've drawn it, though, makes it look a lot like (very localised) SSAO, and we'll have support for some form of that when we implement post-process shaders. It'll have a much smaller performance hit than proper shadows, but will be able to apply an effect like in your screenshot everywhere, not just to feet.

[clort81]

Thanks for the reply AnyOldName3! Can you think of a way whereby the opacity of shadow texture could be reduced when the foot lifts off the ground? Perhaps the size of shadow could increase also with foot height.

Are any modders reading this who could add shadow textures to feet?

[AnyOldName3]

There's probably some Nif property you can make vary over the course of the animation cycle.