Translucency sort idea

[juhdanad]

In this issue I would like to introduce a method to make translucent nodes sorted properly. Since there are advantages and drawbacks, I hope there will be suggestions to improve the method, and also I would like to know if this feature is needed.

Rendering a map block when we are looking at it from one of its corners (the nearest point of the block to the camera is the block's corner) is always can be done with one fixed order:

There are 8 corners, so this would require eight translucent meshes from a map block.

In the general case we have to split the map block to two (the nearest point is on the edge), four (the nearest point is on a side) or eight (the nearest point is in the block) sub-blocks. You can see the second case there, with the appropriate rendering order of the sub-blocks:

To achieve this, we have to render the map block multiple times, with different translucent meshes, while using user-defined clip planes (red on the image) (see also SViewFrustum::planes).

To sum up, the steps are:
On mesh creation: create eight translucent mesh buffers per block (if there are translucent nodes at all)
On render:

1. Render opaque nodes as usual
2. Sort map blocks based on their distance from the camera
3. Render translucent parts of ordered blocks. If the block needs to be split, render it multiple times with different meshes and clipping planes.

This method is efficient if there are a lot of translucent nodes, since you don't have to sort all rendered faces every frame, just rendered map blocks.
Memory usage is increased because there are more meshes per block. However, a mesh will only hold half as much geometry, as we know the viewing direction and can do the backface culling ourselves. Also, only translucent nodes will require more memory.
Minetest will have to process more vertices because some blocks are rendered multiple times. However, many blocks don't have to be split (increasing the viewing range increases the ratio of such blocks) and only one block requires eight renders. Also note that rendering multiple times does not necessarily slow down the game in a noticeable way because these extra vertices will be filtered out by the clipping planes and do not get to the rasterization phase.
And finally, Minetest does not use texture arrays or texture atlases. This means that each material will have to go in a separate draw call. If translucency sort is implemented, material switches will be inevitably more frequent. In my opinion this will be the most expensive part of translucency sort, not with just this implementation.

[juhdanad]

I forgot: this would sort nodes properly, but not translucent entities, particles and clouds. Translucent mesh nodes would also work, but only if their translucent part is convex (like flowing liquids).

[Wuzzy2]

❤️

Kudos for putting some thoughtwork into fixing depth sorting. This is one of the oldest bugs in Minetest, namely #95.

This would finally allow mods to have translucent nodes which don't look like shit (and broken).

[paramat]

Interesting.
Potentially huge problems though, as mesh-creation is already very intensive and is a performance bottleneck, and mesh data takes up a huge amount (1+GB) of memory on the client for the default of only 5000 blocks (only 136 nodes of view range).
But i'm not discouraging you, i appreciate the investigation, keep up the good work.

[Fixer-007]

Some kind of fix is needed for sure, minetest is full of rendering problems because of lack of sorting :(

[paramat]

Somewhere in IRC dev channel logs hmmmm details how he was planning to add depth-sorting, something to do with materials, worth searching for.

[Fixer-007]

@paramat this one? http://irc.minetest.net/minetest-dev/2016-04-10#i_4580752

[paramat]

Yeah that's a later discussion, he goes into more detail before that.

[JustinLaw64]

More than half the corner meshes for many mapblocks will probably never be used. I think corner meshes should only be generated and cached on demand.

Perhaps corner meshes don't even need to be cached since as the player passes by the mapblock, 14 intervening meshes are being regenerated as the player moves assuming the player moves in a straight line. Leaving another one in shouldn't make performance much worse and should save some memory. Leaving us with only one mesh for each mapblock.

A player perspective of a mapblock normally cannot switch corners without going through a bunch of passing by that mapblock in some way anyway.

[Fixer-007]

Pasting this from IRC http://irc.minetest.ru/minetest-dev/2017-10-31#i_5124931, this tool will be useful for someone who wants benchmark of their concepts of alpha sorting:

[01:12:54] sofar> numzero1: http://irc.minetest.ru/minetest-dev/2017-10-30#i_5123833
[01:19:19] -*- numzero1 installs apitrace
[01:19:46] sofar> numzero1: you want this fork and build qapitrace as well
[01:20:24] sofar> sec finding url
[01:20:35] sofar> https://github.com/janesma/apitrace
[01:20:42] sofar> branch frameretrace

[numberZero]

@juhdanad I have another suggestion: render translucent objects in 2 passes. In the first pass fill z-buffer but not color buffer; in the second pass use color buffer. This way, only the nearest translucent face will be seen; that’s not exactly correct but consistent at least, and annoying visual glitches will go away.

[numberZero]

From one side:

From the opposite side:

As you can see, everything is consistent. Only the nearest translucent node is visible.

Sample implementation (very slow and incompatible with waving water, but works otherwise):

//src/client/render/core.cpp:73
void RenderingCore::draw3D()
{
	video::SOverrideMaterial &mat = driver->getOverrideMaterial();
	mat.Material.ColorMask = 0;
	mat.Material.ZBuffer = false;
	mat.EnableFlags = video::EMF_COLOR_MASK | video::EMF_ZBUFFER;
	mat.EnablePasses = scene::ESNRP_TRANSPARENT | scene::ESNRP_TRANSPARENT_EFFECT;
	smgr->drawAll();
	mat.EnableFlags = video::EMF_COLOR_MASK;
	mat.EnablePasses = scene::ESNRP_TRANSPARENT | scene::ESNRP_TRANSPARENT_EFFECT |
			   scene::ESNRP_SKY_BOX | scene::ESNRP_SOLID |
 			   scene::ESNRP_SHADOW;
	smgr->drawAll();
	mat.EnableFlags = video::EMF_COLOR_MASK | video::EMF_ZBUFFER;
	mat.EnablePasses = scene::ESNRP_SKY_BOX | scene::ESNRP_SOLID |
			   scene::ESNRP_SHADOW;
	smgr->drawAll();
	mat.EnableFlags = 0;
	mat.EnablePasses = 0;
	driver->setTransform(video::ETS_WORLD, core::IdentityMatrix);
	if (!show_hud)
		return;
	hud->drawSelectionMesh();
	if (draw_wield_tool)
		camera->drawWieldedTool();
}

[Fixer-007]

Just my curiosity. Can you build small underwater glass tunnel/cube/whatever moderately deep and look through glass into the sea to see if that water on the surface still blinks like crazy?

[Ferk]

look through glass

MTG glass does not use alpha transparency, though. You'll have to make a custom solid node with alpha to really test.

There's also still the issue that anything with alpha would be invisible when looked through an alpha node, including particles (so you cannot see snow/rain particles through a node with alpha, like in #3761).

[numberZero]

Another approach: render everything always.


It won’t be perfectly consistent, but makes everything visible. The only problem is that it causes funny bugs if nodes marked as translucent are really opaque or almost opaque.


The bugs around here are due to extensive use of translucent nodes; see #6726 for the reason.

The code (reasonably fast albeit a bit buggy, for aforementioned reason and the fact that some objects are marked translucent despite they shouldn’t be):

//src/client/render/core.cpp:73
void RenderingCore::draw3D()
{
	video::SOverrideMaterial &mat = driver->getOverrideMaterial();
	mat.Material.ZWriteEnable = false;
	mat.EnableFlags = video::EMF_ZWRITE_ENABLE;
	mat.EnablePasses = scene::ESNRP_TRANSPARENT | scene::ESNRP_TRANSPARENT_EFFECT;
	smgr->drawAll();
	mat.EnableFlags = 0;
	mat.EnablePasses = 0;
	driver->setTransform(video::ETS_WORLD, core::IdentityMatrix);
	if (!show_hud)
		return;
	hud->drawSelectionMesh();
	if (draw_wield_tool)
		camera->drawWieldedTool();
}

[Fixer-007]

Hmm, tried first code, it gave best result, no blinking, good underwater glass tunnel rendering, but fps down from 90+ to 35, second code is fast but looks different and still has ugly blinking. I really hope you or someone find a solution to at least stop blinking.

Second code render example, notice different water squares:

Blinking:

Also, simple comparison in my test tunnel:
code1>

code2>

[numberZero]

@Fixer-007 Thanks for testing.

1. Speed drop in the first code is due to wrong implementation (that involves rendering the whole scene 3 times per frame). With proper implementation there would be little to no performance penalty. But that would require replacing ISceneManager::drawAll().
2. Now I see that the second approach is, basically, wrong. It may be improved, however, but only using MRT and special shaders, in a way similar to postprocessing; that is not supported everywhere.

[Fixer-007]

Shaders also add some performance penalty

[numberZero]

Shaders also add some performance penalty

Depends.

[Fixer-007]

@numberZero any more ideas? This is one of the most annoying and unfixed aspects of minetest.

[Ezhh]

Unsure if related, but seems to be. At certain depths when using nodeboxes, I get a second waterlevel inside nodeboxes. (At other depths it's just mad disco-style flickering, same as when trying to put nodes using alpha directly above water.)

This porthole is several nodes below water level, but shows another water level while looking through it.

Would love any improvement at all to this madness.

[paramat]

Weird, do you have waving water enabled?
It seems to be showing a stone texture too above the internal water level, which i guess shouldn't be there?
Could you link to the node def?

[Ezhh]

@paramat Yes, waving was enabled. The stone texture is an actual stone node, so that part isn't a bug.

Here is with waving disabled:

And here is the difference depending on y coord of the node (no waving):

You can see the two placed on top of the original show water on the outside instead of the inside. No flickering.

Same again but with waving:

You now get the visible water texture not aligning correctly with the nodes, and it flickers wildly even when you stand still and don't move the camera.

Porthole nodes are here: https://github.com/Ezhh/abriglass/blob/master/nodes.lua#L101
but I've seen the flickering effect on ice (using alpha texture) placed next to water as well.

[paramat]

The water tiles are wrongly visible and z-fight with the nodebox faces, i guess because we assume nodeboxes do not have faces that are positioned like cube faces. We would need to analyse each face of the nodebox to see whether the water tile needs rendering, seems non-trivial. Even then a stair or slab node has part up against the cube and part not, so whether we render that water face or not it would be wrong.

Waving water has big problems, it was about to be removed by RBA due to being so buggy, not surprised it doesn't work here.

Why does this node have to be a nodebox if it's a cube? Just wondering.

[Ezhh]

It's a nodebox because glasslike drawtype doesn't allow different textures per face. (Unless that changed since I last checked.)

[Ezhh]

And I could understand the water just staying visible on the outside, but the flickering is a real issue, and the repeated water surface inside the nodebox is just odd.

[paramat]

Ok, maybe it doesn't have to be glasslike drawtype? (maybe it does, just guessing).
Yes the flicker is z-fighting which is unavoidable since the 2 faces are effectively in the same place.

[Ezhh]

I couldn't find a way to do these nodes without using nodeboxes (Same issue with one-way windows). This would easily be fixed by allowing a different texture per side for glasslike, but... that's unrelated to what this thread is about, and I don't want to remove the focus from something that is more important.

[paramat]

Yes i got interested and experimented, best result was glasslike but you have to then surround the node with other nodes to not see the extra portholes. I opened an issue for using 6 textures.
Your porthole issues are actually unrelated to translucency sorting, one issue is buggy waving water, the flicker is this issue #3704 will comment there.

[tyronx]

Another solution to the transparency problem is using order independent transparency renderers. One such is WBOIT. From a technical standpoint fairly simple to implement: http://casual-effects.blogspot.com/2014/03/weighted-blended-order-independent.html

It required a lot of tweaking for me to get it blend decently well with clouds + blocks + particles, but sure appreciate being able to offload most of the transparency work to the gpu

[numberZero]

Hmm, I had a similar idea. But as I wrote, it requires MRT and post-processing and all that stuff that is poorly supported.

[tyronx]

Ow, yea, rendering to an off-screen buffer would be necessary. I didn't see that part.

[numberZero]

Possible close? The issue is #95, this one was for discussion of a particular solution that didn't get much support IIUC.