Multisplat shader

[Zylann]

This is to present a new shader I've been working on shortly after releasing the ARRAY shader, because while it has relatively simple code, it is hard to get right and has a few blending issues.
This new shader is nicknamed MULTISPLAT16.

Pros:

• Allows to use up to 16 materials* on the whole terrain
• Allows to blend up to 4 materials at the same time on a particular pixel. These materials are chosen dynamically amongst the highest weights on every pixel.
• Better usability and results pretty much similar to CLASSIC4
• Almost no artifacts (there are some, but subtle and much easier to avoid than ARRAY)
• Supports same features as CLASSIC4: depth blending, normal maps, roughness, holes and fading to globalmap (no triplanar yet, I don't know exactly how to proceed without tripling texture fetches unnecessarily).
• Is a starting point to support a lot more materials by "tiling" index sets, because it supports dynamic indexes due to its filtering of the 4 highest weights, and the 12 spare ones offer plenty of room to hide switching artifacts. However that's for much later.

Cons:

• High cost, despite sampling only 4 materials at a time (aka 8 texture calls) and not all 16 (which would be 32). For comparison, I tested the following frame times (duration of the whole render with vsync disabled in fullscreen, not just the shader, as Godot doesnt provide the tools to measure that):
  • Single quad with default material: 0.4 ms
  • CLASSIC4: 0.9 ms
  • ARRAY: 0.74 ms
  • MULTISPLAT16: 1.93 ms
• Higher memory usage: instead of a single RGBA splatmap, it uses 4 of them, to get to 16 weights. It's mostly a big deal with huge terrains, so in the future streaming them out early to fallback on globalmap will have to be considered.

This shader also requires texture arrays to work, so it's like a middle ground between ARRAY and CLASSIC4.

I'm still looking for ways to further optimize it, but so far I could not find more tricks to do so. Lowering to 12 materials and blending only 3 is of course a possibility, if considered enough. Fading to globalmap also helps a lot here, because far away pixels can bypass the entire blending system and use the baked albedo instead. Amusingly, putting objects occluding the ground also help (characters, vehicles, grass, trees, roads, buildings), thanks to depth testing avoiding costy ground pixels from being calculated.

It can be tested in the multisplat branch for now.
Source: https://github.com/Zylann/godot_heightmap_plugin/blob/multisplat/addons/zylann.hterrain/shaders/multisplat16.shader

---

*materials: not actual materials. In this plugin, one material actually corresponds to 2 textures: albedo_bump, and normal_roughness.

[RonanZe]

Seems to work great! I didn't have time to test with an heightmap right now. Maybe this WE.

I don't have any references regarding the ms rendering time.
Did you had the opportunity to bench other splatmaps shader on other game engine?

[Arnklit]

Hi @Zylann. I was thinking to give this a go. So I should just follow the same workflow as using the Array one? Would it be possible for you to set it up so you generate the arrays for the user from the separate textures they add? I don't personally worry too much about setting up the texture array, but I think it will scare a lot of people off from using it and they'll just end up sticking with the Classic4.

[Zylann]

@Arnklit I have a terrain texture importer draft in a branch, but I didnt take the time to finish it yet. The main reason being, I can't make a nice UI for it because the importer dock is not as customizable as the inspector (while, at the same time, being a bummer considering this proposal I made a while ago: godotengine/godot#27378).
Also because it's a workflow involving [many source files] => [many destination files], Godot's import/resource pipeline wouldnt work automatically for it so that's where I left it. If I had to finish it now, it would basically require manual action from users.

[Arnklit]

@Zylann I would love for that proposal to go through. Makes a ton of sense to me, I remember being confused about where the import settings were when I was starting out and it still breaks my workflow every time I have to change back and forth.

But I don't really think I understand why you would need to customize the import inspector to do this. Couldn't you just use the same UI as for Classic4, but allow people to add 16 slots and then have a "Generate/Update Array texture" that people had to click after making changes to the 16 textures and you would generate the array in the background and import it correctly and place it in the terrain data folder?

[Zylann]

If I tie this to the terrain UI you would have to redo all 16 slots on every terrain you make, and would require saving a very big texture array resource next to every terrain. I've seen someone making a game with multiple terrains where textures ended up occuping 2 gigabytes (the rest of the game being only 100Mb), which even prevented him from exporting due to a Godot 3 limitation, so it is desirable to keep it distinct.

My plan was to make a standalone texture resource type instead, which describes 2 texture arrays at once (albedo+bump and normal+roughness), and this is what the configuration looks like:

TerrainTextureSet
	Mode: Array
	Textures:
		Texture0:
			Albedo0.png
			Bump0.png
			Normal0.png
			Roughness0.png
		Texture1:
			Albedo1.png
			Bump1.png
			Normal1.png
			Roughness1.png
		...
		...
		Texture15:
			Albedo15.png
			Bump15.png
			Normal15.png
			Roughness15.png

	Output:
		MyAlbedoBump.arraytex
		MyNormalRoughness.arraytex

The idea of using the import system was to allows tweaking textures and have them automatically reimport, and also, to benefit from more import options such as compression. It should also be noted that I can't trigger imports from script either: godotengine/godot-proposals#1615
Unfortunately it still means users have to remember to NOT leave the 64-ish 2K source textures bloat their project at time of export, IF they use a shader based on texture arrays.

At least, if I make the texture list from the terrain node to make users setup all the kinds of textures this plugin supports, it means it would have to support 4 different situations at once...

1. Using a classic shader, where each texture is already packed (so the slots show that)
2. Using a classic shader where each texture is not packed yet, so would also require a specific UI to pack them (and the UI would show the source ones instead)
3. Using an array shader where the texture array is already packed
4. Using an array shader where the texture array does not exist yet, in which case textures shown here would be the source ones... but not actually referenced because once again, Godot should not export these...

It also means users can't edit texture sets if they don't create or edit a terrain first.
Then add to this the ability to re-use the same texture sets between multiple terrains... and you'll see why I'd like to make this a resource of some kind^^

[Arnklit]

ehm... yeah I guess I hadn't really thought that idea through with my small test terrain usage of your tool...

[Zylann]

Btw how was performance for you? It's one of the nicest shaders but also the most expensive of them all. It's still way below 16ms but 2ms can be a lot when adding up everything else :p

[Arnklit]

I'm seeing quite a bit of slowdown on the brushstrokes for some reason? Just using a 50ish size brush it's getting pretty slow? And the undo seems to be broken.
This is using a 4k albedo_bump array test texture I made, no normal map set.
https://youtu.be/sbADtAkh92w

Edit: Just tried the standard array shader afterwards and that one is super fast with giant brushes and has no undo errors.

[Zylann]

The multisplat strokes are slow because there are 4 splatmaps to edit at once, that doesn't come for free :p (also I suspect this hasn't been accelerated with GDNative yet).
Whar are those black spots you are getting?

[Arnklit]

That's from the undo. That's the bigger problem right now. It doesn't seem undo is working correctly. It's fine with both Classic4 and the Array shader.

[Proksimas]

Multisplat shader is really a great idea but i have a little problem.
The first utilization works without bug but i i close the scene (or godot) and i open it i have black texture which erase some of mines:

I don't know why. I have to restart that

[Zylann]

Does it happen too if you do the following steps?

• Create a new terrain
• Paint using multisplat
• Save the scene
• Focus another window (like internet browser or file explorer)
• Focus back to Godot: it should show you a quick import popup
• Close the scene
• Reopen the scene

Also, does it happen too if you get the latest version from master? It is possible that bug was fixed with 7c1c0e8

[Proksimas]

Okay i didn't see you last commit 7c1c0e8. My bad.
And it seem to work perfectly 👍
Thanks :)

[Zylann]

Going to close now since the feature is in master