Terrain normalmapping

[amelentev]

This is continuation of #643

• Adds optional support of normalmaps (_nm.png), specularmaps (_sm.png) and heightmaps (_hm.png) for terrain, decals(tiles) and water.
  • Expect opengl format normalmaps with y=up.
  • Heightmaps are for future use and mods, not used in shaders for now.
• New config option "terrainShaderQuality" to control new rendering/shaders. By default it is on (=1=NORMAL_MAPPING). There is a button in debug window to switch it for now. Will add it to Video options after Make video options scrollable #1823
• New rendering use optional maps, but works fine without them (with minimal performance hit vs original rendering).
• Added decals automipmap. Only highest available resolution decals (upto 512) are used. Lower resolutions can be deleted now.
• New Graphics tab in Debug window for modders. Speedup texture and shaders testing.

Demo with @MaNGusT-'s textures: https://www.youtube.com/watch?v=NjzMcK9Oqmk
Thanks @MaNGusT- for help.

in new rendering:

• diffuse lighting moved from lightmap to the shaders. lightmap still has ambient occlusion.
• terrain shaders use light intensities from piedraw.cpp. With default light intensities the result is brighter than original terrain. I use ambientLight*0.5 in shaders to compensate.

There is a minor performance impact. Benchmarks:
Standard textures without optional maps. cam_1a, first alpha mission. resolution=4k, gpu=GF 1060 3G
tsq - this PR with specified terrainShaderQuality.

       | opengl | vk
ubuntu 20.10    |
 master| 498fps | 328fps
 tsq=0 | 485fps | 325fps
 tsq=1 | 445fps | 310fps

I got similar results on win10 (opengl worse, vk better).

Possible future work (out of scope of this PR):

• pack normalmaps in videomemory to 2 channels (grey+alpha). specular and height maps to just 1 channel (grey)?
• move decals rendering from texture atlas to texture array?
• terrainShaderQuality switch in shaders can be optimized if needed. by different shaders entry points or different shader files. probably not worth it.
• separate fog of war, AO and lights from lightmap.

[lgtm-com]

This pull request introduces 2 alerts when merging 6b7b284 into 44631a1 - view on LGTM.com

new alerts:

• 2 for Multiplication result converted to larger type

[past-due]

@amelentev This is a tremendous effort, and the result is amazing! 🥳

It looks like our traditional merge method of "Rebase and merge" is unavailable with the current commit history (likely the merge commits), and I'd hate to squash everything. I can try poking at rebasing / rewriting the commit history (and eliminating the merge commits) at some point if no one else gets to it first.

EDIT: It does look like there might be an illumination bug with the old terrain rendering, as you noted on the YouTube video. But the new mode looks fantastic.

[MaNGusT-]

Thanks Artem for crazy efforts here.

EDIT: It does look like there might be an illumination bug with the old terrain rendering

fix:
vec4 main_classic() { return 1.75 * texture(tex, uv) * texture(lightmap_tex, uvLight); }
It's because I've changed brightness of diffuse textures and it has to be recovered for classic mode.

I don't know how we are going to handle new textures. If we will use 1 pack of textures then this fix will work, If we will use old diffuse textures for classic mode then no need to multiply by 1.75.

[past-due]

I don't know how we are going to handle new textures. If we will use 1 pack of textures then this fix will work, If we will use old diffuse textures for classic mode then no need to multiply by 1.75.

That was one of the things I was wondering.

How far along are the new terrain textures? If we have replacements for everything, and they look fairly similar in "classic" mode, I don't see why we can't ship a single pack of textures. (Also, where is the latest set of new textures so I can test?)

EDIT: Maybe we can make this classic diffuse parameter a uniform and then allow it to be provided by the set of textures, if we need to support more than 1 pack of textures...

[MaNGusT-]

If we have replacements for everything, and they look fairly similar in "classic" mode, I don't see why we can't ship a single pack of textures

They don't look similar(and can't)... well, they look acceptable, I removed any shadows and highlights from them, so they look a bit unnatural without normal and specular maps. https://www.youtube.com/watch?v=69oiW1tm8qo

How far along are the new terrain textures?

WIP

(Also, where is the latest set of new textures so I can test?)

I'll upload them to my google drive, so everyone could give 'em a try. Also, for now I think we could use specular level = 0.1 instead of specular maps. They aren't needed everywhere, so I need to decide which of them we need to use. This will save a lot of space and video memory. Have tested the game on old GF GTS250 512mb and it works good only with 1024 textures option, not enough memory for 2048.

EDIT: Maybe we can make this classic diffuse parameter a uniform and then allow it to be provided by the set of textures, if we need to support more than 1 pack of textures...

Or other texture packs should have adjusted brightness to look right.

[past-due]

If we have replacements for everything, and they look fairly similar in "classic" mode, I don't see why we can't ship a single pack of textures

They don't look similar(and can't)... well, they look acceptable, I removed any shadows and highlights from them, so they look a bit unnatural without normal and specular maps. https://www.youtube.com/watch?v=69oiW1tm8qo

Hmmm. We may want to keep the prior textures around for "classic" terrain mode, then. We could always ship 1024x1024 versions of them as a compromise between file size and quality.

If we do support different terrain texture packs for each quality level, then we probably want to restrict changing the terrain quality to the main menus (i.e. not during a game - even if we added support for loading+replacing textures on the fly this would likely cause a freeze for other players due to the "stop the world sync" in multiplayer games so shouldn't be permitted there).

Also, for now I think we could use specular level = 0.1 instead of specular maps. They aren't needed everywhere, so I need to decide which of them we need to use. This will save a lot of space and video memory. Have tested the game on old GF GTS250 512mb and it works good only with 1024 textures option, not enough memory for 2048.

Makes sense to me. (We also need to add support for uploading textures in a compressed format to the GPU, but that can come later.)

[amelentev]

Also, for now I think we could use specular level = 0.1 instead of specular maps. They aren't needed everywhere, so I need to decide which of them we need to use. This will save a lot of space and video memory

You can create 1x1 pixel specular map for constant speculars. And it should take 1 pixel in video memory. Or you can create lower res *maps with bigger res textures. It will work for terrain and water textures, but not for decals because of texture atlas. Default decals specular = 0.1.

About texturepacks. I think it can be just regular mods. And we can set terrainShaderQuality parameter/uniform from mod somehow. Or, just put shaders in the mod..
I don't like the idea of bundling 2 big texturepacks into the game distribution.

[past-due]

About texturepacks. I think it can be just regular mods. And we can set terrainShaderQuality parameter/uniform from mod somehow. Or, just put shaders in the mod..
I don't like the idea of bundling 2 big texturepacks into the game distribution.

I'm less concerned about size and more about quality and ensuring all the necessary pieces are built-in. (We have already received complaints that the videos aren't bundled with all of the downloads and that an Internet connection is required during install to fetch them.)

Given the current quirks with the mod system, it's probably best that we build it in (similar to the faction texture switching) based on the terrainShaderQuality mode.

If classic is the "lower quality" option, I think we can easily reduce the size of the textures such that it's not an overwhelming portion of the distribution. (Classic could be shrunk to 1024x1024.) The new textures do look exceedingly flat / unnatural in places without the normal-mapping, so I think this'll produce the best compromise.

Overall, though, I just want to reiterate how awesome this. 😎

[past-due]

@amelentev I've gone ahead and rebased this on current master (eliminating the merge commits) and force-pushed to your branch. (In case I made a mistake somewhere, I've backed up what was previously on this branch at https://github.com/past-due/warzone2100/tree/new_terrain_normals_backup_before_rebase1)

(See suggestions here for how to reset your local branch to the remote force-pushed branch if you aren't familiar: https://stackoverflow.com/questions/9813816/git-pull-after-forced-update)

git checkout master && git branch -D 2019_gfx_api_work_20_terrain_normals && git checkout -b 2019_gfx_api_work_20_terrain_normals amelentev/2019_gfx_api_work_20_terrain_normals

[lgtm-com]

This pull request introduces 2 alerts when merging e7fd995 into de33e81 - view on LGTM.com

new alerts:

• 2 for Multiplication result converted to larger type

[lgtm-com]

This pull request introduces 2 alerts when merging 281a711 into 4eefd78 - view on LGTM.com

new alerts:

• 2 for Multiplication result converted to larger type

[lgtm-com]

This pull request introduces 2 alerts when merging 20de4a2 into 023229b - view on LGTM.com

new alerts:

• 2 for Multiplication result converted to larger type

[amelentev]

@past-due Thanks for rebase. You can also remove commits:

CI: enable Win Portable Build 5c13e75
Revert "CI: enable Win Portable Build" … ec4bee1

[past-due]

@past-due Thanks for rebase. You can also remove commits:

CI: enable Win Portable Build 5c13e75
Revert "CI: enable Win Portable Build" … ec4bee1

Fixed. 👍

[lgtm-com]

This pull request introduces 2 alerts when merging db6c492 into 023229b - view on LGTM.com

new alerts:

• 2 for Multiplication result converted to larger type

[MaNGusT-]

Composed all latest work as Past-due asked for https://drive.google.com/file/d/1cKRjLujumzo7x_SvBZ0BeIGHdm6cIdJ_/view?usp=sharing
You can delete specular maps to test performance on gpu with less than 1gb video memory.

[past-due]

Force-pushed another rebase to fix a merge conflict. (Thanks for the terraindepth catch, @amelentev!)

[lgtm-com]

This pull request introduces 2 alerts when merging a1dce8f into f31c930 - view on LGTM.com

new alerts:

• 2 for Multiplication result converted to larger type

[lgtm-com]

This pull request introduces 2 alerts when merging 8ba4abe into 1fc08cd - view on LGTM.com

new alerts:

• 2 for Multiplication result converted to larger type

[MaNGusT-]

Pre-final or even final one. I'm pretty happy with look of terrain textures but some decals like roads require too much time and probably complete redraw. Did only basic editing for most of the decals.
https://drive.google.com/file/d/1AEzerFYMczJDu0KWsc8a2tmKv-S6VXZq/view?usp=sharing

EDIT: Final - I mean that it's ready to merge. Without parallax mapping at this state. Later, from time to time, I (or someone else) will update textures which I think can be enhanced.

Also, as I understand we don't have a way to change texture quality right. As Past-due said, new textures should be packed into mod with its shaders and have a button in menu to swap through available options. Or use this textures by default and pack old textures into LQ mod.

[MaNGusT-]

What else could be done to raise up quality is to move central vertex of each crater down a bit for all craters. At least for campaign...

look how differ craters. @KJeff01 ? ^.^
Small craters also have a central vertex that can be moved down.

[amelentev]

Here is an experimental terrainDecails.vert/.frag combined shader:
diffs: amelentev/warzone2100@2019_gfx_api_work_20_terrain_normals...amelentev:2019_gfx_api_work_20_terrain_normals-decalTexArr
builds: https://github.com/amelentev/warzone2100/actions/workflows/CI_windows.yml?query=branch%3A2019_gfx_api_work_20_terrain_normals-decalTexArr
@MaNGusT- please check it out. I think it has everything you need.
Looks about the same:

Some details:

• It is based on this PR. Maybe will be merged here (when ready), need testing.
• The shader is opaque, one pass. replacement for terrainDepth, terrain (8+ passes), and decal shaders.
• Decal uv borders are removed, uv are [0, 1] inclusive, without shifts (shifts/borders are needed for old decals shader because of texture atlas texture bleeding. no atlas here).
• Currently, only the new shader is used, but old shaders are still working, so some switch is possible. Maybe we will move all normalmapping to this new shader and leave old shaders as is?
• No vulkan support yet.
• The shader uses texture arrays, which is 2008 year tech, EXT_texture_array.
• No texture reloading button yet. But the shader recompile button was added.
• Currently, decals textures are just alpha-blended with ground in the shader. But we can blend however we want. The problem is decal light calculation is different than ground light (different tangents). If we will unite them we can blend normals, etc.

[MaNGusT-]

Looks exactly as I wanted and as was shawn in link you provided. Will look into it soon. Thanks!

[past-due]

• The shader is opaque, one pass. replacement for terrainDepth, terrain (8+ passes), and decal shaders.
• Decal uv borders are removed, uv are [0, 1] inclusive, without shifts (shifts/borders are needed for old decals shader because of texture atlas texture bleeding. no atlas here).

Awesome. 🥳

• The shader uses texture arrays, which is 2008 year tech, EXT_texture_array.

My understanding (and please correct me if I'm wrong) is that would mean bumping minimum requirements on the GL side to:

• OpenGL 3.0+ (or OpenGL 2.1 with the EXT_texture_array extension)
• OpenGL ES 3.0+

While OpenGL 3.0+ is probably doable, I'm a bit more concerned about bumping OpenGL ES from 2.0 to 3.0 as the minimum requirement at this point in time.

(Main source of concern: libANGLE requires DirectX 11 to translate OpenGL ES 3.0, while only DirectX 9 is required for OpenGL ES 2.0. And we use libANGLE as a fallback on Windows when Windows OpenGL drivers aren't great, often with older hardware.)

Also, we may not be able to bump above OpenGL ES 3.0 as a base minimum requirement at this time (because of libANGLE, again, but I believe OpenGL ES 3.0 has the 2d texture array support needed? @amelentev)

• Currently, only the new shader is used, but old shaders are still working, so some switch is possible. Maybe we will move all normalmapping to this new shader and leave old shaders as is?

Theoretically we could then maintain compatibility with OpenGL 2.1 and OpenGL ES 2.0, and only offer the new terrain rendering if a system meets the minimum requirements.

Just need a switch, and a bit of GL init-time detection to set a flag as to whether 2d texture arrays (and, thus, the new terrain rendering shaders) are available.

That would basically solve the compatibility concerns. 👍

• No vulkan support yet.

And obviously we need this too 😁, but overall this is very exciting (if we can just maintain that backwards compatibility).

[MaNGusT-]

For unknown reasons it doesn't work on my system. Tried portable build 5921ec5. Win10, gtx1080ti, ogl

log file is empty

[amelentev]

@MaNGusT- it seems the issue is in tileNo. It should be positive for decal textures and negative for pre-decal tiles. So it is possible to implement that classic tile-only look (texture2DArray(decalTex, vec3(uvDecal, abs(tile)))). Unfortunately, something is wrong with signed integer passing to the shader, so I've set it 0 for pre-decal tiles. Should fix the issue. Build: https://github.com/amelentev/warzone2100/actions/workflows/CI_windows.yml?query=branch%3A2019_gfx_api_work_20_terrain_normals-decalTexArr
It is strange it was not reproduced for me with your textures, only with vanilla textures.

[amelentev]

About backward compatibility. I don't know much about OpenGL ES.
But yes, I plan to add some switch (same terrainShaderQuality?). So this new shader code will be isolated and won't run then the switch is off.
It should be possible to check we have the right hardware.

[MaNGusT-]

* float(lambertTerm > 0) doesn't work and can't, tested it alot in wmit.
I know that it's improper way to fix "simplness" of current specular light but multiply it by lambertterm works well till ssao will be added. It needed to lower to 0 specular light at sharp angles of view on parts near end of diffuse light and make it smoother.
Specular hard edge happens due to very low poly meshes we use for models and terrain.

[amelentev]

• No vulkan support yet.

Implemented

• Currently, decals textures are just alpha-blended with ground in the shader. But we can blend however we want. The problem is decal light calculation is different than ground light (different tangents). If we will unite them we can blend normals, etc.

Now the shader blends everything (color, normal, gloss) of ground and decals. And bumpmapping is performed only once on resulted blended data: amelentev@de8c1cf

[MaNGusT-]

Works good but we still can't split cliff decals from terrain decals to apply different blending. I have no idea how to implement this with current terrain engine.
I think that I just need to completely re-draw each cliff decal. (

[MaNGusT-]

zip! tested on latest availble build(de8c1cf). Mostly reduced size of all textures and updated water.

* float(lambertTerm > 0) this IF does nothing (practically) - see no changes at all. But spec light of terrain, decals(not sure) and units have to be darkened by diffuse light(specularLight*gloss*gaussianTerm*lambertTerm;) to make overall look a bit "smoother". Ya know that it's a "hack" but I've seen it even in source code of doom3. We could keep this for sometime.
Overlay formula used for normal maps of water I took from here
All changes from zip are necessary.

[vaut]

Thanks a lot for your work.
I have a comment on the brightness of the snow. It is much brighter than the original.

And you can work more on the cliffs.
Now in releases with normals, they look no better than in the original render from 2.3.x versions.

[MaNGusT-]

It is much brighter than the original.

actually it's darker, but now it reflects the light. May be we should set spec light to zero for terrain to exclude this "wet" effect. idk

And you can work more on the cliffs.

Could

[vaut]

I like the way snow looks. Very realistic snow on a sunny day. But on such bright ground it will be very difficult to play.
Thanks a lot for your work.

[MaNGusT-]

update for terrain_all5.zip
contains only changed textures:
-darkened textures with snow
-reverted cliffs

[vaut]

Thanks for the improved snow.

Probably I poorly described what I expect from the cliffs.
In the old old versions (2.3.x) the cuts were sharp and clear.



When updating the render, they were replaced with weak blurry stripes.

1. stripes in new textures in a different direction.
   Instead of vertical water-washed ravines or landslide destruction, we have horizontal stripes in the style of dolomites.
2. The textures of neighboring squares completely dominate the cliff. Cliffs are read only by the height difference and then with difficulty. .
   
   
   

[MaNGusT-]

I understand what you are talking about, but currently I see no way to make them look good from each side. I mean that they look good on shaded side and bad on lightened.

The best way to create and variate cliffs is using feature objects - example

P.S. Well, I have only theoretical solution, but it's only shader related:

1. must sure that cliff decal uses vertex normals of terrain tile it's placed on(not terrain texture normals)
2. texture normals of cliff decal have to be mixed up with vertex normals of terrain tile via overlay blending (as I did for water)

[MaNGusT-]

Also, there is bug with cliff decals. Some decals were rotated by map designer, in game this results that normal maps of these decals are also rotated what breaks lightning.

[MaNGusT-]

Found a way to make ''adequate'' cliffs.

Currently look like classic. wip

[past-due]

Superseded by #3127, which has now been merged! 🥳

Huge thanks to @amelentev for all of this amazing work.