Offsetting UV in the fragment shader is stretching textures on Android GLES2

[williamd1k0]

Godot version: 3.1.1-stable

OS/device including version:

• Android 7.1.2 GLES2 (Mali-400 MP)
• Android 9 GLES2 (Adreno-509)

Issue description:
A friend of mine was doing some experiments with visual shaders, adding TIME to UV.x to simulate texture scrolling.

The equivalent shader code:

void fragment() {
	vec2 uv = UV;
	uv.x += TIME;
	COLOR = texture(TEXTURE, uv);
}

I'm not a shader expert, so I don't know if this code is safe, but it works fine on GLES3 (Windows and Android Adreno-509) and GLES2 on Windows.

Running this shader on Android using GLES2 (Mali-400 MP and Adreno-509) stretches the texture as TIME increases.
So the more TIME increases, the more the texture stretches.

Tests on Android GLES2

First frame

After a few frames

---

I thought it was some issue with UV.x being greater than 1.0f, so I tried the following code:

void fragment() {
	vec2 uv = UV;
	uv.x = mod(uv.x+TIME, 1f); // This should ensure that uv.x never reaches 1
	COLOR = texture(TEXTURE, uv);
}

But it also causes the same stretch bug on Android GLES2.

---

So, is this an actual bug?

Steps to reproduce:
Run the minimal reproduction project on Android with GLES2.

Minimal reproduction project:
UvOffsetTest.zip

[lawnjelly]

At a first guess, this is something which isn't immediately obvious about a lot of GLES 2 devices - I have found on a lot of them you shouldn't touch the texture coords in a fragment shader. It is actually to do with precision.

I wrote a bit about it here:
https://www.gamedev.net/blogs/entry/2264243-android-build-and-performance/
https://www.gamedev.net/forums/topic/694188-debugging-precision-issues-in-opengl-es-2/

If you have a look at this screenshot you will see I'm outputting the GL precision of the device for debugging. Each device will have a bit depth for low, medium and hi precision for vertex shaders, and for fragment shaders. On desktop precision is usually always a high bit depth, but on actual android devices precision is often lower, especially in fragment shaders. It is often 10 bits of precision.

If you work it out that gives you a maximum precision for texture coordinates of 1024 different positions, which is not even enough to give sub-texel accuracy. So what is going on?

What I believe happens is that on these devices they follow a different optimized high precision path, which can offer good texture resolution, until you touch the texture coords in the fragment shader. Once you do that it has to revert to the fragment precision, which may only be 10 bit. And once this happens you get something that looks like point filtering, and all kinds of crazy stuff.

So the upshot is if you want to make your shaders compatible on the most GLES2 devices possible, don't touch the texture coords (or perhaps even some other calculations) in the fragment shader.

If this is indeed the problem here, then it is not strictly speaking a godot bug, more a 'feature' of GLES2 devices. Unfortunately one of the big problems of GLES hardware is that it is like the wild west, with different combinations of hardware caps, hopefully with vulkan that situation might get better (what we would prefer as developers are several tiers of hardware caps, rather than random combinations).

What can we do godot side to help deal with this?

• Well mentioning this somewhere in the docs would be good.
• In addition I've found it very useful to ship some kind of diagnostics as part of the game .. i.e. press some special key combo and you can show diagnostics screen with GL caps etc.

I don't know whether this is something that other devs would consider a good idea, or instead rely on users downloading a second diagnostic app to check caps, or online hardware databases (I don't know if https://opengles.gpuinfo.org currently shows precision caps).

• We could have a warning appear when exporting android projects that have potentially susceptible shaders?

[akien-mga]

Related: GodotVR/godot_oculus_mobile#60

[akien-mga]

Assigning a few GLES2 and shader experts :)

See the analysis in GodotVR/godot_oculus_mobile#60, it does seem to be the same bug.

[akien-mga]

For the reference, I can reproduce the issue with the attached MRP on a Xiaomi Pocophone F1 (Adreno 630) running GLES2. It's a GLES3 and Vulkan-capable device, so I don't think the hardware is to blame here, or at least it's likely that all Android OpenGL ES 2.0 drivers would be affected.

[akien-mga]

So the upshot is if you want to make your shaders compatible on the most GLES2 devices possible, don't touch the texture coords (or perhaps even some other calculations) in the fragment shader.

I can confirm that accessing TIME in the vertex shader instead of the fragment shader works around the issue:

shader_type canvas_item;

uniform bool enabled = true;

void vertex() {
	if (enabled) {
		UV.x += TIME;
	}
}

void fragment() {
	COLOR = texture(TEXTURE, UV);
}

(modified shader code from MRP)

The above works fine on my phone.

[NeoSpark314]

I investigated the issue in GodotVR/godot_oculus_mobile#60; what I found there was that on GLES2 the final fragment shader has at the beginning:

#if defined(USE_HIGHP_PRECISION)
precision highp float;
precision highp int;
#else
precision mediump float;
precision mediump int;
#endif

but USE_HIGHP_PRECISION is not defined on android. (It seems to be only set for JavaScript here:

godot/drivers/gles2/shader_gles2.cpp

Line 180 in ac38f07

strings.push_back("#define USE_HIGHP_PRECISION\n");

).

So all variable declarations and computations without precision modifier default to mediump precision which on most devices is significantly less then highp. (often 10bit mantissa even on new devices)
In the above example with the fragment shader I would assume that declaring the variable as

highp vec2 uv = UV;

would make the computation in high precision and fix the issue on devices where highp is float32.

So one potential solution could be to have an optional "enable high precision" option in visual shader that would declare all variables has highp; an alternative would be a global "use high precision" flag that would #define USE_HIGHP_PRECISION

[lawnjelly]

I've looked through my bookmarks from years ago and amazingly found where I discovered this. It was in a comment in an article on precision. A good read. Enjoy! 😄

https://community.arm.com/developer/tools-software/graphics/b/blog/posts/benchmarking-floating-point-precision-in-mobile-gpus

Recently we have been receiving a lot of questions asking about which of our GPU's handle what levels of precision. Thanks to peterharris for the info.

ARM Mali Midgard range of GPU's:

Shader Cores:
lowp = mediump = fp16
highp = fp32
ARM Mali Utgard range of GPU's:

Geometry Shader Core:
lowp = mediump = highp = fp32
Fragment Shader Core:
lowp = mediump = fp16
highp = not supported*
*We have one special "fast path" for varyings used directly as texture coordinates which is actually fp24.

[Chaosus]

I will check it on my device(Xiaomi Mi Max) soon. If I've not finds this, I cannot help - I cannot make blindly change - it's against my principles.

[Chaosus]

Ah, I see this bug too, will try to fix...

[Chaosus]

I've fairly tried to compile android templates using these instructions http://docs.godotengine.org/en/latest/development/compiling/compiling_for_android.html.
For some reason, its failed and I cannot help without debug build.

[lawnjelly]

So one potential solution could be to have an optional "enable high precision" option in visual shader that would declare all variables has highp; an alternative would be a global "use high precision" flag that would #define USE_HIGHP_PRECISION

Beware here, I think I'd encountered this myself and a quick search suggests this:
"The vertex language requires any uses of lowp, mediump and highp to compile and link without error. The fragment language requires any uses of lowp and mediump to compile without error. Support for highp is optional"
i.e. the Shader might simply not compile on some devices.

However, here:
https://www.khronos.org/opengles/sdk/docs/reference_cards/OpenGL-ES-2_0-Reference-card.pdf

It mentions GL_FRAGMENT_PRECISION_HIGH which is "1 if highp is supported in the fragment language, else undefined", which might be useful.

Also note that setting highp in fragment shader is not a good idea in many devices as even though it may be supported it may be super slow (I think reduz mentioned this too).

I've also read it can be a bad idea to set an overall precision for the whole shader anyway because only certain parts may need higher precision (i.e. it can be better to use the lowest precision necessary for any particular operation).

[akien-mga]

Fixed by #33646.