EMI: On RPI, missing transitions in introduction and video stucks after it

[lmerckx]

Hello,

I'm testing EMI (French version) on a Raspberry PI 2 (but same problem with PI 3).
I'm using a OpenGL ES 2 driver with hardware acceleration. Here are some information for more details:

OpenGL maximum texture size: 2048
OpenGL extensions: GL_OES_compressed_ETC1_RGB8_texture GL_OES_compressed_paletted_texture GL_OES_texture_npot GL_OES_depth24 GL_OES_vertex_half_float GL_OES_EGL_image GL_OES_EGL_image_external GL_EXT_discard_framebuffer GL_OES_rgb8_rgba8 GL_OES_depth32 GL_OES_mapbuffer GL_EXT_texture_format_BGRA8888 GL_APPLE_rgb_422 GL_EXT_debug_marker 
OpenGL GLSL version: 100
OpenGL: GLES2 context initialized
OpenGL: NPOT texture support: 1
OpenGL: Shader support: 1
OpenGL: FBO support: 1
OpenGL: Packed depth stencil support: 0
OpenGL: Unpack subimage support: 0
INFO: OpenGL Vendor: Broadcom
INFO: OpenGL Renderer: VideoCore IV HW
INFO: OpenGL Version: OpenGL ES 2.0
INFO: OpenGL Red bits: 8
INFO: OpenGL Green bits: 8
INFO: OpenGL Blue bits: 8
INFO: OpenGL Alpha bits: 8
INFO: OpenGL Z buffer depth bits: 24
INFO: OpenGL Double Buffer: 1
INFO: OpenGL Stencil buffer bits: 8

There are some minor glitches during the introduction:

• CD icon missing when skipping introduction
• transitions missing: Lucasarts logo transforms to monkey without fading, turning page animation missing, missing background page when opening menu, ...

But the major problem is that the graphics stuck after the introduction. The sound go on, the dialog propositions are displayed but no image.
Could it be a problem with the GLES library ? Some missing features ?

Thank you in advance.

[lmerckx]

Hello,

I'm still trying to solve this problem. So, I added some glGetError() after each GL call to get more information.
I don't know if this is the cause of the problem but the glDrawElements(GL_TRIANGLES,3 * face->_faceLength, GL_UNSIGNED_INT, 0) of drawEMIModelFace() (gfx_opengl_shaders.cpp) exits with error code GL_INVALID_ENUM.

After some searches, I found that GLES v2 doesn't support GL_UNSIGNED_INT (only BYTE and SHORT).
So, I tried to modify the format of the indices table but now, it crashes with error GL_OUT_OF_MEMORY.

Perhaps, still a structure problem ... but, if any idea ?

[Botje]

Good investigative work! Did you also update the corresponding methods?
A quick run-through:

• gfx_opengl_shaders.cpp line 2028, uint32 -> uint16
• gfx_opengl_shaders.cpp line 550: int -> uint16
• modelemi.cpp line 39->41 int -> uint16

I think it should work after changing those. Don't think there are any models with more than 65k vertices.

[lmerckx]

Hello @Botje and thank you for your response.

Yes, it was the changes I did, excepted that I used short instead of int.
I've just tested again with uint16 as you propose but it returns the same error: 505 (GL_OUT_OF_MEMORY).

When googling the problem, I find some references about a problem with complex shaders on RPI.
Is it possible to compile ResidualVM with GLESv2 support but without shaders ?

[Botje]

No, I don't think there is a way to compile with GLESv2 but without shaders.
Does running the program under gltrace reveal anything?
The EMI shaders are not much more complex than the Grim shaders, so I highly doubt they are the cause of the problem.

[Botje]

Another idea: can you simplify the fragment shader? For example, the body of main in the emi_actor.fragment shader can be simply outColor = vec4(1.0f, 1.0f, 1.0f, 1.0f).
That should render all characters as pure white outlines.
If that works we know the indices stuff is good.

[lmerckx]

@Botje Thank you. I've just tested your proposal.
I don't have "out of memory" anymore but a GL_INVALID_OPERATION when calling glUseProgram in Shader::use (shader.cpp).

So, I still have some glitches during the introduction but the game doesn't stuck anymore. In the game, Guybrush is now replaced by a strange blue transparent square.

[lmerckx]

@Botje Hello Botje and sorry for my long silence ...

I've just restarted to analyze this problem from the start and I missed something; because now, with your recommendations, I have no GL error anymore.
But I have a white screen transition between the scenes of the introduction and, in the game, Guybrush is completely blank (no texture, no color).

So, what you expected in fact !

Does that mean that the shaders are too complex for the PI ?
Is there a compromise between the actual emi_actor.fragment and the super-simplified you mentioned above ?

Thanks :)

[Botje]

The most probable culprit are the final few lines:

if (outColor.a < alphaRef)
    discard;

Remove these from the full shader and try again?
If that still fails, also remove the line with outColor.a *= meshAlpha;.
Let me know if one of these two work for you.

[lmerckx]

@Botje Bad news.
I tried what you proposed but the "out of memory" error still persists !

Then, I tested to delete each line one by one and kept only this minimal code:

void main()
{
        outColor = Color;
        if (textured) {
                vec4 texColor = texture(tex, Texcoord);
        }
}

But the error is still there.

Perhaps, is it the texture which is too complex or incompatible with the PI :(

[Botje]

So it does work if you only do the outColor = Color;? Then we need to look at texture coordinates.

[lmerckx]

No, same behavior when keeping only outColor = Color; :

Out of memory error on glDrawElements method.

[lmerckx]

@Botje Hello again ...

After a while, I'm back on this problem and have some new feedback.
The problem is not in emi_actor.fragment but in emi_actor.vertex.

When I remove the whole block if (lightsEnabled) { ... }, it works as expected !
Obviously, the rendering is very simple but it can help !

I will go on analyzing this code to get the exact line of the crash. But if you have any idea, I'm interested !

[lmerckx]

The crash occurs in the last instruction
Color *= vec4(light, 1.0);

When removing it, it works.
When replacing by Color *= vec4(1.0, 1.0, 1.0, 1.0), it works.

When replacing by Color *= vec4(light.x, light.y, light.z, 1.0), it crashes too ...

It seems that the light variable is corrupt but don't know why.

[Botje]

Can you try limiting the for loop for the lighting to just one iteration?

[lmerckx]

I've just tested.
It works with one and two iterations, but crashes at the 3rd.

But note that with only two iterations, Guybrush is really dark.

[Botje]

The proper fix is to investigate why some lights (here the third) produce this crash, but the quick fix is probably to disable lighting entirely on the Raspberry Pi until somebody figures it out. What do you think?

[lmerckx]

Yes, it is probably better.

Because, some new tests seem to indicate an instability in that code for the PI.
For example:

1. for (int i = 0; i < 2; ++i) {
2. for (int i = 0; i < maxLights; ++i) { if (i >= 2) continue;

Code 1 works but code 2 crashes ! :(

I will check to do that and will propose you a merge request.

[lmerckx]

@Botje Great news !
I finally discover and solve all the problems with the Raspberry Pi and its version of OpenGLES2.

Here is a list of all limitations and changed I made:

• As explained above, glDrawElements doesn't support GL_UNSIGNED_INT on GLES2. So, I replaced it by GL_UNSIGNED_SHORT and updated associated structures (Vector3int, _indexes, ...)
• Another limitation is the non-support of array of structures in uniforms. OpenGLES2 supports arrays, structures but not array of structs. In that case, only the first element of the uniform is initialized.
  It is the explanation of why Manny is so dark in Grim Fandango on the PI. To solve the problem, I splitted the array of Light in 4 distinct arrays in the shader.
• Finally, the last limitation, which causes the OutOfMemory and only on the PI (I guess) is the complexity of the shaders: conditional branching in shaders seem very badly implemented by the GPU.
  So, the advice is to split a complex shader in multiple simpler ones (ex: one with light, one without, ...).
  Following this recommendation, I split emi_actor.vertex in 3: one for actor with lights enabled, another for actors with lights disabled and the last one is for sprites (on billboard).
  For GRIM: the two first shaders are enough.

You can check this commit which contains all modifications: https://github.com/lmerckx/residualvm/commit/28b424bc8bcdb42c3242a6e0786559d12e10d63c

For information, I've tested my modifications on Raspberry PI 2, Raspberry PI 3 (both OpenGLES2), Odroid XU 4 (OpenGLES3), X86-64 (OpenGLES3, OpenGL without shaders and TinyGL) and it works for any platforms.

I'm pretty sure that the same modifications should be done for stark engine too. But I will wait your feedback and review before working on it.

A big thanks for your help and support ...

[Botje]

Wow, excellent work! Can you launch that as a pull request for easier review? The int to uint changes can be applied immediately I think, but I'm a bit more wary about the shader changes.
Additionally, can you point me to any sources for limitations you found? Someone on the forums is dealing with shader issues on Stark as well.

[lmerckx]

Yes, no problem. I'll make a pull request.

Here are some sources for problems and limitations:

• For the accepted parameters of glDrawElement in OpenGLES2: http://docs.gl/es2/glDrawElements
• For array of structures, the chapter "Why not an Array of Structures?" of this article made me doubt: http://pyopengl.sourceforge.net/context/tutorials/shader_7.html
  It is for Python but when I debugged the call of glGetUniformLocation, it returned -1 for all uniforms of type "lights[%u]._whatever" except for index 0.
• Finally, here is the post about the OutOfMemory and branching problem on PI (last post is the most important): https://www.raspberrypi.org/forums/viewtopic.php?t=19616