Merge @paroj changes while avoiding the use of the RTSS and OgreBites.

[sercero]

Hello @Altren,

I don't expect you to merge this pull request but I want you to review the code since I am having some problems that I am unable to solve:

• UnitTest_RTTLayer: I don't know what happened with this Test, it was working at some point in time and then I changed something that broke it, perhaps you can figure it out.
• UnitTest_TextureAnimations: Nothing shows, I think it suffers the same problem as UnitTest_RTTLayer, it was also working at some point in my code but now it doesn't.
• UnitTest_Layers: I'm lost on this one, don't even know where to start.
• Demo_Colors: In DirectX11 the color selection texture is half the vertical size, quite a mystery. This also happends in @paroj's code, so it might be an OGRE bug.

The good thing is that thanks to the changes made by @paroj these issues got fixed:

• In DirectX11 now the text is working
• In OpenGL3+ now the text is black instead of white
• Demo_MsdfFont now works (except DirectX9 where OGRE crashes because the shader model 4.0 is not supported)

I will try to keep working on these problems, especially UnitTest_RTTLayer not working since at some point it did work...

However I am getting very frustrated with this and need to do something else for the moment.

Best regards,
Guillermo

[Altren]

Working on it.

[Altren]

I'm having crash in all demos 'read access violation. this->mCurrentVertexProgram was nullptr.' when mRenderSystem->bindGpuProgramParameters(Ogre::GPT_VERTEX_PROGRAM, vertexParams, Ogre::GPV_ALL); is colled in registerShader.
Happens with OpenGL and OpenGL3+, works only with DirectX11.

[Altren]

Also I don't have any issues with Demo_Colors. Looks correct (again, DX11 only)

[Altren]

Just in case, right now I'm working on Windows 10, Ogre 1.12.9+ (actually I'm using master branch from 09 Nov 2020). So hopefully we are on similar setups for now. Once we are done with windows I'll test other platforms.

[Altren]

@sercero Ok, I got some results here:
I removed all mRenderSystem->bindGpuProgramParameters calls, those are no longer needed and cause problems. Simply deleted all lines with this call.
Now we have only next issues remaining:

• Render to texture doesn't work for DirectX11 (works with OpenGL and OpenGL3). So UnitTest_RTTLayer, UnitTest_TextureAnimations and UnitTest_Layers are broken because of that.
• Demos and tests with 3D require shaders, but this should be fixed as a separate issue.

By the way UnitTest_Layers works on OpenGL platform, so you can see what to expect as a result.

I'll try to figure out what's wrong with the RTT on DirectX11.

What about 3d models shaders, did you succeeded with any demos without using RTSS? Because there is only one shader required, there are just simple 3d models with texture and nothing else. (I also removed not used particles files)

[Altren]

I'm also going to try latest Ogre from master, since there was at least one related fix.
Edit: Nope, same issues

[sercero]

@Altren that is strange because at first I was not using the bindGpuProgramParameters() but I am glad that you managed to fix it.

I have the shaders for the 3D models, bear in mind that these shaders just apply the texture and nothing else.

I thought it would be best to use the RTSS because UnitTest_Layers uses materials with multiple textures and blending which I don't know how to do in a shader.

Anyway I will submit a PR with the shaders I made.

Regarding UnitTest_RTTLayer in DirextX11 I found a particular behaviour with the solution made by paroj: in DirectX9 and 11 UnitTest_RTTLayer works OK, but in OpenGL and 3+ the small RTT window is flipped vertically

I think you should take a look at what paroj has done because I think there lies the solution to the problem.

I am very upset that the initial changes I made that had UnitTest_RTTLayer working were lost because I was using git improperly and deleted all the changes in error.

[Altren]

but in OpenGL and 3+ the small RTT window is flipped vertically

That's the point of YFlipScale shader parameter, that was removed by paroj, it fixes texture flipping, that happens on OpenGL platforms.
And it seems that the other difference between your and paroj's version is custom shaders usage. May be RTSS creates more complex shader for DirectX11, that handle the RTT issue

[paroj]

but in OpenGL and 3+ the small RTT window is flipped vertically

That's the point of YFlipScale shader parameter, that was removed by paroj, it fixes texture flipping, that happens on OpenGL platforms.
And it seems that the other difference between your and paroj's version is custom shaders usage. May be RTSS creates more complex shader for DirectX11, that handle the RTT issue

the RTSS generated shader uses the modelviewproj matrix. See #194 (comment)

[sercero]

Thanks @paroj I will try to see the shaders generated by RTSS to see how the modelviewproj is being used in those shaders.

This parameter would be set with setNamedAutoConstant(), right?

At one point I was setting the value of YFlipScale like this: setNamedAutoConstant("YFlipScale", OGRE::ACT_RENDER_TARGET_FLIPPING);

In December I will start learning about shaders, I'm tired of not understanding the basics.

[paroj]

the RTSS uses the matrix instead of YFlipScale

this works as (with mat2 for brevity)

mat2(1, 0, 0, -1) * vec2(x, y)

whereas YFlipScale you do

vec(x, YFlipScale*y)

[Altren]

Thank you @paroj . With setNamedAutoConstant("worldViewProj", Ogre::GpuProgramParameters::AutoConstantType::ACT_WORLDVIEWPROJ_MATRIX); DirectX11 started to render into RTT, but FlipY is still required on OpenGL platform. I guess I also need to use some another auto matrix like texture_viewproj_matrix.
Current glsl shader is
void main() { gl_Position = worldViewProj * position; outUV0 = uv0; outColor = colour; }

@sercero I'll finish all changes soon and merge into master.

[Altren]

@paroj That's odd, but even ACT_RENDER_TARGET_FLIPPING is always 1.0 for me. For OpenGL, OpenGL3 and DirectX11

[Altren]

It comes from GLWindow::requiresTextureFlipping that is false. I guess that wrong render target is asked, because material have RTT texture set as regular texture: pass->getTextureUnitState(0)->setTexture(texture->getOgreTexture());
And shader know nothing about renderTarget, that is hidden behind texture.

[paroj]

ah.. ACT_RENDER_TARGET_FLIPPING is -1 when rendering into the RTT not when rendering the RTT itself

[Altren]

@paroj Sorry, I was wrong, and the code I mentioned is not the RTT, rendered into something, that's actually some part of the GUI skin rendered into RTT.
So what we do is render some parts of Gui into render target. And even without ever displaying that render target on screen it have upside-down content on OpenGL (testes by saving it into file).
The texture is created as TU_RENDERTARGET.

[Altren]

I feel, that I miss some Ogre's API, because I can make it work by hacking into AutoParamDataSource:
const_cast<Ogre::AutoParamDataSource*>(Ogre::Root::getSingleton().getSceneManager("BaseSceneManager")->_getAutoParamDataSource())->setCurrentRenderTarget(renderTarget);
Do I need to tell Renderable where we are rendering?

[paroj]

you are right, we should be actually using SceneManager::manualRender instead of SceneManager::_injectRenderWithPass, so the rendertarget gets updated as well. I missed that in my initial patch.

[Altren]

I tried something like this

auto camera = viewport->getCamera();
mSceneManager->manualRender(&mRenderable, pass, viewport, camera->getViewMatrix(), camera->getProjectionMatrix());

But it is extremely ugly (and probably slow), and actually doesn't work, because we can't get camera info before this frame.

[sercero]

@Altren check these forum threads, perhaps they will be helpful to you:

• Help,Sinbad!!!!SceneManager::manualRender to RTT problem!
• [Solved] MultiRenderTarget output always black
• DrawPrimitive fail?
• _setProjectionMatrix(mProjectMatrix) replacement

[Altren]

@sercero @paroj Thank you for the help making it all better.
I'm done with making manualRender to work and pushed hacky version into master, but at least it works.
If someone wants to make manualRender work there is ogre_fixes branch, where all data is passed to the manualRender call, but it crashes due to mCurrentRenderable being null. Even when calling this mSceneManager->manualRender(&mRenderable, pass, viewport, Ogre::Affine3::IDENTITY, Ogre::Matrix4::IDENTITY);
@paroj it would be great if you'll find some time to play with ogre_fixes branch, but its OK if you don't.

[Altren]

Fixed in master

[Altren]

And even after doing all this changes OpenGL platforms complain about shaders every frame:

MyGUI_Ogre_FP.glsl
 GLSL link result : 
Validation warning! - Sampler value sampleTexture has not been set.
Validation successful.

13:35:24: Error prior to using GLSL Program Object : GL_INVALID_OPERATION
Validation successful.

[sercero]

@Altren see if you can check with this program the OpenGL function calls.

I am checking it on my side

GLIntercept

[paroj]

where all data is passed to the manualRender call, but it crashes due to mCurrentRenderable being null.

fixed in OGRECave/ogre@d06daf6