RetroPlayer: Add renderer for Windows, OpenGL, OpenGLES and MMAL #12715

garbear · 2017-08-25T19:15:17Z

This takes a significant step forward by separating the RetroPlayer and VideoPlayer renderers. The new RP renderer is pretty bare, and just contains enough code to render RGB images delivered from game add-ons.

Motivation and Context

Based on garbear#84

How Has This Been Tested?

Tested on Windows, Linux and OSX. The OpenGLES renderer hasn't been tested yet, but it uses mostly the same code as the OpenGL renderer.

Types of change

Bug fix (non-breaking change which fixes an issue)
Improvement (non-breaking change which improves existing functionality)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

garbear · 2017-08-26T01:01:13Z

Rebased after the merge of #12665. Still TODO are a few remaining steps to fully separate RP and VP, including the separation of CProcessInfo.

xbmc/cores/RetroPlayer/process/RPProcessInfo.cpp

+    m_dataCache->SetRenderClockSync(false);
+    m_dataCache->SetStateSeeking(false);
+    m_dataCache->SetSpeed(1.0f, 1.0f);
+    m_dataCache->SetGuiRender(true); //! @todo


xbmc/cores/RetroPlayer/process/RPProcessInfo.cpp

+    m_dataCache->SetStateSeeking(false);
+    m_dataCache->SetSpeed(1.0f, 1.0f);
+    m_dataCache->SetGuiRender(true); //! @todo
+    m_dataCache->SetVideoRender(true); //! @todo


garbear · 2017-08-26T22:13:13Z

jenkins build this please (to check GLES code)

popcornmix · 2017-08-27T17:06:34Z

Just a warning that rendering through GLES will be significantly less efficient.

Prior to this PR on Pi we will directly render in the game's native bitmap format (e.g. RGB565, RGB888, RGBA32) to a separate layer without any copying required.
Now, then texture upload is effectively a memcpy to GPU memory, followed by a format conversion to a tiled texture format. There may also be an additional format conversion before texture upload to get the image into GL_BGRA format.

A few years ago all ARM platforms rendered video through GL. But performance sucked and now all ARM platforms render video directly on a separate video plane, mostly using zero copy.

I understand that for emulating 3D consoles you need GL rendering (although whether performance will be adequate for this class of emulation is questionable). Also you may want it for shader based scanline effects. But I worry this scheme will rule out a class of emulation that previously worked acceptably but will no longer with the GL renderer.

It is possible we will want to support platform specific (e.g. MMAL) renderers for RP in the same way as we do for VP.

garbear · 2017-08-27T17:45:18Z

Agreed, a MMAL renderer for RetroPlayer would be desirable. Do you have time to implement this? It's pretty much just a matter of copying the VP renderer and removing all the video stuff. If not, I can handle this eventually.

Using MMAL for rendering would preclude the use of shaders, right?

popcornmix · 2017-08-27T17:52:14Z

I can help with MMAL renderer support when I get some time.
Ideally MMAL would be an optional renderer alongside GL.
If game uses GL or custom shaders then GL path is used otherwise use MMAL renderer.

I don't see a good way to use shaders alongside MMAL (you would lose the zero copy path so may as well render with GL as your pixels are already there).

fritsch · 2017-08-27T17:59:28Z

Most ARMs (including IMX) offer some pseudo vendor optimized GL extensions to simulate a zero copy or at least a DMA copy to the 3D GPU. Still that would also be less performant as "passthrough" rendering, but perhaps some custom shader / GL code could be supported that way?

eglCreateImage might be a candidate when doing it in a standard way?

xbmc/cores/RetroPlayer/rendering/VideoRenderers/RPWinRenderer.cpp

+  m_targetPixFormat = AV_PIX_FMT_BGRA;
+  m_targetDxFormat = DXGI_FORMAT_B8G8R8X8_UNORM;
+  if (g_Windowing.IsFormatSupport(DXGI_FORMAT_B8G8R8A8_UNORM, D3D11_USAGE_DYNAMIC))
+    m_targetDxFormat = DXGI_FORMAT_B8G8R8A8_UNORM;


xbmc/cores/RetroPlayer/rendering/VideoRenderers/RPWinRenderer.cpp

@@ -69,7 +69,7 @@ bool CRPWinRenderer::Configure(AVPixelFormat format, unsigned int width, unsigne
  m_renderOrientation = orientation;

  m_targetPixFormat = AV_PIX_FMT_BGRA;
-  m_targetDxFormat = DXGI_FORMAT_B8G8R8X8_UNORM;
+  m_targetDxFormat = g_Windowing.GetBackBuffer()->GetFormat();
  if (g_Windowing.IsFormatSupport(DXGI_FORMAT_B8G8R8A8_UNORM, D3D11_USAGE_DYNAMIC))


garbear · 2017-09-03T20:26:06Z

Updated with fixes from the last week

garbear · 2017-09-07T17:25:46Z

Rebased after #12761, included recent changes from my shaders work and squashed. Just one bit of code in the GLES renderer needed before merging.

xbmc/cores/RetroPlayer/rendering/VideoRenderers/RPRendererOpenGLES.cpp

+  /*
+  glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);
+
+  glBegin(GL_QUADS);


xbmc/cores/RetroPlayer/rendering/GUIRenderTarget.h

+    ~CGUIRenderFullScreen() override = default;
+
+    void Render() override;
+    void RenderEx() override;


garbear · 2017-09-09T19:07:56Z

@popcornmix I've started working on MMAL support. The buffers in RetroPlayer are very basic and can't handle things like buffer pools. I'll let you know when the new buffer architecture is complete so we can get MMAL rendering to work.

One of the features that flew into this PR before the last rebase was the ability to have multiple renderers at the same time. In the GSoC code, renderers are indexed by shader preset and are swapped around as needed. I don't see any reason we can't switch between MMAL and GLES depending on which features they support.

garbear · 2017-09-13T23:21:00Z

@popcornmix I've added a MMAL renderer based on the VideoPlayer code. It needs some more work, but should serve as a good base for MMAL support.

Now that we have a buffer and buffer pool abstraction, I'll port the existing renderers to it so that we can achieve zero-copy support.

xbmc/cores/RetroPlayer/guicontrols/GUIGameControl.h

  CGUIInfoLabel m_viewModeInfo;

-  CGUIRenderSettings m_renderSettings;
+  // Rendering properties
+  bool m_bHasShaderPreset = false;


Rechi · 2017-09-20T09:43:12Z

@garbear this needs a rebase

garbear · 2017-09-24T18:20:53Z

@MilhouseVH try applying this PR to newclock5 now

MilhouseVH · 2017-09-24T19:07:27Z

Thanks, this now builds for both x86.

For RPi (with newclock5), I had to revert fd269e5 and 940050c before applying this PR on top of newclock5, and this now fails to build: http://sprunge.us/NGjJ

garbear · 2017-09-24T19:51:09Z

I applied this PR to newclock5. There was only 1 line that needed to be changed to fix the build breakage you just got. I've pushed the rebased PR to: https://github.com/garbear/xbmc/commits/rp-renderer-pi. Shall I open a PR against newclock5, or can you just cherry-pick the commits?

MilhouseVH · 2017-09-24T20:48:02Z

I'm not sure you need to open a PR against newclock5 as newclock5 already contains the changes you have in this PR (1, 2), hence the two conflicts when applying this PR on top of newclock5.

I'm locally reverting the two commits from this PR that I mentioned previously in order to be able to apply this PR on top of newclock5, and that works for me. Assuming this PR goes in first then @popcornmix can just rebase the newclock5 branch as normal and drop his two commits.

I can cherry-pick garbear@af7aa57 for now. With it, this PR now builds for RPi (on top of newclock5), and x86 also builds so looking good. Not yet run-time tested, but will include in tonight's release if no obvious problems with video playback.

FernetMenta · 2017-09-25T07:28:44Z

@garbear legacy OpenGL aka fixed function pipeline does not use shaders. That means that we have to migrate GUI rendering and picture rendering to profile 3.2. VP does already make use of shaders.

VelocityRa · 2017-09-25T11:58:01Z

@FernetMenta

VP does already make use of shaders.

Yeah it does, but AFAIK making use of shaders doesn't mean that it's not fixed function. The OpenGL renderer (CLinuxRendererGL) is still fixed function right? [1], [2], etc. GLES isn't of course since it's not even available there.

FernetMenta · 2017-09-25T17:03:02Z

@VelocityRa you are right, there are some parts still legacy.
I am working on migration: https://github.com/FernetMenta/kodi-agile/commit/5a7d4d1a0a5164667c90a5581a7d40aa1faea54e
Then I will bump (OpenGL) shaders to >= 1.5

garbear · 2017-09-27T00:13:31Z

I fixed the GLES renderer on the RPi. MMAL isn't ready yet, so it's been disabled. This PR is ready to go in now. @FernetMenta can you sign off on the two guilib changes?

xbmc/guilib/Texture.h

@@ -91,7 +91,7 @@ class CBaseTexture

  virtual void CreateTextureObject() = 0;
  virtual void DestroyTextureObject() = 0;
-  virtual void LoadToGPU() = 0;
+  virtual void LoadToGPU(bool bFreeSysMem = true) = 0;


This splits several enums and a typedef into a new header file, RenderSystemTypes.h. This avoids a dependence on the base rendering system class if external code only uses the type.

This better encapsulates RetroPlayer quirks.

This allows for a single frame to reach the RenderManager after pausing.

This renderer is based on a combination of the old RetroPlayer code and VP's renderer. Incomplete and untested, but a good starting point.

garbear · 2017-09-28T17:38:02Z

@FernetMenta sign off on the updated guilib commit?

garbear · 2017-09-28T17:46:15Z

jenkins build this please

FernetMenta · 2017-09-28T19:09:30Z

@garbear much better. thanks

garbear · 2017-09-28T19:10:44Z

win32 error unrelated

garbear added Component: Games v18 Leia labels Aug 25, 2017

garbear added this to the L 18.0-alpha1 milestone Aug 25, 2017

garbear force-pushed the rp-renderer branch 2 times, most recently from ddc9dac to b6a6b3b Compare August 26, 2017 00:43

garbear force-pushed the rp-renderer branch from d3e972a to 1a5e94d Compare August 26, 2017 21:56

garbear commented Aug 26, 2017

View reviewed changes

xbmc/cores/RetroPlayer/process/RPProcessInfo.cpp Outdated

m_dataCache->SetStateSeeking(false);

m_dataCache->SetSpeed(1.0f, 1.0f);

m_dataCache->SetGuiRender(true); //! @todo

m_dataCache->SetVideoRender(true); //! @todo

This comment was marked as spam.

Sign in to view

afedchin reviewed Aug 29, 2017

View reviewed changes

garbear force-pushed the rp-renderer branch from 663f477 to 8e5b287 Compare August 31, 2017 21:24

garbear force-pushed the rp-renderer branch 3 times, most recently from 1b90589 to 606dfde Compare September 7, 2017 17:23

garbear commented Sep 7, 2017

View reviewed changes

VelocityRa reviewed Sep 8, 2017

View reviewed changes

xbmc/cores/RetroPlayer/rendering/GUIRenderTarget.h Outdated

~CGUIRenderFullScreen() override = default;

void Render() override;

void RenderEx() override;

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

garbear changed the title ~~RetroPlayer: Add renderer for Windows, OpenGL and OpenGLES~~ RetroPlayer: Add renderer for Windows, OpenGL, OpenGLES and MMAL Sep 13, 2017

Rechi reviewed Sep 14, 2017

View reviewed changes

xbmc/cores/RetroPlayer/guicontrols/GUIGameControl.h Outdated

CGUIInfoLabel m_viewModeInfo;

CGUIRenderSettings m_renderSettings;

// Rendering properties

bool m_bHasShaderPreset = false;

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

garbear mentioned this pull request Sep 15, 2017

Add Volume to Game OSD settings #12765

Merged

4 tasks

garbear force-pushed the rp-renderer branch from b79a319 to f1681f8 Compare September 24, 2017 18:18

garbear closed this Sep 24, 2017

garbear reopened this Sep 24, 2017

garbear force-pushed the rp-renderer branch from f1681f8 to 43f33a8 Compare September 24, 2017 19:49

garbear force-pushed the rp-renderer branch from 43f33a8 to f1681f8 Compare September 24, 2017 19:51

garbear force-pushed the rp-renderer branch 2 times, most recently from 7385696 to 4c2bcda Compare September 27, 2017 01:26

FernetMenta reviewed Sep 27, 2017

View reviewed changes

garbear added 7 commits September 28, 2017 10:36

guilib: Add support for Nearest Neighbor filtering

ada3181

guilib: Add support for caching pixel data

235a855

Split render system types into new header

d24b921

This splits several enums and a typedef into a new header file, RenderSystemTypes.h. This avoids a dependence on the base rendering system class if external code only uses the type.

Remove PauseUnpause() function

1ac3234

This better encapsulates RetroPlayer quirks.

Pause game asynchronously

4e67b17

This allows for a single frame to reach the RenderManager after pausing.

RetroPlayer: Add renderer for Windows, OpenGL and OpenGLES

88db54f

RetroPlayer: Add renderer for MMAL

e00cb97

This renderer is based on a combination of the old RetroPlayer code and VP's renderer. Incomplete and untested, but a good starting point.

garbear force-pushed the rp-renderer branch from 9fb157e to e00cb97 Compare September 28, 2017 17:36

garbear merged commit f0ec8aa into xbmc:master Sep 28, 2017

garbear deleted the rp-renderer branch September 28, 2017 19:11

RetroPlayer: Add renderer for Windows, OpenGL, OpenGLES and MMAL #12715

RetroPlayer: Add renderer for Windows, OpenGL, OpenGLES and MMAL #12715

Conversation

garbear commented Aug 25, 2017 • edited Loading

Motivation and Context

How Has This Been Tested?

Types of change

garbear commented Aug 26, 2017

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

garbear commented Aug 26, 2017

popcornmix commented Aug 27, 2017

garbear commented Aug 27, 2017

popcornmix commented Aug 27, 2017

fritsch commented Aug 27, 2017 • edited Loading

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

garbear commented Sep 3, 2017

garbear commented Sep 7, 2017

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

garbear commented Sep 9, 2017

garbear commented Sep 13, 2017

This comment was marked as spam.

This comment was marked as spam.

Rechi commented Sep 20, 2017

garbear commented Sep 24, 2017

MilhouseVH commented Sep 24, 2017

garbear commented Sep 24, 2017

MilhouseVH commented Sep 24, 2017

FernetMenta commented Sep 25, 2017

VelocityRa commented Sep 25, 2017 • edited Loading

FernetMenta commented Sep 25, 2017

garbear commented Sep 27, 2017 • edited Loading

This comment was marked as spam.

This comment was marked as spam.

garbear commented Sep 28, 2017

garbear commented Sep 28, 2017

FernetMenta commented Sep 28, 2017

garbear commented Sep 28, 2017

garbear commented Aug 25, 2017 •

edited

Loading

fritsch commented Aug 27, 2017 •

edited

Loading

VelocityRa commented Sep 25, 2017 •

edited

Loading

garbear commented Sep 27, 2017 •

edited

Loading