LinuxRendererGLES: implement hq scalers #12474

lrusak · 2017-07-12T02:13:33Z

This is not working properly yet. It can run with the fbo but the video does not get scaled properly, AKA. a 1280x720 video will only be 1280 pixels wide in a 1920 display. Everything including the gui isn't scaled properly so maybe that is a clue.

I haven't been able to track down the problem yet.

xbmc/cores/VideoPlayer/VideoRenderers/FrameBufferObject.cpp

@@ -53,10 +53,14 @@ CFrameBufferObject::CFrameBufferObject()

 bool CFrameBufferObject::IsSupported()
 {
+#if HAS_GLES >= 2


xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.cpp

@@ -75,12 +76,70 @@ BaseVideoFilterShader::BaseVideoFilterShader()
    "gl_FragColor.a = gl_Color.a;"
    "}";
  PixelShader()->SetSource(shaderp);
+#elif HAS_GLES >= 2


xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.cpp

      m_internalformat = GL_RGBA;
+#elif HAS_GLES >= 2


xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.cpp

@@ -152,6 +217,8 @@ ConvolutionFilterShader::~ConvolutionFilterShader()

 void ConvolutionFilterShader::OnCompiledAndLinked()
 {
+  BaseVideoFilterShader::OnCompiledAndLinked();


yol · 2017-07-12T12:00:20Z

@lrusak just cherry-picked this for wayland (yol/xbmc@1da938569), same problem - so it's not gbm. whole UI gets drawn at the video resolution when I switch to a HQ scaler. Switching back does not fix it.

FernetMenta · 2017-07-12T18:55:06Z

btw: you get a much better performance if you scale the yuv image first, then do conversion to rgb.

xbmc/cores/VideoPlayer/VideoRenderers/LinuxRendererGLES.cpp

@@ -873,7 +873,7 @@ void CLinuxRendererGLES::RenderToFBO(int index, int field, bool weave /*= false*
      return;
    }
    //if (!m_fbo.fbo.CreateAndBindToTexture(GL_TEXTURE_2D, m_sourceWidth, m_sourceHeight, GL_RGBA))
-    if (!m_fbo.fbo.CreateAndBindToTexture(GL_TEXTURE_2D, m_sourceWidth, m_sourceHeight, GL_RGBA, GL_SHORT))
+    if (!m_fbo.fbo.CreateAndBindToTexture(GL_TEXTURE_2D, m_sourceWidth, m_sourceHeight, GL_RGBA16_EXT, GL_SHORT))


koying · 2017-07-13T10:11:00Z

@lrusak You miss https://github.com/xbmc/xbmc/pull/12357/files#diff-41e0ceb3de6f45669f617deae0dbeb0cR1638 for whatever reason.
That's the cause of the "corner" issue.

lrusak · 2017-07-13T16:46:58Z

@koying thanks, that indeed fixes the issue.

xbmc/cores/VideoPlayer/VideoRenderers/LinuxRendererGL.cpp

@@ -1109,6 +1109,12 @@ void CLinuxRendererGL::RenderToFBO(int index, int field, bool weave /*= false*/)

  if (!m_fbo.fbo.IsValid())
  {
+    if (!g_Windowing.IsExtSupported("GL_EXT_framebuffer_object"))


xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.h

    virtual void SetSourceTexture(GLint ytex) { m_sourceTexUnit = ytex; }
    virtual void SetWidth(int w) { m_width  = w; m_stepX = w>0?1.0f/w:0; }
    virtual void SetHeight(int h) { m_height = h; m_stepY = h>0?1.0f/h:0; }
    virtual void SetNonLinStretch(float stretch) { m_stretch = stretch; }
    virtual bool GetTextureFilter(GLint& filter) { return false; }
+#if HAS_GLES >= 2
+    virtual GLint GetVertexLoc() { return m_hVertex; }


lrusak · 2017-07-14T04:05:20Z

Updated, this now works properly with all scalers. This should be tested with GL also as it changes the format used for the 6x6 convolutions to use one with an alpha channel.

FernetMenta · 2017-07-14T05:16:37Z

before testing with openGL you should address my comments. It won't have a change to get in with those ifdefs.

lrusak · 2017-07-14T05:18:08Z

@FernetMenta yes of course. This is still very much a WIP. I'd like to clean it up as much as possible.

xbmc/cores/VideoPlayer/VideoRenderers/LinuxRendererGLES.cpp

+    pYUVShader->SetField(1);
+  else if(field == FIELD_BOT)
+    pYUVShader->SetField(0);
+


lrusak · 2017-07-19T03:19:20Z

@FernetMenta I've updated to remove much of the ifdefs. I hope that is better!

lrusak · 2017-07-19T06:30:20Z

jenkins build this please

xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.h

@@ -57,11 +57,41 @@ namespace Shaders {
    GLint m_hStretch = 0;
  };

+  class BaseVideoFilterShaderGLES : public BaseVideoFilterShaderGL


xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.h

+  };
+
+#if HAS_GLES >= 2
+  using BaseVideoFilterShader = BaseVideoFilterShaderGLES;


xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.cpp

@@ -77,49 +77,116 @@ BaseVideoFilterShader::BaseVideoFilterShader()
  PixelShader()->SetSource(shaderp);
 }

+BaseVideoFilterShaderGLES::BaseVideoFilterShaderGLES()


lrusak · 2017-07-19T21:53:34Z

I'm not going to split the file. I think that is counterproductive. If OpenGL breaks OpenGLES then it will get fixed. If it was split then OpenGLES will be forgotten about.

I'm not happy about the VAAPI change but it was the only way forward. The RendererVAAPIGL.cpp and RendererVAAPIGLES.cpp are the exact same, there is no reason for them to be split other than the fact that that's how you wanted them to be.

In my opinion the path forward is to make the OpenGL code compatible with OpenGLES. This should be done by only using OpenGLES compatible calls.

FernetMenta · 2017-07-20T06:27:18Z

Without splitting the file this won't get merged. No discussion about this. You can do by yourself or wait until I'll find time to do so. The latter will delay this PR.

xbmc/cores/VideoPlayer/VideoRenderers/LinuxRendererGL.cpp

-  glMultiTexCoord2fARB(GL_TEXTURE0, 0.0f, 0.0f);
-  glVertex4f(m_rotatedDestCoords[0].x, m_rotatedDestCoords[0].y, 0, 1.0f );
+  GLint vertLoc = m_pVideoFilterShader->GetVertexLoc();
+  GLint loc     = m_pVideoFilterShader->GetcoordLoc();


xbmc/cores/VideoPlayer/VideoRenderers/LinuxRendererGL.cpp

+  }
+
+  // Setup texture coordinates
+  tex[0][0] = tex[3][0] = 0.0f;


xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.cpp

@@ -56,70 +56,106 @@ BaseVideoFilterShader::BaseVideoFilterShader()

  m_stretch = 0.0f;

+  m_hVertex = -1;
+  m_hcoord = -1;
+  m_hProj   = -1;


lrusak · 2017-09-08T02:31:15Z

Updated to conform to @FernetMenta 's requests.

lrusak · 2017-09-08T05:08:16Z

jenkins build this please

xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShaderGLES.cpp

+
+  if (m_floattex)
+  {
+    m_internalformat = GL_RGBA16F_EXT;


lrusak · 2017-09-08T18:20:05Z

I'm assuming android, rbpi, aml, and imx don't use LinuxRendererGLES can anyone confirm?

FernetMenta · 2017-09-08T18:45:50Z

android does use it: https://github.com/xbmc/xbmc/blob/master/xbmc/cores/VideoPlayer/VideoRenderers/HwDecRender/RendererMediaCodec.h#L29

koying · 2017-09-08T19:00:19Z

I'm assuming android, rbpi, aml, and imx don't use LinuxRendererGLES can anyone confirm?

Well, all GLES platforms use it in ffmpeg/software mode, really (besides maybe rpi)

lrusak · 2017-09-08T19:24:06Z

Well, all GLES platforms use it in ffmpeg/software mode, really (besides maybe rpi)

You're right, if I disable mediacodec on android I can use the HQ scaler methods.

I've updated to not build only on RPi

xbmc/cores/VideoPlayer/VideoRenderers/CMakeLists.txt

-                       NOT CORE_PLATFORM_NAME STREQUAL aml AND
-                       NOT CORE_PLATFORM_NAME STREQUAL android AND
-                       NOT CORE_PLATFORM_NAME STREQUAL imx))
+if(OPENGLES_FOUND AND NOT CORE_PLATFORM_NAME STREQUAL rbpi)


xbmc/cores/VideoPlayer/VideoRenderers/CMakeLists.txt

+   CORE_PLATFORM_NAME STREQUAL gbm OR
+   CORE_PLATFORM_NAME STREQUAL imx OR
+   CORE_PLATFORM_NAME STREQUAL mir OR
+   CORE_PLATFORM_NAME STREQUAL wayland)


lrusak · 2017-09-08T20:16:04Z

Updated with the best possible solution that our build system provides.

lrusak · 2017-09-09T18:30:49Z

Rebased and squashed. Would be nice if I could get an ok from other GLES platform maintainers. ping @koying @Memphiz @peak3d

Memphiz · 2017-09-09T19:58:49Z

I have not the slightest amount of time for testing this currently. But given the fact that the last test definitely didn't break anything, and I am pretty sure this is still the case in the current state of this PR - you have my OK ;)

lrusak · 2017-09-11T23:10:30Z

jenkins build this please

Rechi · 2017-09-12T08:29:30Z

jenkins build this please

xbmc/cores/VideoPlayer/VideoRenderers/CMakeLists.txt

+                       CORE_PLATFORM_NAME STREQUAL gbm OR
+                       CORE_PLATFORM_NAME STREQUAL imx OR
+                       CORE_PLATFORM_NAME STREQUAL mir OR
+                       CORE_PLATFORM_NAME STREQUAL wayland))


lrusak · 2017-09-12T18:31:48Z

squshed, rbpi built successfully. Ready to merge 👍

Memphiz · 2017-09-13T06:00:03Z

Thx for getting it through :)

lrusak added Type: Feature non-breaking change which adds functionality v18 Leia Component: Video WIP PR that is still being worked on labels Jul 12, 2017

lrusak force-pushed the gles-hq-scalers branch from 265e3d9 to 166b395 Compare July 12, 2017 04:39

FernetMenta reviewed Jul 12, 2017

View reviewed changes

xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.cpp Outdated

@@ -75,12 +76,70 @@ BaseVideoFilterShader::BaseVideoFilterShader()

"gl_FragColor.a = gl_Color.a;"

"}";

PixelShader()->SetSource(shaderp);

#elif HAS_GLES >= 2

This comment was marked as spam.

Sign in to view

FernetMenta reviewed Jul 12, 2017

View reviewed changes

xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.cpp Outdated

m_internalformat = GL_RGBA;

#elif HAS_GLES >= 2

This comment was marked as spam.

Sign in to view

FernetMenta reviewed Jul 12, 2017

View reviewed changes

koying reviewed Jul 13, 2017

View reviewed changes

FernetMenta reviewed Jul 13, 2017

View reviewed changes

koying reviewed Jul 18, 2017

View reviewed changes

xbmc/cores/VideoPlayer/VideoRenderers/LinuxRendererGLES.cpp

pYUVShader->SetField(1);

else if(field == FIELD_BOT)

pYUVShader->SetField(0);

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

FernetMenta reviewed Jul 19, 2017

View reviewed changes

xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.h Outdated

@@ -57,11 +57,41 @@ namespace Shaders {

GLint m_hStretch = 0;

};

class BaseVideoFilterShaderGLES : public BaseVideoFilterShaderGL

This comment was marked as spam.

Sign in to view

FernetMenta reviewed Jul 19, 2017

View reviewed changes

xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.h Outdated

};

#if HAS_GLES >= 2

using BaseVideoFilterShader = BaseVideoFilterShaderGLES;

This comment was marked as spam.

Sign in to view

FernetMenta reviewed Jul 19, 2017

View reviewed changes

xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.cpp Outdated

@@ -77,49 +77,116 @@ BaseVideoFilterShader::BaseVideoFilterShader()

PixelShader()->SetSource(shaderp);

}

BaseVideoFilterShaderGLES::BaseVideoFilterShaderGLES()

This comment was marked as spam.

Sign in to view

FernetMenta reviewed Aug 25, 2017

View reviewed changes

xbmc/cores/VideoPlayer/VideoRenderers/LinuxRendererGL.cpp Outdated

}

// Setup texture coordinates

tex[0][0] = tex[3][0] = 0.0f;

This comment was marked as spam.

Sign in to view

FernetMenta reviewed Aug 25, 2017

View reviewed changes

xbmc/cores/VideoPlayer/VideoRenderers/VideoShaders/VideoFilterShader.cpp Outdated

@@ -56,70 +56,106 @@ BaseVideoFilterShader::BaseVideoFilterShader()

m_stretch = 0.0f;

m_hVertex = -1;

m_hcoord = -1;

m_hProj = -1;

This comment was marked as spam.

Sign in to view

lrusak force-pushed the gles-hq-scalers branch from 7c26f28 to e7003b5 Compare September 8, 2017 02:30

FernetMenta approved these changes Sep 8, 2017

View reviewed changes

Rechi reviewed Sep 8, 2017

View reviewed changes

FernetMenta reviewed Sep 8, 2017

View reviewed changes

lrusak changed the title ~~[WIP] LinuxRendererGLES: implement hq scalers~~ LinuxRendererGLES: implement hq scalers Sep 8, 2017

lrusak removed the WIP PR that is still being worked on label Sep 8, 2017

lrusak force-pushed the gles-hq-scalers branch from 81dd6b8 to 3ed6668 Compare September 9, 2017 18:29

Rechi reviewed Sep 12, 2017

View reviewed changes

lrusak added 2 commits September 12, 2017 11:27

[cmake] only build LinuxRendererGLES for specific renderers

93e96e4

[cmake] only build videoshaders for specific renderers

dc4cfa1

lrusak force-pushed the gles-hq-scalers branch from 0f5c5ed to dc4cfa1 Compare September 12, 2017 18:30

lrusak merged commit aa6c18e into xbmc:master Sep 12, 2017

Rechi added this to the L 18.0-alpha1 milestone Sep 12, 2017

stefansaraev mentioned this pull request Sep 12, 2017

[fix] aml build after pull/12474 #12793

Merged

10 tasks

Memphiz mentioned this pull request Sep 15, 2017

GLES: allow using limited color range #12506

Merged

lrusak deleted the gles-hq-scalers branch February 11, 2022 21:24

LinuxRendererGLES: implement hq scalers #12474

LinuxRendererGLES: implement hq scalers #12474

Conversation

lrusak commented Jul 12, 2017

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

yol commented Jul 12, 2017

FernetMenta commented Jul 12, 2017

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

koying commented Jul 13, 2017

lrusak commented Jul 13, 2017

This comment was marked as spam.

This comment was marked as spam.

lrusak commented Jul 14, 2017

FernetMenta commented Jul 14, 2017

lrusak commented Jul 14, 2017

This comment was marked as spam.

This comment was marked as spam.

lrusak commented Jul 19, 2017

lrusak commented Jul 19, 2017

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

lrusak commented Jul 19, 2017

FernetMenta commented Jul 20, 2017

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

lrusak commented Sep 8, 2017

lrusak commented Sep 8, 2017

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

lrusak commented Sep 8, 2017

FernetMenta commented Sep 8, 2017

koying commented Sep 8, 2017

lrusak commented Sep 8, 2017

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

lrusak commented Sep 8, 2017

lrusak commented Sep 9, 2017

Memphiz commented Sep 9, 2017

lrusak commented Sep 11, 2017

Rechi commented Sep 12, 2017

This comment was marked as spam.

This comment was marked as spam.

lrusak commented Sep 12, 2017

Memphiz commented Sep 13, 2017