Add VA/EGL/GLESv2 support #1302

Closed
wants to merge 22 commits into
from

Conversation

Projects
None yet
7 participants

Hi,

This patch series adds support for VA-API over EGL/GLESv2. There are four main parts:

  • Configury preparatory work. Use pkg-config wherever possible. Note that I opted for a mixed approach whereby we first try with pkg-config, so that to allow stacks to be installed into non-standard locations like /opt/something/*, and next fallback to the old-school AC_CHECK_LIB() only macros, that assumes standard search paths or correctly amended INCLUDES/LIBS variables first.
  • Plug a few bugs related to EGL/X11. i.e. fix compilation, fix crash while loading the XBMC icon texture, and fix RGB32 rendering mode. Note for the latter: not that I get a clear crash, I did not try actually; but for 32-bit pixel modes, I'd expect to use depth = 24.
  • Add general purpose rendering of VA surfaces to GL textures through texture-from-pixmap. This is the same approach than for VA/GLX, but is not fully efficient. While effort is made to use a minimally sized Pixmap, this still makes it an indirection. At least this solution works with both the Intel GenX driver (Sandy Bridge, Ivy Bridge et al.) but also with the Intel PowerVR driver for Cedar Trail.
  • Add support for new VA/EGL API that allows for zero-copy rendering from a VA surface. The idea here is to map each individual plane of the VA surface as a texture. This normally makes it possible to re-use existing rendering pipelines with XBMC specific post-processing filters in GLSL. This solution works only with the Intel GenX driver, currently. It requires a patched Mesa.

I will probably split this pull request but I was undecided on the number of parts. While writing this pull request, it became apparent that 4 to 6 parts would probably be needed. Just let me know your preference. :)

Thanks,
Gwenole.

gbeauchesne added some commits Aug 10, 2012

@gbeauchesne gbeauchesne configure: fix check for EGL and GLESv2.
Ensure that the right libraries and includes are added for non-standard
installations of the EGL/GLESv2 stack.

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
b315964
@gbeauchesne gbeauchesne configure: fix check for OpenGL.
Ensure that the right libraries and includes are added for non-standard
installations of the OpenGL stack.

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
d97da5a
@gbeauchesne gbeauchesne configure: fix screensavers/rsxs to allow for pkg-config checks.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
0111200
@gbeauchesne gbeauchesne vaapi: add support for EGL and texture-from-pixmap.
This initial approach relies on VA/X11 and vaPutSurface() to a Pixmap.
Then, an EGLImage is created from that pixmap and bound to a texture
with glEGLImageTargetTexture2DOES().

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
c734a0b
@gbeauchesne gbeauchesne egl: fix build on X11 systems.
Declare displays with EGLNativeDisplayType and ensure that windows
use integral values for initialization. This is still valid for cases
where the window type is a pointer because `0' is also a valid value
for them.

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
abdf416
@gbeauchesne gbeauchesne vaapi: fix check for VA/GLX API.
Use pkg-config to check for VA/GLX API. This fixes build with VA-API
installations into non-standard locations.

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2b6d480
@gbeauchesne gbeauchesne egl: fix creation of XBMC icon texture.
CTexture iconTexture; will create a 0x0 texture and CTexture::LoadFromFile()
won't update that texture. Rather, it creates and returns a new object.
That's this object we want to use for SDL_WM_SetIcon() as initialization
data. i.e. that newly created object will have the right size.

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
196b26a
@gbeauchesne gbeauchesne egl: fix GLES2 support in RGBA 8:8:8:8 mode.
... aka fix DEPTH to be 24, not 8 in this case. Other change includes
a minor cosmetics (vertical alignment).

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
f4492f8
@gbeauchesne gbeauchesne vaapi: rework GLX support.
Make CSurfaceGL a base class for OpenGL-based surface objects backing
regular VA surfaces, or actually wrapping them (future implementation).

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
72571bb
@gbeauchesne gbeauchesne configure: add check for pkg-config.
This ensures that the necessary pkg-config machinery is in place prior
to calling PKG_CHECK_MODULES() in various places, thus preventing the
configure script from failing in unexpected scenarios.

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
d6b55f1
@gbeauchesne gbeauchesne ffmpeg: use pkg-config to check for VA-API.
Fix build with VA-API installation into non-standard locations.
e.g. /opt/libva/<branch>/ used as a prefix.

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
97cd8af
@gbeauchesne gbeauchesne vaapi: move up creation of vaapi textures.
Creation of GL textures needs to wait until we actually have a decoded
VA surface, and thus making it possible to determine the actual surface
format and how many textures (planes) are needed.
df0f8c3
@gbeauchesne gbeauchesne shaders: add generic planar formats for OpenGL rendering.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
88e9842
@gbeauchesne gbeauchesne shaders: add support for more planar YUV formats to GLESv2.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
e6e434f
@gbeauchesne gbeauchesne vaapi: add support for new VA/EGL API.
This API allows for mapping a VA surface into as many EGLImages/textures
as there are planes in the surface. e.g. an NV12 VA surface can be mapped
into 2 textures, one for the Y plane and another one for the UV plane.

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
b59aa75
@gbeauchesne gbeauchesne shaders: fix texture uniforms for NV12 rendering.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
421a543

I'm a bit hesitant to do this. Some (proprietary) vendors do some insane things with their pkgconfig

theuni commented on f4492f8 Aug 19, 2012

This is depth buffer, not color depth. We don't use a depth buffer, we only provide a small one for vis/screensavers.

Owner

gbeauchesne replied Aug 19, 2012

OK, I will remove this patch. Thanks.

theuni commented on abdf416 Aug 19, 2012

I'm surprised that's all it took after the Android merge.

Thanks!

theuni commented on 196b26a Aug 19, 2012

We shouldn't fail to create the window just because the icon didn't load.

Owner

gbeauchesne replied Aug 19, 2012

OK, thanks, I will propagate the fix from WinSystemX11.cpp then. I just realized there was one there. Though, iconTexture probably needs to be checked for prior to delete'ing it. :)

Is there no extension for a quick buffer->fb swap/blit? Granted it'd only be of use when we have no GUI up and no postprocessing but it'd still be worth having if available imo.

Makes more sense in the begging, no?

*beginning

Needed? Wouldn't it be a simple flip and less delete logic without it?

theuni commented on c734a0b Aug 19, 2012

Very nice overall. I have a few general org recommendations (as we discussed via mail) though I don't see any reason to call them blockers.

I'd prefer to see the symbol getters and pixmap creation/uploads in our gles renderer or base renderer. Reason being I have global domination plans for a render manager one of these days, so that we have a single processor for rendering, rather than everyone doing it themselves all over the show.

I'm also having trouble understanding why the pixmap upload/render are tied to vaapi. Is this just because that's it's first user? I ask because if this is a general kms/drm feature, It'd be useful for skin texture uploads as well (for off-thread uploads). If they're not tied to vaapi, see the first note above again :)

Owner

gbeauchesne replied Aug 20, 2012

@theuni: the pixmap upload could better fit VAAPI.cpp because the EGL/X11 path actually suits "desktop" OpenGL, GLESv1 and GLESv2. So, moving it to the GLES renderer may not be fully correct. I will try to move a few things to base renderer then.

Owner

gbeauchesne replied Aug 20, 2012

@theuni: wrt. quick buffer to fb swap/blit, there is vaPutSurface() that is specific to X11 and doesn't use any form of GL. On some occasions, you could even use an overlay. However, there could be some issues when rendering the UI on top of it. Some VA drivers support overlay + 3D UI in another plane though.

theuni commented on b59aa75 Aug 19, 2012

Lots of dupe here. I'll have to look over again when it's not so late.

Owner

gbeauchesne replied Aug 19, 2012

Yes, I know, that's why I created RenderRGB() and RenderYUV() functions. The hope is to let other renderers re-use those as well, e.g. RenderSinglePass(), the CoreVideo renderers, OMX as well probably, etc. In this case, I'd need to properly fill in the target .rect fields.

Member

theuni commented Aug 19, 2012

@elupus @bobo1on1 ping.

@gbeauchesne I gave it a really quick look through, let's see what the others have to say.

Owner

gbeauchesne commented on b315964 Aug 19, 2012

@theuni: the alternative to pkg-config could probably be --with-{vaapi,opengl,lib{egl,gles2}}-path=... configure option? What kind of nasty things could happen in .pc files? Or, probably overriding INCLUDES=... LIBS=... before the ./configure command could be enough to let the script thing all dependencies installed in alternative directories?

Owner

gbeauchesne commented on df0f8c3 Aug 20, 2012

@theuni: AFAIK, the only thing that is not valid GLESv2 is to glEnable()/glDisable() with GL_TEXTURE_2D target. This is valid GLESv1 though.

Specifically I meant glPixelStorei(GL_UNPACK_ALIGNMENT, 4);. IIRC this is only available via an extension with gles?

Hmm, the spec says it's fine. Wonder what I was thinking of...

ah, mixed up with GL_UNPACK_ROW_LENGTH, no clue how I managed to do that. Sorry for the noise.

Member

elupus commented Aug 23, 2012

It's a big diff. If you could splitt the pull between GLES and GL, i would appreciate it. The GL stuff i can review, but i need assistance on the GLES since i can't test that.

Member

elupus commented Aug 23, 2012

First glance is good thou.

gbeauchesne added some commits Sep 5, 2012

@gbeauchesne gbeauchesne gles: disable NV12 rendering (FIXUP e6e434f).
NV12 is not used in GLES rendering, and the corresponding shader code is wrong
anyway.
f024e4c
@gbeauchesne gbeauchesne gles: drop commented code (FIXUP 421a543).
NV12 format is not used at all.
3609c7e
@gbeauchesne gbeauchesne gles: fix YV12 rendering (FIXUP e6e434f). f249e65

theuni was assigned Jan 3, 2013

wmyrda commented Jan 30, 2013

Guys is there any progress on this one? vaapi is my current way of using hardware acceleration and any way of improving this would be welcome 👍
Hopefully adding EGL/GLES would also contribute into running XBMC skins at less than 100% cpu power ;) I mean there should be a way of it taking less cpu than 1080p content :)

Member

opdenkamp commented Jan 30, 2013

could you please stop posting useless comments like this like you have been doing all day. use the forums

wmyrda commented Jan 31, 2013

sorry if I got a bit carried away as I did not mean to offend anyone, but rather just wanted to thank those that wrote the code. I looked at the all pull requests and commented only on 4 of them out of over 160 which IMHO where most important ones and hence giving in them +1 or thumbs up

Owner

MartijnKaijser commented Jan 31, 2013

and we don't appreciate you doing that because it has nothing to do we the actual code and is spamming a lot of people

Member

elupus commented Aug 4, 2013

this looks like it's slipped through the cracks. I wonder how much it would be to rebase and get up to speed again.

Owner

MartijnKaijser commented Sep 19, 2013

@gbeauchesne could you please update this PR?

Owner

MartijnKaijser commented Jun 10, 2014

@gbeauchesne small nudge for updating

Hi,

Sorry for being terribly lagging here. The VA/EGL API as exposed here is
not going to be the one that I will push to libva. A much simpler mechanism
is to be provided through dma_buf imports. Hopefully, the bits for libva +
intel-driver are going to happen by the end of the week, and the Mesa
support code by the end of June.

Meanwhile, could you please comment on the new APIs?

I plan to publish a simple FFmpeg/vaapi based player for your convenience
too, in case I don't get the time to update the XBMC code myself.
Basically, the idea remains the same: (i) export VA buffer store through
dma_buf, (ii) import buffer in EGL space on a per-plane basis, (iii) use a
shader to perform the conversion.

Regards,
Gwenole.

2014-06-10 20:50 GMT+02:00 Martijn Kaijser notifications@github.com:

@gbeauchesne https://github.com/gbeauchesne small nudge for updating


Reply to this email directly or view it on GitHub
#1302 (comment).

@LongChair LongChair added a commit to plexinc/plex-home-theater-public that referenced this pull request Nov 21, 2014

@LongChair LongChair Add keybinding to play trailer for a movie #1302 976258f
Owner

MartijnKaijser commented May 23, 2015

i'm closing this one as it's partially obsolete or outdated. Feel free to open a new one if needed :)

Member

fritsch commented May 23, 2015

@gbeauchesne we are planning to transition VAAPI completely from GLX to EGL. If I see that fully right, in the meantime everything we need is either in mesa and also in libva / intel driver. Is this correct?

lrusak referenced this pull request in OpenELEC/OpenELEC.tv Jun 5, 2015

Closed

Wayland display server support #4069

4 of 7 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment