egl_render: Conditions to let it emit FillBufferDone unclear

[sdroege]

Hi,

I have some OpenMAX code that tunnels video_render and egl_render directly (without a scheduler or anything else between both). video_render successfully sets the output format, egl_render is successfully tunneled to the video_render output port, EGLImages are passed to the egl_render output port (via OMX_FillThisBuffer()) and both components are in Executing state. No errors or anything.
The decoder also continues to consume buffers and "vcdbg log msg" shows the decoder generates lots of output frames and egl_render receives them:
758573.609: egl_renderRIL:display image(f47afe8) state 4
758573.624: egl_renderRIL:display image, dropping input image(f47af68)

However egl_render never calls FillBufferDone.

hello_videocube works on my device with the same movie and the main difference I see in the logs is that at some point, right before the first frame is shown, OMX_IndexConfigBrcmEGLImageMemHandle is set on the egl_render output port. This never happens with my code, and is also called from nowhere in hello_videocube.

From where is this called and what is necessary to have it called?

I see that in hello_videocube the texture from which the EGLImage is created is always bound to the texture rendering unit and it is drawn continously as fast as possible. This does not happen in my code and can't for various reasons (but the texture is of course still allocated).

Is OMX_IndexConfigBrcmEGLImageMemHandle somehow called from the GL side whenever something tries to access the EGLImage and only then egl_render fills the EGLImage and emits FillBufferDone?

Would it be possible to change egl_render to just emit FillBufferDone and fill the EGLImage whenever a frame is ready instead?

[popcornmix]

This call to eglIntOpenMAXILDoneMarker:
https://github.com/raspberrypi/userland/blob/master/interface/vmcs_host/vcilcs_out.c#L585
should provoke the message of OMX_IndexConfigBrcmEGLImageMemHandle on GPU side.

Can you build userland libs and debug whether eglIntOpenMAXILDoneMarker is called?

[sdroege]

Yes, it is called (no debug build was necessary, just a breakpoint on it was enough).

[sdroege]

When exactly does the GPU side call OMX_IndexConfigBrcmEGLImageMemHandle after that, and under which conditions?

[popcornmix]

Looks like eglIntOpenMAXILDoneMarker => "egl_renderRIL:SetConfig(OMX_IndexConfigBrcmEGLImageMemHandle)"
apart from errror handling. Are you sure there's been no errors reported and openmax is setup the same as hello_videocube?

[sdroege]

It's directly called from eglIntOpenMAXILDoneMarker()? Are there any checks before OMX_IndexConfigBrcmEGLImageMemHandle that could cause it not to be called?

There are no errors from the OpenMAX side at least and I couldn't find anything in the logs either. The setup is basically the same, same sequence of calls just that video_decode and egl_render are directly tunneled.

[sdroege]

I just checked the logs again, there's nothing fishy in there and it all looks like it should work... but doesn't:
http://pastebin.com/ZT3DsqW0

[sdroege]

Do you have any further information you can share about what exactly could cause eglIntOpenMAXILDoneMarker() to not call OMX_IndexConfigBrcmEGLImageMemHandle, any error conditions or anything between those two steps? The publically available source code is of course not very useful here because eglIntOpenMAXILDoneMarker() just does RPC on the videocore

[julianscheel]

Any news on this? I'd be very interested in it as well.

[popcornmix]

If you send me an example (source or binary) I can step through the GPU code and indentify the condition that is failing.

[sdroege]

I'll provide you an (extremely) large tarball with sources and binaries of GStreamer with everything set up... once it is uploaded. Instructions how to use that will follow

[sdroege]

Will be at http://people.collabora.com/~slomo/gstreamer.tar.bz2
md5sum will be 3b6357fea5c38396df3340b6b0f3016f
ETA is 1:30h currently but I have to leave in a few minutes, so just pick it from there later.

You would need to run raspbian, extract this as /home/slomo/gstreamer and /home/slomo/bin and if your user has a different name (which he probably has ;) ) you would create links to bin and gstreamer in your own home directory.

Then run bin/gst-master and then run
GST_DEBUG=omx*:6 GST_DEBUG_NO_COLOR=1 gst-launch-1.0 filesrc location=/opt/vc/src/hello_pi/hello_video/test.h264 ! h264parse ! omxh264dec ! eglglessink sync=false 2>&1 | tee log

One thing this will do is to create lots of debug output which would be useful if doesn't run for you (please upload the log if it doesn't run for you).

The relevant code is in gstreamer/master/gst-omx/omx/{gstomx.c,gstomxvideodec.c} and in gstreamer/master/gst-plugins-bad/ext/eglgles

[OtherCrashOverride]

Not sure if this is the same issue I had or not, but I will comment here so its considered. I went through a similar ordeal with egl_render as documented here
http://www.raspberrypi.org/phpBB3/viewtopic.php?f=33&t=6577&start=25

The issue is that there is a sub function with the same name that gets called unless you link in a specific order.
" You must link the OpenMax layer before linking the other libraries to prevent the stub from being called."

[sdroege]

It is indeed using the version from libbcm_host.so. What's the reason for this weird hack to have both libraries export this?

As I'm loading the OpenMAX layer via dlopen() this is a bit annoying... will try to find a solution for that.

[popcornmix]

@OtherCrashOverride
Do you remember what the stub functions were? Removing them sounds like the best solution.

[sdroege]

Yes, that's apparently the problem... wtf

eglIntOpenMAXILDoneMarker() is exported by libbcm_host.so and libopenmaxil.so. Problem in my case is that they're all loaded via dlopen() in plugins, I have no way to guarantee that libopenmaxil.so is loaded before libbcm_host.so.