Skip to content
This repository

Add: [droid] hw acceleration via libstagefright #1832

Closed
wants to merge 18 commits into from

8 participants

Chris Browet Sascha Montellese Memphiz davilla da-anda Joakim Plate ulion Lars Op den Kamp
Chris Browet
Collaborator

This PR to monitor progress on providing accelerated android decoding via libstagefright

Sascha Montellese
Collaborator

OK I gave this a try on my SGS2 and apart from the mentioned issue it compiled fine. I was able to play normal AVI files as I used to and then I tried a 1080p mp4 which I recorded with my phones camera. The result was a very green-blueish image and it didn't seem accelerated to me (i.e. very jerky). This is the part of the log where I start playing the video: http://xbmclogs.com/show.php?id=15380. So it looks like it uses libstagefright.
Afterwards I wasn't able to close XBMC anymore (only force close through android's task manager) and after a restart of xbmc I couldn't play any h264 files anymore. It just kept on going in the busy dialog and there was nothing in logcat.

Chris Browet
Collaborator

IIRC, the blue-greenish ugly image is what you get when libstagefright (the sys so) is software rendering...

Chris Browet
Collaborator

Note that will checking other players (e.g. vplayer, where hw decoding can be switched), I came to realize most are actually able to render my test file (a 720p) in software flawlessly, so there might be more than HW decoding at stake...

Chris Browet
Collaborator

Moved the libstagefright stuff out of ffmpeg and inside dvdplayer as an hardware codec. That allows proper fallback to ffmpeg software decoding if a stagefright hw codec is not available.

Working great in 720p on my nexus 7

Memphiz
Owner

Nice work! Sry if my comments came twice. Fuckin IE8 here ;)

I really like the fact of having a stagefright codec. Also putting the needed sources into the deps is a nice idea imo.

3 questions out of curiousity:

  1. Does it handle 1080p aswell?
  2. Does it work different/better as the internal ffmpeg impl?
  3. You tried anything else beside h.264? (seems a part of the code is already prepared for mpeg2, mpeg4part2 and vc1)?
davilla
Collaborator

nice

Chris Browet
Collaborator

@Memphiz
1) Not yet properly. I suspect scaling issues. Working on it...
2) This one works seamlessly while the ffmpeg one didn't. Plus I can now specify that I want hw codecs only and fallback to ffmpeg otherwise
3) Not yet. 1080p h264 is my primary focus for now...

Chris Browet
Collaborator

Note that this is ICS+ only. If we want to support both 2.3 and 4.0 with the same package, we'll have to go the DllImpl path...

Memphiz
Owner

I would say we should focus on ICS and let gingerbread die in fire - but I guess TheUni would come up with better arguments then my stomac haha.

Chris Browet
Collaborator

Yeah. Anyway, I somewhat doubt a device still having GB is a good candidate to run XBMC...

da-anda
Collaborator

XBMC runs pretty well on my GB phone, and it's capable of playing 720p (tested with VLC I think) - so IMO we should support it if it doesn't add to much overhead

Memphiz
Owner

I tried this branch and it just gives me a black screen (tested a 720p mkv h.264) - audio is working without issues - but black screen. Overlay shows stf-h264 usage.

http://pastebin.com/UCycLzdJ

In the log i also tried it with software decoding (which worked - but cpu load is to high for making it smooth).

This was tested on odroid-x running ICS 4.0.4 (kernel 3.0.41)

Chris Browet
Collaborator

Ok. I'll first focus on hw I have, i.e. tegra3 Nexus 7. If I get expected results, i.e. smooth h264 1080p, we'll see how we can build from there to support more devices...

xbmc/cores/dvdplayer/DVDCodecs/Video/StageFrightVideo.cpp
((498 lines not shown))
  498
+  if (status != OK)
  499
+  {
  500
+    CLog::Log(LOGERROR, "%s::%s - %s\n", CLASSNAME, __func__,"Error getting picture from frame");
  501
+    return false;
  502
+  }
  503
+
  504
+  pDvdVideoPicture->format = RENDER_FMT_YUV420P;
  505
+  pDvdVideoPicture->pts = DVD_NOPTS_VALUE;
  506
+  pDvdVideoPicture->dts = DVD_NOPTS_VALUE;
  507
+  pDvdVideoPicture->color_range  = 0;
  508
+  pDvdVideoPicture->color_matrix = 4;
  509
+  pDvdVideoPicture->iFlags  = DVP_FLAG_ALLOCATED;
  510
+  pDvdVideoPicture->iWidth  = frame->width;
  511
+  pDvdVideoPicture->iHeight = frame->height;
  512
+  pDvdVideoPicture->iDisplayWidth  = frame->width;
  513
+  pDvdVideoPicture->iDisplayHeight = frame->height;
1
Joakim Plate Collaborator
elupus added a note December 08, 2012

this is very likely not right. this should reflect aspect of frame

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
davilla
Collaborator

see https://github.com/Pivosgroup/xbmc, for new --enable-codec configure switch. I'd like to move any alternate codecs to use this switch. It can handle multiple items too. So something like

--enable-codec=amcodec,stfcodec

would enable the building of both amcodec and stfcodec

Chris Browet
Collaborator

ack

davilla
Collaborator

what is your current dev branch ? I want to start resolving some issues.

Chris Browet
Collaborator

Please hold on. I'm currently trying to avoid buffer copying thru textures.
I'll remove the [WIP] when the "final" PR will be ready for actual review/patching.

Chris Browet
Collaborator

Quite happy with the current state, so opened for review / patches. Still an issue with FF/Rewind.
ICS+ only.

I'll put up test builds in the forum as soon as they're built.

davilla
Collaborator

nice, look forward to testing

davilla
Collaborator

oh, --enable-codec=amcodec,stfcodec ?

Chris Browet
Collaborator

Re --enable-codec, did you already commit yours? I planned to rebase upon it..

davilla
Collaborator

k

Chris Browet
Collaborator

Updated for build issues

davilla
Collaborator

m_keyguardLock is not defined

Chris Browet
Collaborator

Has been removed in latest push, hasn't it?

davilla
Collaborator

yes, my bad, I'm a little behind.

Droid-U2, this worked before, now does not.

V/XBMC ( 2309): 19:54:17 T:1556104592 DEBUG: CDVDPlayer::HandleMessages - player started 2
V/XBMC ( 2309): 19:54:18 T:1556105112 DEBUG: CDVDPlayerAudio:: Discontinuity1 - was:554500.791667, should be:428009.082667, error:-126491.709000
V/XBMC ( 2309): 19:54:18 T:1556105112 DEBUG: CDVDPlayerAudio:: Discontinuity1 - was:761432.166667, should be:864000.000000, error:102567.833333
V/XBMC ( 2309): 19:54:18 T:1556105112 DEBUG: CDVDPlayerAudio:: Discontinuity1 - was:1267193.250000, should be:1396245.624667, error:129052.374667
E/OMXCodec( 2309): [OMX.SEC.AVC.Decoder] Timed out waiting for output buffers: 0/6
V/XBMC ( 2309): 19:54:20 T:1557301896 ERROR: CStageFrightVideo - decoding error (-110)
V/XBMC ( 2309): 19:54:20 T:1557301896 DEBUG: CDVDPlayerVideo - video decoder returned error
E/OMXCodec( 2309): [OMX.SEC.AVC.Decoder] Timed out waiting for output buffers: 0/6
V/XBMC ( 2309): 19:54:23 T:1557301896 ERROR: CStageFrightVideo - decoding error (-110)
V/XBMC ( 2309): 19:54:23 T:1557301896 DEBUG: CDVDPlayerVideo - video decoder returned error
E/OMXCodec( 2309): [OMX.SEC.AVC.Decoder] Timed out waiting for output buffers: 0/6
V/XBMC ( 2309): 19:54:26 T:1557301896 ERROR: CStageFrightVideo - decoding error (-110)
V/XBMC ( 2309): 19:54:26 T:1557301896 DEBUG: CDVDPlayerVideo - video decoder returned error
V/XBMC ( 2309): 19:54:27 T:1556105112 WARNING: CDVDMessageQueue(audio)::Get - asked for new data packet, with nothing available
V/XBMC ( 2309): 19:54:27 T:1556105112 DEBUG: CSoftAEStream::Flush
E/OMXCodec( 2309): [OMX.SEC.AVC.Decoder] Timed out waiting for output buffers: 0/6

Chris Browet
Collaborator

Fragmentation, fragmentation :-(

Could you share the full log, please. There might be messages earlier indicating the root cause..

davilla
Collaborator

http://pastebin.com/dS1i8EbL using your fulldebug image.

Chris Browet
Collaborator

Right. I think I remember this codec (and probably others) only allocate 2 output buffers, and both are locked by the renderer...

davilla
Collaborator

sounds about right.

davilla
Collaborator

time to update ?

Chris Browet
Collaborator

Coming... I think I found the best compromise: FBO's

davilla
Collaborator

davilla@rootcoder:/xbmc/xbmc-android$ file xbmc/cores/dvdplayer/DVDCodecs/Video/StageFrightVideo.cpp
xbmc/cores/dvdplayer/DVDCodecs/Video/StageFrightVideo.cpp: ASCII C program text, with CRLF line terminators

opps :)

xbmc/cores/dvdplayer/DVDCodecs/Video/StageFrightVideo.cpp
((847 lines not shown))
  847
+    //frame->pts = (dts != DVD_NOPTS_VALUE) ? pts_dtoi(dts) : ((pts != DVD_NOPTS_VALUE) ? pts_dtoi(pts) : 0);
  848
+    frame->pts = (pts != DVD_NOPTS_VALUE) ? pts_dtoi(pts) : ((dts != DVD_NOPTS_VALUE) ? pts_dtoi(dts) : 0);
  849
+    frame->duration = 0;
  850
+    frame->medbuf = p->getBuffer(demuxer_bytes);
  851
+    if (!frame->medbuf)
  852
+    {
  853
+      free(frame);
  854
+      return VC_ERROR;
  855
+    }
  856
+    fast_memcpy(frame->medbuf->data(), demuxer_content, demuxer_bytes);
  857
+    frame->medbuf->meta_data()->clear();
  858
+    frame->medbuf->meta_data()->setInt64(kKeyTime, frame->pts);
  859
+
  860
+    if (p->mPrevPts >= 0)
  861
+    {
  862
+      frame->duration = frame->pts - p->mPrevPts;
7
Joakim Plate Collaborator
elupus added a note February 04, 2013

This will most often be wrong. Note that pts is not in decode order. B frames will (and must come first). Don't try to set duration from anything but the demuxer packet.

Chris Browet Collaborator
koying added a note February 05, 2013

You mean from the decoded packet? We don't get the duration in ::Decode...

Joakim Plate Collaborator
elupus added a note February 05, 2013
Chris Browet Collaborator
koying added a note February 05, 2013

The only point is to feed ::GetTimeSize. But I might get the concept wrong...

Joakim Plate Collaborator
elupus added a note February 05, 2013
Chris Browet Collaborator
koying added a note February 05, 2013
Joakim Plate Collaborator
elupus added a note February 05, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
xbmc/cores/dvdplayer/DVDCodecs/Video/StageFrightVideo.cpp
((1,049 lines not shown))
  1049
+        pDvdVideoPicture->iLineSize[0] = p->videoStride;
  1050
+        pDvdVideoPicture->iLineSize[1] = p->videoStride;
  1051
+        pDvdVideoPicture->iLineSize[2] = 0;
  1052
+        pDvdVideoPicture->iLineSize[3] = 0;
  1053
+        pDvdVideoPicture->data[0] = data;
  1054
+        pDvdVideoPicture->data[1] = pDvdVideoPicture->data[0] + (p->videoStride  * p->videoSliceHeight);
  1055
+        pDvdVideoPicture->data[2] = 0;
  1056
+        pDvdVideoPicture->data[3] = 0;
  1057
+        break;
  1058
+      default:
  1059
+        CLog::Log(LOGERROR, "%s::%s - Unsupported color format(%d)\n", CLASSNAME, __func__,p->videoColorFormat);
  1060
+    }
  1061
+  #if defined(DEBUG_VERBOSE)
  1062
+    CLog::Log(LOGDEBUG, ">>> pic pts:%f, data:%p, col:%d, w:%d, h:%d, tm:%d\n", pDvdVideoPicture->pts, data, p->videoColorFormat, p->videoStride, p->videoSliceHeight, XbmcThreads::SystemClockMillis() - time);
  1063
+  #endif
  1064
+  }
2
Joakim Plate Collaborator
elupus added a note February 04, 2013

You really should avoid any attempt to render here. Pass the frame out from decode down to the normal renderers.

Chris Browet Collaborator
koying added a note February 05, 2013

Not sure what you mean by "render", here. I'm just passing the YUV image the normal renderer...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Chris Browet
Collaborator

Cleaned up and squashed in #2504 .
Closing this one

Chris Browet koying closed this March 26, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.