Skip to content

Commit

Permalink
avcodec/mediacodecdec: fix SEGV on modern nvidia decoders
Browse files Browse the repository at this point in the history
This code came originally from gstreamer, where it was added in [1]
as a work-around for the Tegra 3. (The alignment was changed in [2]
as a response to [3], from 32-bit to 16-bit).

gstreamer only used this workaround in the case where the decoder
didn't return a slice-height property, but when the code was copied
into avcodec the conditional got lost. This commit restores the guard
and prefers the slice-height from the decoder when it is available.

This fixes segfaults decoding 1920x1080 h264 and mpeg2 videos on the
NVidia SHIELD after upgrading to Android Oreo.

[1] GStreamer/gst-plugins-bad@a870e6a
[2] GStreamer/gst-plugins-bad@21ff3ae
[3] https://bugzilla.gnome.org/show_bug.cgi?id=748867

Signed-off-by: Aman Gupta <aman@tmm1.net>
  • Loading branch information
tmm1 committed Aug 3, 2018
1 parent 5aeb3b0 commit 476fd6b
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions libavcodec/mediacodecdec_common.c
Expand Up @@ -389,13 +389,14 @@ static int mediacodec_dec_parse_format(AVCodecContext *avctx, MediaCodecDecConte
s->stride = s->stride > 0 ? s->stride : s->width;

AMEDIAFORMAT_GET_INT32(s->slice_height, "slice-height", 0);
s->slice_height = s->slice_height > 0 ? s->slice_height : s->height;

if (strstr(s->codec_name, "OMX.Nvidia.")) {
if (strstr(s->codec_name, "OMX.Nvidia.") && s->slice_height == 0) {
s->slice_height = FFALIGN(s->height, 16);
} else if (strstr(s->codec_name, "OMX.SEC.avc.dec")) {
s->slice_height = avctx->height;
s->stride = avctx->width;
} else if (s->slice_height == 0) {
s->slice_height = s->height;
}

AMEDIAFORMAT_GET_INT32(s->color_format, "color-format", 1);
Expand Down

0 comments on commit 476fd6b

Please sign in to comment.