mp4 with a height of 1080 does not return correct widthHeightRatio on a device screen that is 480x800

[42footPaul]

I've been implementing ExoPlayer in our Adobe AIR project as an ANE in order to do progressive download and save of mp4 files. This is working rather well and the player seems to work like a charm for our case. I'm using master and used the demo player as a start and added functionality as we needed such as the controls UI and a custom HttpDataSource and TransferListener to save the files.

In our latest testing we are getting a video scaling issue on devices with 480x800 screen resolution with videos that are 1080 in height.

I don't have access to the device at the moment so all I can do is have our tester run it and email our own logs.

For the 800x480 device we get this back for a video that is 1920x1080
PlayerActivity::VIDEO SURFACE CHANGED format=4 w=800 h=480
PlayerActivity::VIDEO SIZE CHANGED w=1920 h=540 widthRatio=1.0
PlayerActivity::widthHeightRatio=3.5555556

this is what we get for a video that is 1440x1080
PlayerActivity::VIDEO SURFACE CHANGED format=4 w=800 h=480
PlayerActivity::VIDEO SIZE CHANGED w=1440 h=540 widthRatio=1.0
PlayerActivity::widthHeightRatio=2.6666667

On a device that is 854x480 we get this back for the exact same 1920x1080 video
PlayerActivity::VIDEO SURFACE CHANGED format=4 w=854 h=480
PlayerActivity::VIDEO SIZE CHANGED w=1920 h=1080 widthRatio=1.0
PlayerActivity::widthHeightRatio=1.7777778

We verified that this happens in the Demo App as well. Any suggestions here?

[ojw28]

What exactly do you see in the demo app? I'm not sure where you've added logging. Are you seeing an incorrect height reported to PlayerActivity.onVideoSizeChanged (540 instead of 1080)?

Is this a problem on multiple devices with that resolution, or just a single device? I suspect it's more likely to be a device specific than a problem that's caused by the size of the screen itself. Given the device only has a 480p screen, I also wouldn't be at all surprised if the hardware isn't really powerful enough to play 1080p content. What device is it?

[42footPaul]

The screenshots I have attached are from the demo app.
We are in fact seeing an incorrect height reported to PlayerActivity.onVideoSizeChanged (540 in that case instead of 1080) but the widthHeightRatio=3.5555556 is also very wrong and I suspect this may be part of the issue?

We are seeing this on multiple devices
Karbonn A34 Lite JellyBean 512MB 480x800
Karbonn A99i JellyBean 512MB 480x800
Gionee Pioneer P2 JellyBean 512MB 480x800

They play the videos just fine it's just that they are rendered with an incorrect height. As you can see by the screenshots they are squeezed vertically.
These devices play the videos outside of the demo app with whatever media player it uses just fine.

I am part of a remote team and don't have direct access to the devices otherwise I could send better logs. If there is something specific you want me to log I can get that done.

For what it's worth the videos render correctly on tis device
Lava iris 500 KitKat 512MB 480x854

[ojw28]

I don't know what widthHeightRatio=3.5555556 is in this context, because I don't know where you've added logging. Where does that come from?

[42footPaul]

my bad, that was unclear - that is actually the height calculation

(width * pixelWidthAspectRatio) / height

public void onVideoSizeChanged(int width, int height, float pixelWidthAspectRatio) {
shutterView.setVisibility(View.GONE);
surfaceView.setVideoWidthHeightRatio(height == 0 ? 1 : (width * pixelWidthAspectRatio) / height);
Log.d(TAG, "VIDEO SIZE CHANGED");
}

[ojw28]

Right, got it. So the only thing wrong is that the height is half what it should be. If it were correct, then the widthHeightRatio would be correct too.

[42footPaul]

right - so it seems that the video height is being reported incorrectly?

[ojw28]

Please try adding logging to MediaCodecVideoTrackRenderer.onOutputFormatChanged. Specifically, if you could get the values of KEY_WIDTH, KEY_HEIGHT, and (if they exist) the values of the four KEY_CROP_X entries.

Please also add logging to MediaCodecVideoTrackRenderer.onInputFormatChanged to print holder.format.width, holder.format.height and holder.format.pixelWidthHeightRatio.

My guess would be that the decoder is downscaling the video by a factor of 2 in the vertical direction because these devices can't support 1080p buffers. The results of the suggested debugging above should help to validate this theory.

[42footPaul]

Here's the logs you were asking for - the interesting thing is that this function seems to get called twice in a row only for these 1080 vids. (log shows up twice back to back with exact same results - didn't bother pasting the 2nd output)

MediaCodecVideoTrackRenderer::onOutputFormatChanged
MediaCodecVideoTrackRenderer::KEY_WIDTH = 1440
MediaCodecVideoTrackRenderer::KEY_HEIGHT = 540
MediaCodecVideoTrackRenderer::KEY_CROP_RIGHT = 1439
MediaCodecVideoTrackRenderer::KEY_CROP_LEFT = 0
MediaCodecVideoTrackRenderer::KEY_CROP_TOP = 0
MediaCodecVideoTrackRenderer::KEY_CROP_BOTTOM = 539

[42footPaul]

Any suggestions on this?

[42footPaul]

Realized I didn't give you the input format changed log you were looking for.

MediaCodecVideoTrackRenderer::onInputFormatChanged
MediaCodecVideoTrackRenderer::holder.format.width = 1920
MediaCodecVideoTrackRenderer::holder.format.height = 1080
MediaCodecVideoTrackRenderer::holder.format.pixelWidthHeightRatio = 1.0

[42footPaul]

doing some more digging.

this is a list of the video decoders on the device
OMX.MTK.VIDEO.DECODER.MPEG4
OMX.MTK.VIDEO.DECODER.H263
OMX.MTK.VIDEO.DECODER.AVC
OMX.MTK.VIDEO.DECODER.VPX
OMX.MTK.VIDEO.DECODER.VC1
OMX.MTK.VIDEO.DECODER.DIVX
OMX.MTK.VIDEO.DECODER.DIVX3
OMX.MTK.VIDEO.DECODER.XVID

The decoder chosen is OMX.MTK.VIDEO.DECODER.AVC

Since the stock Video Player on the device plays the video correctly could it be using the MPEG4 decoder?

[42footPaul]

This is a bit of a blocker in our next release and need to find out if you think this is something that can be addressed in the framework.

[42footPaul]

More investigation
OMX.MTK.VIDEO.DECODER.AVC supports the following profiles
profile:1
profile:2
profile:4
profile:8
profile:16
profile:32
profile:64
profile:128
profile:256
profile:512
profile:1024
profile:2048
profile:4096
profile:8192
profile:16384
profile:32768

when I convert the decimal values here to hex and compare them with the table of supported types in OpenMax found here https://www.khronos.org/files/openmax_il_spec_1_0.pdf on page 285/286 I see that this decoder supports up to and including Level 5.1 which when compared to the list found here https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels tells me the decoder itself supports 1080.

You are saying that these devices can't support 1080p buffers yet the 1080 vids play fine on the stock player and it looks to me like the decoder supports 1080. I don't understand. The issue is still open and flagged with need more info. Is there anything I can do to further the cause?