Secure decoder performance points might be invalid on API 35 devices

[matamegger]

Version

Media3 main branch

More details

A while ago the performance point workaround was disabled for devices running on API level 35 or later. According to the commit message and code comment, the reason why it shouldn't be required anymore, is that the CTS contain tests that ensure valid performance points.

Reference commit: 737bf08

We suspect that the performance points for secure decoders are still incorrect on some devices after API 34.

Specifically, we have been able to identify a list of devices that run on API 35 that still seem to suffer from the underlying problem with the reported performance points for secure decoders. Most of these devices though are not initially shipped with API 35, but received OS updates to get there.

Unfortunately, the change in the player does not mention which test(s) in the CTS specifically relate to this. I tried to find them, but I don't have a lot of knowledge about the CTS code base. So take my findings with a grain of salt.
I believe I have found that the tests might have some blind spots, especially in regard to secure decoders, which are not tested at all if my understanding of the tests is right.
Entrypoint to the probably related CTS tests

---

Even if there are tests and I was just not able to find the proper one, there is still the list of devices which are on API 35+ and seem to suffer from insufficient/invalid performance point reporting. (See the quite long list in the devices section).

It might make sense to open up the workaround for higher API levels again, but I would also very much appreciate your thoughts and insights on the topic.

Example Decoder info dump

 Decoder “c2.mtk.avc.decoder” supports
    aliases: [
      “OMX.MTK.VIDEO.DECODER.AVC” ]
    attributes: 0xa: [
      encoder: 0,
      vendor: 1,
      software-only: 0,
      hw-accelerated: 1 ]
    owner: “codec2::default”
    rank: 512
    profile/levels: [
          1/32768 (Baseline/5.1),
      65536/32768 (ConstrainedBaseline/5.1),
          2/32768 (Main/5.1),
          8/32768 (High/5.1) ]
    colors: [
      0x7f000789 (Surface),
      0x7f420888 (YUV420Flexible),
      0x13 (YUV420Planar),
      0x15 (YUV420SemiPlanar),
      0x14 (YUV420PackedPlanar),
      0x27 (YUV420PackedSemiPlanar) ]
    details: AMessage(what = 0x00000000) = {
        string bitrate-range = “1-60000000"
        string block-count-range = “16-14400”
        string block-size = “16x16"
        string blocks-per-second-range = “16-432000”
        int32_t feature-adaptive-playback = 0
        int32_t feature-can-swap-width-height = 1
        string max-concurrent-instances = “15"
        string measured-frame-rate-1280x720-range = “111-244”
        string measured-frame-rate-1920x1080-range = “69-152"
        string measured-frame-rate-320x240-range = “158-349”
        string measured-frame-rate-720x480-range = “156-343"
        string performance-point-1280x720-range = “120-120”
        string performance-point-1920x1080-range = “60-60"
        string performance-point-2560x1440-range = “30-30”
        string size-range = “64x64-2560x1440"
        int32_t feature-detached-surface = 0
      }
  Decoder “c2.mtk.avc.decoder.secure” supports
    aliases: [
      “OMX.MTK.VIDEO.DECODER.AVC.secure” ]
    attributes: 0xa: [
      encoder: 0,
      vendor: 1,
      software-only: 0,
      hw-accelerated: 1 ]
    owner: “codec2::default”
    rank: 512
    profile/levels: [
          1/32768 (Baseline/5.1),
      65536/32768 (ConstrainedBaseline/5.1),
          2/32768 (Main/5.1),
          8/32768 (High/5.1) ]
    colors: [
      0x7f000789 (Surface),
      0x7f420888 (YUV420Flexible),
      0x13 (YUV420Planar),
      0x15 (YUV420SemiPlanar),
      0x14 (YUV420PackedPlanar),
      0x27 (YUV420PackedSemiPlanar) ]
    details: AMessage(what = 0x00000000) = {
        string bitrate-range = “1-60000000"
        string block-count-range = “16-8100”
        string block-size = “16x16"
        string blocks-per-second-range = “16-243000”
        int32_t feature-adaptive-playback = 0
        int32_t feature-can-swap-width-height = 1
        int32_t feature-secure-playback = 1
        string max-concurrent-instances = “1"
        string performance-point-1920x1080-range = “30-30”
        string size-range = “64x64-1920x1080"
        int32_t feature-detached-surface = 0
      }

Devices that reproduce the issue

Device Models:
SM-A556B, SM-A528B, SM-A546B, SM-S928B, 2312DRA50G, SM-S918B, SM-S911B, SM-A536B, SM-S921B, SM-S901B, 22101316G, 23090RA98G, M2101K6G, SM-A346B, CRT-NX1, REA-NX9, SM-A336B, SM-A566B, 2201116SG, 2312DRAABG, SM-A236B, SM-S938B, SM-A356B, 24090RA29G, SM-A137F, SM-X216B, Redmi Note 9 Pro, ELI-NX9, SM-S908B, SM-X205, T Tablet, SM-A715F, SM-G991B, 2201117TY, Lenovo TB128XU, SM-S931B, SM-A725F, SM-A156B, 2406APNFAG, MAR-LX1A, 2109119DG, SM-A515F, SM-X200, SM-S926B, ALI-NX1, SM-A226B, RMO-NX1, SM-A525F, LLY-NX1, SM-A145R

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Yes

Reproduction steps

We could reproduce the issue with a DASH stream, that contains DRM protected tracks with a frame rate of 50 also for HD ready (720p) resolution.
The affected device has 'limited' performance point reporting for the secure decoder. e.g. there is only one performance point at the maximum resolution the decoder would support, which is 1080p for most of the affected devices.

Expected result

Performance points might be ignored on all API levels, if the devise seems to inaccurately report it's capabilities. Tracks should not are marked as not supported.

Actual result

Some tracks are marked as not supported, despite them being playable without issues.

Media

I don't have a particular media file I can share here right now. In case you find it necessary to have one, please let me know.

Bug Report

• You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.

[tonihei]

Thanks for reporting, that's very useful! I've followed up on the internal bug that tracked adding these CTS tests to see if we can improve the coverage. It may depend on whether the CDD requirements require secure decoders to support the same set of frame rates as the non-secure ones.

Regardless of whether we can make the CTS tests better for future devices, I agree we should re-enable the workaround if possible. One issue here is that we specifically test the H264 non-secure decoder as a proxy for "does this device implement performance points correctly?". The example dump you provided shows that the workaround would in fact no longer work even if it was still enabled.

[matamegger]

I don't see why the example dump snipped wouldn't work with the workaround.

As far as I can see the workaround is also using the secure decoder. Still only avc codec as a proxy, but both the plain as well as the secure decoder.

If I am not mistaken, this if condition and it's body, would detect the invalid performance points setup and ignore them on the device:
https://github.com/androidx/media/blob/release/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecPerformancePointCoverageProvider.java#L147

[matamegger]

Thanks for your quick response and looking into it already.

[tonihei]

Yes, you are right, I was looking at an old state of the code that didn't include f0fb386 yet. (which attempts to fix the same issue in fact!)

[tonihei]

From your logs/testing, can you confirm that this only affects secure decoders? If so, we can make the API check conditional on whether or not we are evaluating a secure decoder.

[tonihei]

Re-enabled the check for the secure codecs only on API35+. Please let us know if that doesn't cover the cases you've found.

For the updated CTS testing, we'll follow up internally and if we get a fix may limit the workaround again.