Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ExoPlayer doesn't check if AVC/HEVC profile/level are supported when playing media file directly #8393

Closed
equeim opened this issue Dec 23, 2020 · 4 comments
Assignees

Comments

@equeim
Copy link
Contributor

equeim commented Dec 23, 2020

It seems that ExoPlayer can only check if video profile/level is supported by decoder (as reported by MediaCodecInfo.getCapabilitiesForType) only when profile/level information is provided in DASH/HLS manifest.

The only exception is MPEG-TS extractor that can get profile/level information from AVC bitstream.

If that is not an intentional design decision, I would like to make a pull request that will allow ExoPlayer to extract profile/level information from AVC and HEVC bitstreams for other extractors too.

@ojw28
Copy link
Contributor

ojw28 commented Dec 23, 2020

I don't think there was ever an intentional decision not to implement this. That said, the value of doing so seems smaller than for DASH/HLS. For DASH/HLS, it might be the difference between playback succeeding and failing, since variant streams may use different profiles/levels, and a subset may be supported. For progressive streams I would assume it's the difference between playback failing with a clear error and playback failing with a less clear error.

Does that align with your understanding, and is your goal here just to make the failure mode clearer? If so, that seems fine, although given the relatively limited usefulness, I think we'd want to keep the implementation fairly straightforward. If it's fundamentally extremely complicated then it might be preferable not to implement it at all. Hopefully that's not the case though!

So, I think the next steps here would be:

  • If your goal is to make the failure mode clearer and if you think it's fairly straightforward, feel free to go ahead and send a pull request to the dev-v2 branch.
  • If you have other goals, let's discuss what those are :).

As a final note, it's possible this might also fix #5882.

Thanks!

@equeim
Copy link
Contributor Author

equeim commented Dec 23, 2020

For progressive streams I would assume it's the difference between playback failing with a clear error and playback failing with a less clear error.

The problem is that not all decoders fail with error in these cases. Some will "decode" frames, but show black screen or distorted image, for example. This makes it hard for application to appropriate action in these cases (which is especially important for things like ads when you would want to skip unsupported ad video and show actual content).

I suspect that it's decoders fault to not indicate error when unsupported profile is passed for decode, but there is an API for check if is supported beforehand and it makes for player to use it.

Regarding implementation complexity: for AVC it is very easy, since the code is already there. All that is needed is a few lines to actually hook it up in extractors. For HEVC it would require a bit of additional SPS parsing, but it's quite straightforward too.

@ojw28
Copy link
Contributor

ojw28 commented Dec 23, 2020

Thanks for the clarification! Perhaps send a pull request for AVC to get started, in that case, and follow up with the HEVC one separately :)?

@equeim
Copy link
Contributor Author

equeim commented Dec 23, 2020

That's what I plan to do!

equeim added a commit to equeim/ExoPlayer that referenced this issue Dec 24, 2020
This will allow ExoPlayer to check if video track's profile
and level are supported by decoder when playing progressive media sources.

Also fix typo in AvcConfig.
equeim added a commit to equeim/ExoPlayer that referenced this issue Dec 27, 2020
This will allow ExoPlayer to check if video track's profile
and level are supported by decoder when playing progressive media sources.
equeim added a commit to equeim/ExoPlayer that referenced this issue Jan 5, 2021
This will allow ExoPlayer to check if video track's profile
and level are supported by decoder when playing progressive media sources.
icbaker pushed a commit that referenced this issue Jan 8, 2021
Imported from GitHub PR #8401

This will allow ExoPlayer to check if video track's profile and level are supported by decoder when playing progressive media sources.
Merge e582fb9 into 1347d57

Issue: #8393
COPYBARA_INTEGRATE_REVIEW=#8401 from equeim:hevc-codecs e582fb9
PiperOrigin-RevId: 350738065
icbaker pushed a commit that referenced this issue Jan 11, 2021
*** Original commit ***

Merge #8401: Initialize Format.codecs from HEVC SPS NAL unit (#8393)

Imported from GitHub PR #8401

This will allow ExoPlayer to check if video track's profile and level are supported by decoder when playing progressive media sources.
Merge e582fb9 into 1347d57

Issue: #8393

***

PiperOrigin-RevId: 350871621
icbaker pushed a commit that referenced this issue Jan 11, 2021
*** Original commit ***

Rollback of ff8c864

*** Original commit ***

Merge #8401: Initialize Format.codecs from HEVC SPS NAL unit (#8393)

Imported from GitHub PR #8401

This will allow ExoPlayer to check if video track's profile and level are supported by decoder when playing progressive media so...

***

PiperOrigin-RevId: 351139861
@ojw28 ojw28 closed this as completed Feb 4, 2021
@google google locked and limited conversation to collaborators Apr 6, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants