2160p60 playback after ExoPlayer 2.12.1 update

[CiNcH83]

@yuliskov

I now tried 2160p60 playback on a BRAVIA ATV2 (MediaTek MT5891 @ 1GHz) with STubeNext 10.54 with the updated ExoPlayer 2.12.1. Looks much better now, similar to the original YouTube app. There is some minor micro-stuttering every few seconds left.

I wonder which MediaCodec mode ExoPlayer operates in. I don't see one being explicitly set by STubeNext. So I assume it operates in OPERATION_MODE_SYNCHRONOUS? How about Tunneled playback mode?

[foreveryoungforty]

Same over here on Sony 65xf9005, some micro-stuttering mostly in the beginning of 4k videos.

[yuliskov]

Thanks for the positive feedback about exoplayer 2.12.1.
Yes, current mode is OPERATION_MODE_SYNCHRONOUS.
Which mode do you suggest to test instead?
I've tested Tunneled playback mode before.
It receives multiple negative feedbacks from other users.

[CiNcH83]

Which mode do you suggest to test instead?

Probably all modes in order to be able to compare the outcome on a slow SoC.

OPERATION_MODE_ASYNCHRONOUS_PLAYBACK_THREAD has the charm of working from API 21 onwards. Then again I think that dequeuing might make sense to be done outside of the playback thread. So OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD might be worth trying in case the first one isn't good enough.
I don't think that enqueuing and dequeuing really need to be on separate threads. But devs say that it further improved playback of high frame rate content. Also depends on the SoC probably.

[CiNcH83]

Tried 10.56 with OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD_ASYNCHRONOUS_QUEUEING. Didn't really help with the minor micro-stuttering every now and then. Looks about the same as before.

Wonder whether you can query and display vfpo (Video Frame Processing Offset) from ExoPlayer?

[foreveryoungforty]

Tried 10.56 with OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD_ASYNCHRONOUS_QUEUEING. Didn't really help with the minor micro-stuttering every now and then. Looks about the same as before.

Wonder whether you can query and display vfpo (Video Frame Processing Offset) from ExoPlayer?

I agree, still some tiny micro stutter with 10.56 at the beginning of 4k and higher videos. Maybe loading or buffer issue at the start ? As from 5 or 10 seconds onwards, it plays perfect here.

[CiNcH83]

After some more testing I have to say that playback is close to perfection now. I typically test with The World in HDR in 4K as it has quite some panning shots in it.

[foreveryoungforty]

Indeed that's a great video to test with, as it has panning and slow motions.
And indeed the playback is close to perfection now...

[CiNcH83]

Damn. It reverted to 1080p30 AVC. Need to re-evaluate...

[foreveryoungforty]

Strange, here i have vp9 60fps

[CiNcH83]

In order to downgrade I had to uninstall STubeNext. That's how my preferred format (being VP9 HDR) got reset. I will need to do another comparison using the following versions:

• STubeNext 10.56: ExoPlayer 2.12.1 (MediaCodec async)
• STubeNext 10.54: ExoPlayer 2.12.1 (MediaCodec sync)
• STubeNext 10.44: ExoPlayer 2.10.6

[foreveryoungforty]

For me the 10.56 version is the best release so far, aside from a bit micro lag at the beginning of 4k+ videos...

[CiNcH83]

Will try 10.58 with OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD as well. Think I will find the time tomorrow.

[foreveryoungforty]

10.58 has more stutter than 10.56 on my Sony 4K.

[kuafor41]

For me the 10.56 version is the best release so far, aside from a bit micro lag at the beginning of 4k+ videos...

How about the status when you watch 1.5 or 2x speed ?
In normal speed you’re correct. I mostly watch videos double speed and previously with smartyoutubetv this was not the case this is what i want to reach to say build is o.k.

[CiNcH83]

10.58 has more stutter than 10.56 on my Sony 4K.

Haven't tried 10.58 but I am not 100% sure about what you just wrote. One time Sony works better, one time worse. It depends on services that might run in parallel in the background. What I can say is that 2160p60 playback pretty much exhausts the CPU which is the reason why playback might be fragile. I really need to do a more thorough investigation, testing on a longer term and creating the same conditions for all versions.

[foreveryoungforty]

Well for me 10.56 was better (just reinstalled), and in version 10.48 I had no lagging/stutter at all. And even the former SYTV-app still plays flawless without lagging.

[CiNcH83]

Just for completeness... 10.48 was with ExoPlayer 2.12.1 in OPERATION_MODE_SYNCHRONOUS mode.

Will add my results tomorrow.

[foreveryoungforty]

For me the 10.56 version is the best release so far, aside from a bit micro lag at the beginning of 4k+ videos...

How about the status when you watch 1.5 or 2x speed ?
In normal speed you’re correct. I mostly watch videos double speed and previously with smartyoutubetv this was not the case this is what i want to reach to say build is o.k.

I got your point, but to test the HQ settings, normal 1.0 speed is the standard/reference of course.

[CiNcH83]

I now finished profiling 2160p60 playback on a slow Sony BRAVIA X900F using different SmartTubeNext/ExoPlayer versions with different MediaCodec operational modes (sync/async).

Device: Sony BRAVIA KD-65XF9005 (X900F)
CPU: 4X ARM Cortex A53 @ 1GHz
Player: SmartTubeNext
Video: The World in HDR in 4K (17mbps @ 1440p60; 30mbps @ 2160p60)

SmartTubeNext 10.44

ExoPlayer: 2.10.6
Video buffer: Medium

2160p60 (4K/UHD)
There is slight micro-stuttering every few seconds which is mostly visible in panning shots. CPU usage is close to 100% when this happens. No frames are being dropped. So the problem might be VSync timing related and happen inside the SurfaceFlinger.
There is lots of frame dropping when browsing recommendations on top of the playing video. As soon as ExoPlayer finishes downloading towards the end of the clip, rendering becomes smooth, even when browsing recommendations on top of the playing video. CPU usage drops considerably. So I assume that the streaming and rendering threads are fighting for resources.

2X speed exhibits extreme frame dropping and bad stuttering.

1440p60 (2K/QHD)
Video playback is perfectly fine. CPU usage never goes high enough to cause stuttering. There is hardly any frame dropping when browsing recommendations on top of the playing video.

2X speed exhibits extreme frame dropping and bad stuttering.

SmartTubeNext 10.54

ExoPlayer: 2.12.1
Mode: OPERATION_MODE_SYNCHRONOUS
Video buffer: Medium

2160p60 (4K/UHD)
Heavy stuttering happens in the beginning due to very high CPU usage. Initial buffering seems to be more resource intensive than with ExoPlayer 2.10.6. Playback also suffers a bit more from the recurring micro-stuttering.
There is lots of frame dropping when browsing recommendations on top of the playing video.

2X speed exhibits extreme frame dropping and bad stuttering.

1440p60 (2K/QHD)
Video playback is perfectly fine. CPU usage never goes high enough to cause stuttering. There is hardly any frame dropping when browsing recommendations on top of the playing video.

2X speed exhibits extreme frame dropping and bad stuttering.

SmartTubeNext 10.56

ExoPlayer: 2.12.1
Mode: OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD_ASYNCHRONOUS_QUEUEING
Video buffer: Medium

Pretty similar to 10.54. Async MediaCodec does not seem to have an impact.

SmartTubeNext 10.58

ExoPlayer: 2.12.1
Mode: OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD
Video buffer: Medium

Same as 10.56.

Conclusion

ExoPlayer 2.10.6 seems better than 2.12.1. The MediaCodec modes do not seem to be having a real impact. Are we sure that they get applied at all?

2160p60 suffers from recurring micro-stuttering on BRAVIA due to playback exhausting the slow CPU. Streaming high bitrate content (30mbps isn't exactly high though...) seems to have quite an impact on CPU usage. Also quite some frames get dropped when browsing recommendations on top of the playing video. 1440p60 is fine already (which is half the bitrate). 2160p60 is also fine after the clip has finished downloading. So it clearly looks like streaming and rendering threads are fighting for resources.

[foreveryoungforty]

@CiNcH83 @yuliskov
Great technical analysis (I also have Sony BRAVIA KD-65XF9005) - as I mentioned in other post/issues regarding HDR and stutter, I also use the preset 2K 60fps vp9 hdr on my Sony 4K to get flawless playback. I also uncheck energy optimalization for STube in the special app settings on android 9, but I don't think that makes much difference. Changing the buffer to medium or even to maximum with 4K 60fps vp9 hdr had absolutely no influence on the playback here on my system ... same stutter.
The solution to put the video preset on 2K 60fps vp9 hdr for a 4K-device remains a bit "strange/weird" from a user-standpoint, but it is a great work-a-round with still splendid picture.
There is only one thing that keeps me wondering and hoping that there must be hope for a solution: what is the reason that this video still plays without any stutter on SYTV 6.17.730 (which has exo 2.10.6) with the optimal video preset 4K ? (just checked it a minute ago). If I am logically thinking, the reason must be found in a technical difference between the two apps....

[CiNcH83]

what is the reason that this video still plays without any stutter on SYTV 6.17.730 (which has exo 2.10.6) with the optimal video preset 4K ?

Can't really confirm. It is on a level with STubeNext 10.44 here. Micro-stuttering still happens and is also caused by CPU usage going to almost 100% at times.

[yuliskov]

I can't find anything suspicious except of exoplayer version.
Seems that exo 2.10.6 plays more smoothly on Sony BRAVIA KD-65XF9005

[CiNcH83]

I posted my findings over at the ExoPlayer repository.

[yuliskov]

@foreveryoungforty what do you say about 10.50?
It should perform exactly the same as the old app.
https://github.com/yuliskov/SmartTubeNext/releases/download/10.50/STubeNext_stbeta_v10.50_r.apk

[foreveryoungforty]

@foreveryoungforty what do you say about 10.50?
It should perform exactly the same as the old app.
https://github.com/yuliskov/SmartTubeNext/releases/download/10.50/STubeNext_stbeta_v10.50_r.apk

I will re-install and let you know by mail.

[kuafor41]

SmartTubeNext 10.56 is play smoothly (acceptable than others) except the video beginnig stutterings on MiBoxS with 1.0 playback speed, just fyi ...

[kuafor41]

I am receiving many frame drops [more frame drops than undropped ones -also some frame drops at FHD video-] with 10.59 version at 4k 60fps (1.0 playback speed). I dont know which operation mode is using on 10.59 exoplayer 2.10.6 but sytnext 10.56 was way better on 4k 60fps it has no frame drops with its exoplayer playback mode via MiBoxS !

Please turn back;
ExoPlayer: 2.12.1
Mode: OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD_ASYNCHRONOUS_QUEUEING

PS: old seeking preview style was better than in this version (10.59) from my point of view.

[yuliskov]

Thanks for the feedback.
I'll add an option to select preview style.
ExoPlayer also will be upgraded, but later.

[yuliskov]

I've added a build script for Windows named gradlew.bat. Now should be fine.

…

On Wed, Dec 30, 2020 at 2:41 PM Yuriy Liskov ***@***.***> wrote: update repo and try again On Wed, Dec 30, 2020 at 1:56 PM CiNcH83 ***@***.***> wrote: > JDK error solved. Now I get the following using this gradlew.bat > <https://github.com/gradle/gradle/blob/master/gradlew.bat>: > > > Task :smarttubetv:compileStorigDebugJavaWithJavac > D:\private\SmartTubeNext\smarttubetv\src\main\java\com\liskovsoft\smartyoutubetv2\tv\ui\playback\other\StoryBoardSeekDataProvider.java:10: error: class StoryboardSeekDataProvider is public, should be declared in a file named StoryboardSeekDataProvider.java > public class StoryboardSeekDataProvider extends PlaybackSeekDataProvider { > ^ > Note: Some input files use or override a deprecated API. > Note: Recompile with -Xlint:deprecation for details. > Note: Some input files use unchecked or unsafe operations. > Note: Recompile with -Xlint:unchecked for details. > 1 error > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#259 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABM7Z6G56KODTBI5YBN6EBLSXMINZANCNFSM4U2REZXA> > . >

[CiNcH83]

Again cloned into your master. Building still fails towards the end using your gradlew.bat. Attaching the command line output.
output.txt

[yuliskov]

Seems Windows naming glitch. Pls rename D:\private\SmartTubeNext\smarttubetv\src\main\java\com\liskovsoft\smartyoutubetv2\tv\ui\playback\other\StoryBoardSeekDataProvider.java to D:\private\SmartTubeNext\smarttubetv\src\main\java\com\liskovsoft\smartyoutubetv2\tv\ui\playback\other\StoryboardSeekDataProvider.java

…

On Wed, Dec 30, 2020 at 3:13 PM CiNcH83 ***@***.***> wrote: Again cloned into your master. Building still fails towards the end using your gradlew.bat. Attaching the command line output. output.txt <https://github.com/yuliskov/SmartTubeNext/files/5754293/output.txt> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#259 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABM7Z6HNOXFJNVQXN3QIUOTSXMRP5ANCNFSM4U2REZXA> .

[CiNcH83]

Build went through. Can't find any useful build result though. Does it take another step to create an installer apk?

[CiNcH83]

Hi @yuliskov

Google will close the respective ticket in 7 days if no further information concerning the above questions is delivered. I unfortunately failed to create an APK (no build\outputs\apk folder gets created) or deploy my build to the TV. Can we somehow pull off the above test?

I took a closer look at your code sample and I see your renderer factory is also overriding buildAudioRenderers() and buildVideoRenderers(). The code is replacing the MediaCodec audio and video renderers with custom ones. However, the MediaCodec operation modes need to be passed down to the MediaCodecVideoRenderer and MediaCodecAudiorRenderer instances, and I think the code does not do it at the moment (at least on the link you posted before). You should be able to set the operation mode on AudioDelayMediaCodecAudioRenderer and FrameDropFixMediaCodecVideoRenderer.

When playback starts, you should see this line being executed in the MediaCodec audio/video renderers when they initialize the MediaCodec internally.

Can you please verify the operation mode is passed down the MediaCodecRenderers correctly?

[CiNcH83]

Ticket has been closed.

[CiNcH83]

The issue is still pinned. So is this still a thing?

[yuliskov]

Yes. It's very important. I will return to it someday. Regards, Yuriy

…

On Tue, Mar 2, 2021, 10:26 CiNcH83 ***@***.***> wrote: The issue is still pinned. So is this still a thing? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#259 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABM7Z6HBPHKGFH6AOIEDZKLTBSOLXANCNFSM4U2REZXA> .

[CiNcH83]

OK, cool.

[costinmx]

I suppose this is the right place to report this. Just after version 10.122 I see many of dropped frames on some high bitrate (~30Mbps) videos. This one, very intensive, Bulgaria 8K HDR 60P (FUHD) is yet again causing the video player to drop a lot of frames at 5-10% rate. Earlier versions (last month) of STNext had only extreme minor issues with the same video. Maybe this video should be used for checking player or software upgrades before release, as it looks like one of the most proceesing intensive I've ever tried. Tested on Sony Bravia ATV3 49XG8377 (X850G)

[yuliskov]

@costinmx thx for the info. I wasn't aware that there is an regression.

[foreveryoungforty]

Indeed, serious amount of dropped frames and stuttering in that Bulgaria-video on my Sony 4K BRAVIA KD-65XF9005 too.

[flappy81]

Still micro stuttering on 4K@60fps on my Mi Box S. Maybe it's caused by a generally slow CPU as some have suggested earlier on other devices?

[costinmx]

After version 10.127 (up to current 10.141) looks like there is a major improvment regarding that Bulgaria HDR 60fps video. Only a few, rare, dropped frames now. After testing many other videos, it looks like it has something to do more with processing resources on HDR, than bitrate. I have seen a lot of 4K non-HDR videos with 40-60 Mbps bitrate, without a single frame dropped. Yet, that Bulgaria HDR video used to cause problems at just 30 Mbps.
Main thing is, I think it's a proper test video for a software release. If this Bulgaria HDR plays smooth, chances are you will never find another one with problems. At least, I didn't...
Thanks, Yuliskov.

[foreveryoungforty]

@costinmx
I agree with you.
After 10.127, there were some experimental releases (seperately published) with higher exo-players, but they were worse regarding 4K+HDR.
Version 141 returned to exo-player 2.10.6, which still is by far the best performant exo-player for SmartTubeNext when it comes to 4K (and higher) + HDR videos.
But indeed, I think the drops/lagging-problem is more a HDR-related issue than 4K/8K is.
For instance on my Sony 4K BRAVIA KD-65XF9005, if I set the video preset in the app from 4K 60fps VP9+HDR to 4K 60fps VP9, there is no lag/frame drop on test-video The World in HDR in 4K , and almost none visible on the Bulgaria video.
Maybe there is also a correlation with issue 556 (bandwidth limited to 10 mbps), which is being handled if i see the submits of Yuriy on april 1. Looking forward to the new version. Thanks indeed for all his persistent work to make this great app still better.

[costinmx]

if I set the video preset in the app from 4K 60fps VP9+HDR to 4K 60fps VP9, there is no lag/frame drop on test-video

Sorry I have just tested this today in ver 11.18 - I didn't notice any difference when changing max video preset to 4K 60fps (non-HDR) on my Bravia 49XG8377 (X1). Strange - test video droped frames exactly the same when playing non-HDR version also. Could not find any logic in this behavior other than maybe some 4K 60fps videos have a particular interaction with Bravia X1 video processor. But this looks more and more like a Sony issue, not SmartTube Next.

Hopefully soon I wiil get to test this issue against the newest 4K Chromecast with AndroidTV, see how it behaves.

[foreveryoungforty]

if I set the video preset in the app from 4K 60fps VP9+HDR to 4K 60fps VP9, there is no lag/frame drop on test-video

Sorry I have just tested this today in ver 11.18 - I didn't notice any difference when changing max video preset to 4K 60fps (non-HDR) on my Bravia 49XG8377 (X1). Strange - test video droped frames exactly the same when playing non-HDR version also. Could not find any logic in this behavior other than maybe some 4K 60fps videos have a particular interaction with Bravia X1 video processor. But this looks more and more like a Sony issue, not SmartTube Next.

Hopefully soon I wiil get to test this issue against the newest 4K Chromecast with AndroidTV, see how it behaves.

On version 11.18, I tested some videos regarding this open issue, and again I must conclude that on my Sony 4K BRAVIA KD-65XF9005 HDR, if I set the video preset in the app to 4K 60fps VP9, there is no lag/frame drop on test-video The World in HDR in 4K , and almost none visible on the Bulgaria video. If I put it on 4K 60fps VP9+HDR, there is always some little lag/stutter at the beginning of that 4K video, and visible more in several scenes on that 8k Bulgaria video.
That's the reason why I still think it's HDR/SmartTubeNext-related and the exo-player it uses (2.10.6 remains the best), especially when I realize I never had the lag/stutter issue with the last versions of the SmartYouTubeTV-app on the very same android tv. To be continued I think....

[Mysticodex]

So even if we switch the video to not 4k and not 60fps, it still happens the bad buffering... Any idea why?

[costinmx]

4k 60fps HDR is once again in trouble since versions 11.4x on Sony Android. I got 10-33% dropped frames on many high framerate HDR videos. Looks like it's a new exoplayer version (2.13.3). It was never mentioned in the update changelog, as I would have avoided this.
At some point everthing worked just fine, then comes either an Youtube api update or an ST Next update which ruins all the experience. So sad........ Will these issues ever end?...

Later edit: I had to correct and delete some claims I made in original post, since some issues were related to some interferring background apps. Sorry about that.

[yuliskov]

Exo has been upped just for the test. I'll revert old version (2.10.6) soon. Regards, Yuriy

…

On Sun, Jun 27, 2021, 03:21 costinmx ***@***.***> wrote: 4k 60fps HDR is once again almost unplayable since versions 11.4x on Sony Android. I got 33-50% dropped frames virtually on every high framerate HDR video. It's a bigger mess than ever before, all those videos have some degree of visible playback frame dropping issues. Looks like it's a new exoplayer version (2.13.3). It was never mentioned in the update changelog, as I would have avoided this. Also got continous buffering issues with many live streams (not all) that never had issues before. They never buffer more than 2-4 seconds max, then stall. At some point everthing worked just fine, then comes either an Youtube api update or an ST Next update which ruins everything. So sad........ Will these issues ever end?... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#259 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABM7Z6H773XKWXRSPMXAARLTUZVHJANCNFSM4U2REZXA> .

[costinmx]

Thank you, Yuriy. I know its really hard work these days. I also thought this exo was updated to solve some issues.
Then may I suggest, let it stay for the test as long as it's useful to you, and have some more feedback on this issue from other HDR fans also. I also like to test more to this issue, especially against original YT app. As always, Jacob + Katie Schwarz HDR channel is the best ground for testing on this issue.
I appologize again - Looks like buffering in live streams was not an issue at this time - as I rushed into conclusions. Sometimes in Android, there are some apps left in background having a bad influence on playback video performance, but also on buffering. In example - Live streams with less than 5 seconds buffering tend to stall when Android TV is low on resources. But they work just fine even with 0-2 seconds after you clear some apps from background.

[foreveryoungforty]

Indeed , 4k 60fps HDR is once again in trouble on Sony Android, even without HDR, way more stutter in 4K videos.

[costinmx]

I also notice a slight degradation in image sharpness on some regular 4K videos... Especially in 25Hz videos where Sony motion compensation does not seem to work properly anymore. 30/50/60 Hz are fine in motion.

Edit: Just tested - Strange, some of this also happens in original YT app... Does not look very different than 2.13 exoplayer. Maybe something changed on YT end, also?

[costinmx]

Everything works fine on Sony XG83 (X850G) again after 11.46 update. 4K and HDR back to a smooth sharp playback.
Some of the HDR test videos even play better with this Exoplayer than the actual original YT app.
Try these clips to see how it works for you:
Uzbekistan 8K HDR 60p
Watch the scenes with small blue tiles
STNext: Dropping few frames / YTube: Dropping all the frames
Blooming HDR 8k Dolby Vision
Watch the carriage scene
STNext: Dropping frames / Ytube: Dropping all frames (carriage not visible)

Also Super Walker channel HDR videos in Paris, and VIRTUAL JAPAN channel HDR videos in Tokyo, Osaka play so smooth now. With old Exoplayer they were small glitches once in a while

For my TV everything seems fine, for now. Live streams buffering ok, including 4K, no buffer issues, no video decoder errors or "can't download video"... Let's hope it's the same for everybody. Thanky you very much Yuri for your commitment to this project. Even if someone decide to go premium on YT, it will still miss the amazing features ST Next has, both the smooth android TV adapted interface and the player (PIP function is fantastic).

[CiNcH83]

I wonder what the problem actually is with newer versions of ExoPlayer in SmartTubeNext. I am currently watching European Football Championship via ORF TVthek in Austria. The app uses ExoPlayer 2.12. Stream is 2160p50 and it is perfectly smooth.

[costinmx]

Could not help to notice on some tests - somehow 11.78 plays best 2160 60fps HDR videos. Very rare hard to spot frame drops.

Pity this update is having such bad issues, like crashing on Sony Action Menu. Are these two matters connected?
#787 (comment)

[CiNcH83]

ExoPlayer devs now changed the way to enable asynchronous MediaCodec in the dev-v2 branch. It is not experimental anymore and will be part of the next release 2.16. Asynchronous MediaCodec is enabled by default for devices running Android 12. It can however be enabled for devices running Android 6 Marshmallow and above via DefaultMediaCodecAdapterFactory.

Commit

[CiNcH83]

ExoPlayer 2.16 has now been released.

Enable MediaCodec asynchronous queueing by default on devices with API level >= 31. Add methods in DefaultMediaCodecRendererFactory and DefaultRenderersFactory to force enable or force disable asynchronous queueing (6348).