Video throttling and/or 403s during playback with Android client

[user234683]

Following the fix to #165, updating the android client version, videos using these Android URLs would give 403s at a defined position into the file (which manifests itself as the video freezing and then being unable to play it at all, not even by switching quality). I can not figure out why this is happening.

Other projects (invidious, yt-dlp, newpipe) are not having this issue from what I can tell, and I can't find any difference between how we're making the request that accounts for this (there are minor differences in what is included in the client field, such as including osName and such, or the exact version of android in the user agent, but making these align did not fix the problem). I do see a couple of crashes in Newpipe in the last week that look similar to what we're experiencing, but no mention of 403s and not a lot of activity in them

Because of this, I reverted to the URLs returned from regular requests using the mobile client. These are throttled by default unless we were to implement nsig decryption as exists in yt-dlp (android URLs do not require any decryption). That is why there is constant buffering.

At the moment, both of these solutions more frustrating than refreshing three times, but that AB testing appears to be totally closed off, always returning video unavailable, so reverting both commits isn't an option either

There's two options:

1. Figure out why the android URLs are doing this
2. Copy the nsig implementation from yt-dlp to fix (most) throttling. Useful starting point. More info

I don't have time at the moment to do this so would appreciate any help from someone who knows what's going on

[God-damnit-all]

I believe it's because both those projects download in 10MB chunks

Here's Invidious: https://github.com/iv-org/invidious/blob/9dfa268204f20b6eba65d3f0977a590e3fbfc120/src/invidious/routes/video_playback.cr#L120 (HTTP_CHUNK_SIZE is 10485760 which is ~10MB)

And here's yt-dlp: https://github.com/yt-dlp/yt-dlp/blob/4d9280c9c853733534dda60486fa949bcca36c9e/yt_dlp/extractor/youtube.py#L3671

Unfortunately I don't really know how to implement this into youtube-local, but I think this is the core of the issue.

Also, you should also pass 'params': '8AEB', in the root of the JSON request, as this lets the Android client play YouTube Stories.

[user234683]

@ImportTaste Current chunk size is 262144 bytes, set in line 116 in server.py if you want to try it. Can't check if it works until tomorrow night

Edit: Those are the TCP chunks, so that would be the wrong value to change. Looking more closely, now I see what you're pointing at is that they split up the single range request into multiple range requests in 10 MB chunks, so we would need to split it up into a for loop like they do.

[God-damnit-all]

@ImportTaste Current chunk size is 262144 bytes, set in line 116 in server.py if you want to try it. Can't check if it works until tomorrow night

Edit: Those are the TCP chunks, so that would be the wrong value to change. Looking more closely, now I see what you're pointing at is that they split up the single range request into multiple range requests in 10 MB chunks, so we would need to split it up into a for loop like they do.

That seems to be the case. it's beyond my ability to implement, but I hope I was of some help regardless.

[user234683]

It appears that's not the issue. As it is, video file requests using av-merge.js already split the requests into small range requests. Additionally, I ran an experiment where I inserted urls from yt-dlp into youtube-local, and the video played fine without 403s, so something is wrong with the URLs themselves. Looking into it

[user234683]

The params was the problem, thanks for pointing that out. Latest release fixes it

[God-damnit-all]

There's an age-restricted video that is exhibiting the same throttling behavior that I experienced with this issue. Is it related to #146?

[user234683]

There's an age-restricted video that is exhibiting the same throttling behavior that I experienced with this issue. Is it related to #146?

Yes, because for those it uses the TVHTML5_SIMPLY_EMBEDDED_PLAYER client instead of the ANDROID client. Android doesn't require decryption. TV embedded does, and one encryption thing is that nsig which causes the video to be throttled