Fix throttled streaming by using a new route

[RNavega]

Edit: For anyone dropping by, this PR had a bug.

If you want to grab the files for testing, you should take the ones from anxdpanic's fix in this later PR: https://github.com/anxdpanic/plugin.video.youtube/pull/216/files

Original text below:

---

Special thanks to @tssajo for pointing this solution out, and to @FireMasterK for publishing it in the first place.

This PR adds a specific API request that gets the ytInitialPlayerResponse data and it comes with all the streams without the n parameter and without the need to decipher the signature parameter. So that n calculation and signature deciphering code can be removed.

This PR also changes some header usage so that all video items use a User-Agent spoofing an Android device (otherwise Kodi sends out a MIME type request with Kodi's own User-Agent, making it obvious it's coming from a Kodi plugin).

Some testing needs to be done with age-restricted videos to see how / if they still work under these new changes, but logged-out playing is working again without throttling at least.

[RNavega]

In case this route is ever blocked and you're back to having to deal with n parameters and signature deciphering, there's yet another challenge, which is the need to have each chunk request not using the Range: bytes={start}-{end} header that Kodi uses, but a custom &range={start}-{end}&rn={requestNumber}&rbuf={receiveBuffer} set of parameters in the chunk URL used by the YouTube player.

There's more info about those in here:

• raszpl's comment in youtube-dl
• Candid with YouTube: Adaptive Streaming Behavior and Implications on Data Consumption (a PDF, explains about this range parameter, as well as the rbuf and rn that change per each chunk request)

If you don't use those parameters you get throttling. I think the only robust way to handle these, if ever needed, is to make a dummy HTTP server in localhost and tell Kodi to stream the file through it. Then you take Kodi's chunk requests and translate them to these YouTube parameters, stream the data and pass it back to Kodi. So while Kodi is playing the video this custom server handled by the plugin is being a middleman. Edit: or making a fork of InputStream.Adaptive that does this.

[anxdpanic]

@RNavega
Thanks again for keeping on top of it, I should have more time after the next week and a half-ish.
The YouTube add-on has it's own httpd that serves the mpd via proxy.

Does this return an mpd url, or just urls that are currently unaffected?

[anxdpanic]

@RNavega
This was broken (by the last commit). I've pushed a fixup and clean up, but a few issues remain before I can push an alpha.
Age-gated and some other videos don't work now, and My Channel videos are broken as well.

[RNavega]

Thank you for the fixes and cleanup!

Does this return an mpd url, or just urls that are currently unaffected?

If I understood you right, it gives the same kind of player_response data that the original code used, but the URLs are unprotected. So the generate_mpd() function will still build a MPD file out of the URLs in the adaptiveFormats list.

a few issues remain before I can push an alpha.
My Channel videos are broken

As you noticed, that was being caused by that last change I made, restoring the "Authorization" header. It looks like the API request fails with a HTTP 400 "Bad Request" error if you do that, I don't know why. But thankfully you fixed that by removing the lines that added this header when the user is logged in.

The problem is that if we can't use this login token, then age-gated or restricted videos won't be watchable. I don't think I'll be able to fix this because I can't debug it.
I wonder how @FireMasterK came across that API route, as I think it's from the YouTube app on Android. I tried sniffing the traffic from that app on my phone using Fiddler + a proxy, but all the requests came as "tunneled" (no way to see the contents or headers). So this makes it impossible to try and find out what kind of authorization or user login info it sends when requesting streams for age-gated, unless someone else can find this out.

Edit: some more info in these:

• https://gist.github.com/leptos-null/8792b9c50fddc00cf525ed5055a872dc#steps-to-step-1 (How to use this innertube authorization?)
• Alternative ways to bypass age restriction

[tssajo]

So this makes it impossible to try and find out what kind of authorization or user login info it sends when requesting streams for age-gated, unless someone else can find this out.

@RNavega Thanks for the patch! I'll test it later today on libreelec ( Pi 3 B+ )

Regarding age-gated videos not working, please see this comment:
TeamNewPipe/NewPipeExtractor#562 (comment)

[FireMasterK]

I wonder how @FireMasterK came across that API route, as I think it's from the YouTube app on Android. I tried sniffing the traffic from that app on my phone using Fiddler + a proxy, but all the requests came as "tunneled" (no way to see the contents or headers). So this makes it impossible to try and find out what kind of authorization or user login info it sends when requesting streams for age-gated, unless someone else can find this out.

I installed the certificate as a root (through Magisk's systemless hosts module) one and used mitmproxy since Fiddler can't read protobuf.

[RNavega]

@tssajo ok, before you test this PR please note that it added a bug.
This has been fixed in this later one: https://github.com/anxdpanic/plugin.video.youtube/pull/216/files

Thanks for that pointer on the age gating. When I tested other client names, as Unixfox pointed out in that thread, any client name other than just "ANDROID" will give out streams protected by the n and signature parameters that need decoding.
So we're stuck using this "ANDROID" client name, unless there's a convenient way to handle those chunk parameters mentioned above because those chunk parameters are used with these protected streams.

I installed the certificate as a root (through Magisk's systemless hosts module) one and used mitmproxy since Fiddler can't read protobuf.

@FireMasterK Thank you, that makes sense.