Prevent YouTube throttling

[putara]

Hi there, I'm trying to fix #2194.

Though this is unfinished and a bit unstable, the parch seems working on PC/RPI4.
(I'm testing on a docker instance. #2147 needs RUN apk add --no-cache make to build this patch.)

Unfortunately it now requires a real JS interpreter (duktape) because the decryption code was too complicated without it.

ytdl-org/youtube-dl#29326 says

This does change with different player versions, so youtube-dl will need to extract this for every video that it fetches and then modify the n parameter as such.

but it's not changed so often, and fetching it for every video is pretty expensive. This needs more investigating.

Anyway, I implemented two level caches to prevent refetching it for every video. The decryption function is called dozens times on page load. (you can see cache #1 and cache #2 in the debug log)

invidious_1  | 2021-06-28 05:54:47 UTC [debug] decrypt_n: 1LaKH5etJoE fn = cha n = RFcl_9VK-CZhPMH89IYrX -> Fcw3Gl6gxDsX_A
invidious_1  | 2021-06-28 05:54:47 UTC [debug] decrypt_n: 1LaKH5etJoE fn = (cache #1) n = RFcl_9VK-CZhPMH89IYrX -> Fcw3Gl6gxDsX_A
invidious_1  | 2021-06-28 05:54:48 UTC [debug] decrypt_n: 1LaKH5etJoE fn = (cache #2) n = Fbc5fFBLqpj0fJOfAj4Gr -> 48_sUCeKYn_LHg
invidious_1  | 2021-06-28 05:54:48 UTC [debug] decrypt_n: 1LaKH5etJoE fn = (cache #1) n = Fbc5fFBLqpj0fJOfAj4Gr -> 48_sUCeKYn_LHg
invidious_1  | 2021-06-28 05:54:48 UTC [debug] decrypt_n: 1LaKH5etJoE fn = (cache #1) n = Fbc5fFBLqpj0fJOfAj4Gr -> 48_sUCeKYn_LHg
invidious_1  | 2021-06-28 05:54:48 UTC [debug] decrypt_n: 1LaKH5etJoE fn = (cache #1) n = Fbc5fFBLqpj0fJOfAj4Gr -> 48_sUCeKYn_LHg
   :

[unixfox]

Hello,

Thank you for your contribution.

In the PR #2220 I'm working on fetching the videoplayback URLs from the internal YouTube API using an Android client.

I do think the given videoplayback URLs aren't throttled but I'm not sure that's why could you take a look at?
By either running Invidious patched with the PR #2220 or by fetching the videoplayback URLs manually using the technique described here: TeamNewPipe/NewPipeExtractor#562

[putara]

@unixfox Thanks, I will install #2220 locally and see if it can avoid throttling.

If the internal API works without any decryption, that will be great.
I personally am not a fan of polling a decryption function every minute. 😅

[SamantazFox]

You should split the fetching and execute parts, btw. Add a Job to fetch the function every hour or every day, idk, store the string, and then run it when a video needs it.

[SamantazFox]

Marking this as "blocked" because this fix is not required anymore since #2220 has been merged.

@putara While we honnestly appreciate and thank you for your work, we don't want to add an extra dependency now, and will reconsider this PR if the problem occurs again.

[putara]

@SamantazFox Yes, I totally agree with you. I will close this PR.