Use Redis for caching

[SamantazFox]

Closes #3628

[github-actions]

This pull request has been automatically marked as stale and will be closed in 30 days because it has not had recent activity and is much likely abandoned or outdated. If you think this pull request is still relevant and applicable, you just have to post a comment and it will be unmarked.

[syeopite]

What do you think about splitting the video cache into two? One for the streaming urls and the other for the metadata. We'd be able to persist the metadata for a lot longer than 6 hours then.

[SamantazFox]

What do you think about splitting the video cache into two? One for the streaming urls and the other for the metadata. We'd be able to persist the metadata for a lot longer than 6 hours then.

That could be doable, I think. I guess you want the metadata to be used for imports or playlists?

[syeopite]

I guess you want the metadata to be used for imports or playlists?

Yep. From what I can see Invidious also often calls #get_video to fetch video metadata only so splitting the cache can probably also save us quite a few calls to YouTube.

[SamantazFox]

In the end, I think I'll go with that separation:

videos:<id>:title

• Stores: the video title
• Format: raw text
• Retention: as long as possible (2 months?)

videos:<id>:release_date

• Stores: the exact date at which the video was released
• Format: ISO date
• Retention: as long as possible (1 year?)
• Updated from: watch page or channel RSS feed

videos:<id>:playback

• Stores: the video's playback infos (captions/storyboards/streams)
• Format: JSON
• Retention: 5h, to keep the links fresh, except for livestreams where this is reduced to 1h.

videos:<id>:metadata

• Stores: all the miscellaneous metadata for the watch page
• Format: JSON
• Retention: a few hours if the video was released recently, to keep views, likes and description fresh. After that, keep for a day or two.

[syeopite]

Titles should probably just be included as a part of metadata imo. It's something that changes fairly frequently (especially when the video is new) and in some cases frequent title changes are the entire point of the video such as this one https://redirect.invidious.io/watch?v=BxV14h0kFs0

As for playback, you can probably split off storyboards from it. It's not something that should change often if at all.

[SamantazFox]

Titles should probably just be included as a part of metadata imo. It's something that changes fairly frequently (especially when the video is new) and in some cases frequent title changes are the entire point of the video such as this one https://redirect.invidious.io/watch?v=BxV14h0kFs0

I know that titles may change (especially true when the video was recently released, to boost views). The main goal is to address #1356 and/or #3959 + #3035, which requires a long cache time.

As for playback, you can probably split off storyboards from it. It's not something that should change often if at all.

So you'd wand it with the metadata? Or separated? In all cases, all data will be refreshed whenever the video needs to be played.

[syeopite]

So you'd wand it with the metadata? Or separated?

Either or is fine. I'm just not entirely sure why captions and storyboards are placed under videos:<id>:playback

[github-actions]

This pull request has been automatically marked as stale and will be closed in 30 days because it has not had recent activity and is much likely abandoned or outdated. If you think this pull request is still relevant and applicable, you just have to post a comment and it will be unmarked.