Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache requests from older periods #15

Open
HugoJF opened this issue Nov 11, 2020 · 1 comment
Open

Cache requests from older periods #15

HugoJF opened this issue Nov 11, 2020 · 1 comment
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@HugoJF
Copy link
Owner

HugoJF commented Nov 11, 2020

After the update I released attempting to circumvent Twitch's weird API pagination that lead to lots of clips missing from bigger channels, fetching the every single clip from a big channel started taking a long time.

The idea is to locally cache requests for clips from periods older than the current day, that way the tool can be restarted without having to paginate the entire API again.

@HugoJF HugoJF added enhancement New feature or request help wanted Extra attention is needed labels Nov 11, 2020
@HugoJF
Copy link
Owner Author

HugoJF commented Nov 13, 2020

This was partially implemented in 51f860e.

It currently caches every call to the fetchClips() function in cache/{streamerId}-clips, where each call (currently a period of 7 days) is stored in it's own file.

A few improvements for the future:

  • Transparent cache implementation, it's currently a bit too verbose;
  • Don't cache the most current period;
  • Use actual stream name in the cache directory;
  • Handle exceptions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant