Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use etag to cache youtube responses #90

Merged
merged 5 commits into from
Oct 23, 2020

Conversation

dotted
Copy link
Contributor

@dotted dotted commented Oct 22, 2020

This should drastically reduce quota usage for youtube api.

Questions:

  • Should the old cache logic be removed?
  • Should we dump the this.etags into the DB to persists the etags further reducing quota cost?

Todo:

  • Fix tests

@johnpyp
Copy link
Collaborator

johnpyp commented Oct 22, 2020

Since the bot is generally long-running for 24h+ I doubt it's worth it to dump etags into the db, unless we're still running into api quota issues after this change.

Other than deleting a stale comment, LGTM

lib/services/youtube.js Outdated Show resolved Hide resolved
@shadedred
Copy link
Collaborator

This should hopefully provide better caching, sufficient enough to keep us under the quota. The next solution here, should this still hit the quota, is to not us the search API, and get destiny to generate an authenticated key on his account itself that can use the https://developers.google.com/youtube/v3/live/docs/liveBroadcasts/list list (my broadcasts) endpoint, eliminating the search call, which is about 160 times more expensive than every other call in terms of quota.

@dotted dotted marked this pull request as ready for review October 23, 2020 01:03
@dotted
Copy link
Contributor Author

dotted commented Oct 23, 2020

Pretty sure dumping this.etags into the DB and loading it on startup is much less involved than trying to get Destiny to get us that key. 🤣

@shadedred
Copy link
Collaborator

I agree 😆 but lets give this a shot and see how the quotas fair tomorrow

@shadedred shadedred merged commit 306fb4b into destinygg:master Oct 23, 2020
@dotted dotted deleted the youtube-omega-cache branch October 23, 2020 01:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants