Skip to content

Missed videos (and tests)

BluABK edited this page Aug 30, 2018 · 1 revision

What's the deal with these "missed videos"?

YouTube API's list() and search()

YouTube API offers two methods to look for videos, list() and search().

list() returns a title-sorted list of items that is loosely (not consistent) sorted by recently published, it seems to group videos of the same title-sort/series if they're released within a certain time span, but then again sometimes it does something else entirely. Bottom line is, it's completely unpredictable and unreliable.

search() allows you to send in arguments like sort-by-date, which is processed server-side and thus you get a sane date-sorted list in return.

The YouTube API quota conundrum

So it should be a simple decision, right? Just use search(). The kicker is that search() costs a whopping 100 quota units per request, whereas list() - in our case - costs 3 units per request. So purely using search() makes you hit the 1M quota unit daily cap in no time at all.

The compromise

At the end of the day we ended having to make our own little algorithm, which will compare requests from list() and search() and check for missing videos in the former. This greatly reduced the API quota use.

The config

  • Test pages: How many pages of list() responses required to have the result of a search() page.
  • Missed video limit: How far to traverse in search() before you hit a video that is not in list()