Simple web-server for collecting tweets, metadata, and comments for YouTube videos and estimating their slant based on the approach used in our audit of YouTube's recommendations. Read more about it here. The server caches responses inside data/database.db so as to avoid rate limits on the used APIs.
- Clone the repository.
- Generate your Twitter v2 and YouTube API tokens.
- Copy the Twitter tokens to data/twitter-tokens.txt and the YouTube tokens to data/youtube-tokens.txt respectively, one per line.
- Download youtube-dl to a location in your PATH.
- For slant estimation, download the
landmark-counts.tar.xzby filling out this form and extracting its contents to the data directory. - Install the requirements using
pip install -r requirements.txtinside the repository. - Start the server using
flask run --port 5000.
Consider the following URL: https://www.youtube.com/watch?v=XpeOIww_l4A. The part of the URL after the watch?v= is the video ID. The various endpoints of the server take this video ID and return the corresponding data. For example, to get the metadata for this video, you can use cURL as follows: curl http://localhost:5000/getMetadata/XpeOIww_l4A
In all these endpoints, replace <video-id> with the ID of the video, which can be extracted from the URL as in the example above.
/getMetadata/<video-id>: Get video metadata usingyoutube-dl./getTweets/<video-id>: Get tweets mentioning the video using the Twitter API./getComments/<video-id>: Get comments on the video using the YouTube API./getSlant/<video-id>: Get the estimated slant for the video using our approach.
This tool was developed as part of an effort by researchers at UC Davis to audit the recommendations on YouTube. Read more about it here.
The primary maintainer is Muhammad Haroon.