DownloadThisVideo
On April 4, 2023, Twitter suspended @this_vid, after nearly 5 years and millions of users. 😢
Easily download videos/GIFs off Twitter. Mention the bot (@this_vid) in a reply to the tweet containing the video, and it'll reply with a download link in a few minutes.
How this works
Stack
- AWS Lambda with the Serverless Framework
- AWS SNS
- Redis
- Node.js
Implementation
The bot consists of several AWS Lambda functions that work in tandem:
fetchTweetsToDownload
This function runs every 4 minutes and checks for new mentions. It publishes these new mentions as a new notification on an SNS topic. The 4-minute interval is so as to not hit Twitter's rate limits and minimize AWS Lambda usage time, while being near-realtime.
sendDownloadLink
This is triggered by new notifications on the SNS topic. It:
- processes the tweets in the message body,
- calls Twitter's API to retrieve media links. Any video links retrieved for a tweet are stored in Redis for faster repeated access (other users requesting the same video).
- adds the download details to the user's store in Redis. The user's store is an entry in Redis where all downloads requested by a user are cached for a certain period (48 hours).
- attempts to reply to the user with a link to the user's download page (see section below). "Attempts" because Twitter enforces tweet limits (2400 per day, counted in 15-minute periods). If the API limits have been reached, the bot will "cool down" (not send any replies) for 10 minutes.
getDownloads
This is triggered by a HTTP request to the / (for instance, http://thisvid.space/jack). It renders a page showing a list of the user's recent downloads.
getHomePage
Renders the homepage 😁. See http://thisvid.space.
retryFailedTasks
This re-publishes failed tasks (stored in Redis) as a new SNS message. For now, it can only be triggered manually.
