Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Twitter Discovery

Twitter Discovery lets you search articles referenced from your tweets, retweets, and favorites.


Install Twitter Discovery from source using pip install -e .. (And do yourself a favor and use venv.)

Also install twitter-to-sqlite using pip install twitter-to-sqlite. This is the tool that you'll use to download data from Twitter. Follow the instructions in the twitter-to-sqlite docs and set up authentication with Twitter.


Note that dnouri is my own Twitter handle, so whenever you encounter that in the commands below, you should replace dnouri with your own Twitter handle.

Here's how you first download data from Twitter:

  • Retrieve your tweets and retweets using twitter-to-sqlite user-timeline dnouri.db.
  • Add favorites using twitter-to-sqlite favorites dnouri.db.

Now you're ready to find in the tweets that you downloaded any links to news articles and blogs. The following command will find all such URLs and it will attempt to download the linked documents and save them for further use: twitter-discovery articles dnouri.db dnouri.

Finally, use the web-based GUI to search all those articles. To start it up, run: streamlit run twidis/ gui articles-dnouri.csv.gz.

The GUI allows you to search articles by text, order them by date or website, and so on. The text search is case insensitive and supports the use of quotation marks to group words, such that a search for "big finance" will return articles that contain the phrase big finance, while searching for big finance without the quotes will return articles that contain both the word big and the word finance. The text search will match text inside the article's body, but also in the article's title and the text of the tweet that referenced the article.


No description, website, or topics provided.







No releases published


No packages published