A twitter scraper written in python using BeautifulSoup, Selenium, and SQLite. Stores individual tweet urls for given user in a SQLite3 database. Requires Firefox/GeckoDriver or Google Chrome/ChromeDriver for Selenium.
TO DO LIST (in no particular order):
- Change scroll method to speed up url collection
- Proper logging
- Individual tweet parsing (likes, retweets, replies)
- Add ability to scroll until end of timeline
- Add different web drivers
- Add different database types (maybe)
- Add ability to choose time frame
- Add ability to view/browse the database
- Add ability to distinguish between retweets and user tweets
- Add ability to update an account instead of having to reparse everything