An automated web scraper built with Python that mines social post data from a feed of search results. The data is exported into a CSV for further analysis.
The user can specify:
- Search term
- Recent vs popular posts
- Quantity of posts
The below steps are for installation on MacOS.
- Python 3.11.1 or higher with pip. Install with brew:
brew install python && python3 -m pip install --upgrade pip
- Selenium and BeautifulSoup Python packages:
pip install selenium beautifulsoup4
-
Chrome or Chromium-based browser installed on your system.
-
Valid social account.
NOTE: Multi-factor authentication (MFA) must be turned OFF on your social account.