tScrape

A simple Python scraper and parser for Twitter pages using Beautifulsoup 4 and Selenium webdriver. Parser output is json, see details below.

Install requirements

pip install -r requirements.txt

In run.py:

python run.py

For each Twitter handle the parser will output two json files:

handle-stats.json with the keys:

handle-tweets.json with an entry for each tweet containing the keys:

To import the results as a Python dictionary use the jsonLoad function:

from tScrape import jsonLoad
results = jsonLoad(handle, path)

Use the standard dictionary methods to inspect the content:

results.keys()
results.values()

for key en results.keys():
    print key, results[key]