Skip to content
Scrape the Twitter Frontend API without authentication.
Python
Branch: master
Clone or download
bisguzar Merge pull request #64 from mpeg31/master
Bugfix for TypeError: get_tweets() got an unexpected keyword argument 'user'
Latest commit 2183779 Jul 17, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore Pycharm's .idea folder now ignored May 7, 2019
LICENSE Add MIT license Mar 4, 2018
Pipfile updates Mar 21, 2018
Pipfile.lock
README.rst Add test and update README file Mar 17, 2019
setup.py updated setup.py to install .. code-block:: pycon Jun 29, 2019
test.py Fix test.py Jul 17, 2019
twitter_scraper.py Better Stability Jun 30, 2019

README.rst

Twitter Scraper

Twitter's API is annoying to work with, and has lots of limitations — luckily their frontend (JavaScript) has it's own API, which I reverse–engineered. No API rate limits. No restrictions. Extremely fast.

You can use this library to get the text of any user's Tweets trivially.

Very useful for making markov chains.

Usage

>>> from twitter_scraper import get_tweets

>>> for tweet in get_tweets('kennethreitz', pages=1):
>>>     print(tweet['text'])
P.S. your API is a user interface
s3monkey just hit 100 github stars! Thanks, y’all!
I’m not sure what this /dev/fd/5 business is, but it’s driving me up the wall.
…

It appears you can ask for up to 25 pages of tweets reliably (~486 tweets).


If you're interested in financially supporting Kenneth Reitz open source, consider visiting this link. Your support helps tremendously with sustainability of motivation, as Open Source is no longer part of my day job.

Markov Example

First, install markovify:

$ pipenv install markovify
>>> import markovify

>>> tweets = '\n'.join([t['text'] for t in get_tweets('kennethreitz', pages=25)])
>>> text_model = markovify.Text(tweets)

>>> print(text_model.make_short_sentence(140))
Wtf you can’t use APFS on a prototype for “django-heroku”, which does a lot out of me.

Installation

$ pipenv install twitter-scraper

Only Python 3.6+ is supported

Test

$ python test.py

LICENSE

MIT

You can’t perform that action at this time.