Skip to content

digitalepidemiologylab/mkondo

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

64 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Mkondo is Kiswahili for 'stream'. 

This library contains code that helps download Twitter Streaming API data and manage that data. 

It relies on the Salathegroup's fork of Tweepy
(https://github.com/salathegroup/tweepy) which allows the raw JSON received to
be written out to disk. 

Mkondo provides the following features: 

1. extends Tweepy classes to include appropriate backoffs for Twitter's Streaming API. 

2. code to store and backup tweets from the Streaming API (this includes
rotating directories and data files when appropriate and logging errors as
necessary)

3. classes to help parse and analyze the tweets as stored on disk.

4. a class to help use the Twitter REST API continuously, taking appropriate
breaks when the hourly limit for REST API calls is reached. 

We use this module to continuously collect data from Twitter with limited downtime. 

khandelwal@psu.edu, salathe@psu.edu

About

A library for downloading and managing Twitter streams of data.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages