Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an option for limiting the age of collected tweets #29

Open
milo-trujillo opened this issue Oct 8, 2020 · 0 comments
Open

Add an option for limiting the age of collected tweets #29

milo-trujillo opened this issue Oct 8, 2020 · 0 comments

Comments

@milo-trujillo
Copy link
Member

At present we collect the most recent ~2000 tweets from each user, per Twitter API limits. This can give an uneven approximation of a community, since it does not represent "recent" interactions in a true time sense, and will include the most recent tweets from a user even if they haven't tweeted in months.

Propose adding a --maxage -A flag to the script for specifying an integer (or float?) number of days. No tweets older than this threshold will be collected.

Implementation will require adding optional oldestDate argument to acquireTweets, which can take an epoch time stamp. If any tweets older than that timestamp are detected, break out of the for-loop currently at:

SocMap/acquire.py

Lines 101 to 113 in c8e9f40

for tweet in limit_handled(api, cursor.items()):
mentions = getMentionsFromText(tweet.text)
date = tweet.created_at
text = tweet.text
source = tweet.user.screen_name.lower()
if( hasattr(tweet, "retweeted_status") ):
orig_author = tweet.retweeted_status.user.screen_name.lower()
rt_count = tweet.retweeted_status.retweet_count
rt = Retweet(source, text, date, mentions, orig_author, rt_count)
tweets.append(rt)
else:
tw = Tweet(source, text, date, mentions)
tweets.append(tw)

This will not require any changes to analysis.py, or auxiliary tools, or any new interactions with tweepy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant