Skip to content


Subversion checkout URL

You can clone with
Download ZIP


New feature: Shrink Your ThinkUp Database #1313

ginatrapani opened this Issue · 3 comments

3 participants


By default, ThinkUp is greedy about how much data it captures from social networks, and much of it (like posts from friends) is never displayed in ThinkUp's interface.

At the same time, for busy accounts on hosting providers with database size limits, TU's database can grow to a point where either a.) the user has to upgrade to a hosting account with higher limits and/or b.) database maintenance, migrations, moving, and export features become untenably slow and difficult to complete.

"Shrink Your ThinkUp Database" is a new feature which:

  • Deletes all inactive follower relationships from the follows table
  • Deletes all posts which are not authored by the service user, in reply to, in retweet of, or favorited by the service user
  • Deletes all users who are not in follows or posts
  • Averages out follower count history by day and deletes all but one record with a follower count for that day from the follower_count table

What else can we delete?

We could also consider adding a "Keep my ThinkUp database light" admin setting which will run this shrink job every week or so during crawls.


Should default behavior on these be to write them into an export file that's not web-accessible, but is app-accessible if needed? That way, if the user upgrades or migrates or just wants to bring the data back in, it's not lost, it's just not taking up space in the DB and is retrievable, albeit awkwardly. I can think of, for example, inactive follower relationships or unused user records being something that may become interesting to a user in the future.


It would be difficult to keep these backup files from not being web accessible. (TU doesn't have access to a folder that isn't in the web path by default.)

One suggestion: we could create posts_archive and follows_archive tables to store the stuff we're taking out of the active tables.


I'd like to push for this feature. I've been happily running thinkup for a few months now and nearly did a spit take when seeing that it was using 400+ megs of database storage. I'd happily purge most of this just to save that space.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.