django-syncr

A collection of Django applications for saving your activity on other websites (such as Flickr, Twitter, YouTube, etc) in your local database.

This project was begun by Jesse Legg at Google Code, which seemed to stop in 2008. Dan Fairs did some updates until 2010. Phil Gyford is currently working on this fork. See "Changes from earlier versions" below for what's new. Other than this README, docs have not been updated for a while.

NOTE: As of v0.60, this branch has switched from using django-tagger to django-taggit for those apps which feature tags. Upgrading an existing django-syncr install to this version will require some manual changes (the included South migrations may help).

See the Issues for known new issues, or the Google Code issues for older ones, possibly still extant.

I have only tested these apps using Django 1.3 with Python 2.6.

Overview

django-syncr consists of one app per third-party service. Each app has a set of models to represent its entities (users, photos, tweets, etc) in your database, and utilities to fetch the remote data and store it locally.

It's possible to install only the apps for the services you want to sync with. Syncing is only one-way (changes to your local database are not pushed to third-party services). If changes are made on third-party services, it's not automatic that those changes will be pulled back to your local database.

django-syncr provides no views or templates for viewing your locally-stored data, although you can access it all via the Django admin if you're using that.

South migrations are available for all apps.

Phil Gyford, 13 August 2011.

Services

Here are the services covered, with their dependencies, and the rough known state of their functionality.

Twitter

Roughly tested, appears to work.

Requires: python-twitter

Can fetch a user's Tweets, lists of Friends and Followers, and Friends' Tweets. The quantity of Friends and Followers is currently quite limited. Stores the in-reply-to and latitude/longitude of each Tweet, if available, and most of the users' data.

Flickr

Roughly tested, appears to work.

Requires: flickrapi, django-taggit.

Can fetch a user's public photos, recent photos, public favorites, photo sets. The Photo model can store most/all data, including geo and EXIF data, and data about tags. Comments on photos are also stored. The images themselves are not stored locally.

Delicious

Roughly tested, appears to work.

Requires: django-taggit.

Fetches a user's bookmarks. It can fetch all of them, the most recent, a specific tag, or a specific date.

YouTube

Roughly tested, appears to work.

Requires: django-taggit

Fetches a user's playlists, favorites and uploads. Stores data about videos such as view counts, length, descriptions, tags, etc. The videos themselves are not stored locally.

Picasa

Roughly tested, appears to work.

Requires: gdata, django-taggit.

Fetches data about photos from all of a user's Albums, or a specific Album.

Google Code, Readernaut, Tumblr

Not yet checked.

Brightkite

No longer works.

Brightkite have removed their "check in" functionality which this app used to sync.

### Magnolia

Service no longer exists.

Installation

Based on the original docs at Google Code.

1. Create your own fork or check out this one:

   git clone git@github.com:philgyford/django-syncr.git

2. Add syncr to your PYTHONPATH. This might just involve putting the 'syncr' directory in your Django project, or symlinking it there.

3. Add the required apps to the INSTALLED_APPS in your Django settings file. eg:

   INSTALLED_APPS = (
   ...
   'syncr.flickr',
   'syncr.twitter',
   )

4. Write code that will access the methods in the appropriate module in syncr/app/*.py eg:

   from syncr.app.flickr import FlickrSyncr
   f = FlickrSyncr(FLICKR_API_KEY, FLICKR_API_SECRET)
   
   # sync all my photos from the past week...
   f.syncRecentPhotos('username', days=7)
   
   # sync my favorite photos list
   f.syncPublicFavorites('username')

Changes from earlier versions

If you're using a version of django-syncr last updated prior to May 2011, here are the major changes I've made:

• Uses django-taggit instead of django-tagger This is a major change if you're using any of the apps that use tagging. Database structure is slightly different and although there are South migrations you may need to do some manual work.
• Made Twitter work with OAuth. It now requires you to register a new Twitter app and provide the app's consumer key, consumer secret, access token and access token secret.
• Differentiate Twitter users by Twitter ID, not username. Because usernames can change. This means that previously-synced users may be duplicated if you start using the new code.
• Add in-reply-to and lat/long coordinates for tweets. The former means we also sync any earlier tweets in a conversation, which results in more requests.
• Record Twitter users' privacy setting. Each user now has a 'protected' boolean property, as do their tweets.
• Add South migrations.

More documentation may come at a later date.