Simple App Engine Flickr Backup
After I read about that flickr accidentally deleting all of some dudes photos I decided that I wanted to backup all of my photos to another datastore. Something that I pay for and allows me to do some shit with them.
I decided that I would push them into the Google App Engine blobstore.
This is the result of about 4 hours of hacking. I used Straup's FlickrApp container to jumpstart the app. It is a hacked up version of webapp framework to allow you to use flickr as your SSO. It is pretty awesome - but probably a bit of overload for this app.
The app was built to be multitenant - but i haven't tested it extensively. YMMV.
When I backed up my photos, I was pushing about 0.24 QPS over a 24 hour period. This will greatly increase if you allow multiple users to back up their photos. You can tweak how fast things happen in the queue.yaml
Uses tasks to handle all the magic.
There are some limitations to this that i need to get past, but for the most part it backed up 80% of my photos rather quickly.
- Can't seem to back up photos over 1mb (limit of GAE request size) (i might be able to do it client side) (any blobstore haxors?)
- It can get expensive to run (~20 bucks a week if you are constantly backing up)
- It is not very efficient
- It doesn't backup sets, comments, etc
- It only backs up photos
- It currently runs forever (this will be fixed shortly)
I am sure there are more things to fix.
- copy the config.py.example to config.py
- add your api credentials to config.py
- edit the app.yaml file to point to the correct app engine app id
- visit your app
- start the process