Uploadr.py

Uploadr.py is a simple Python script for uploading your photos to Flickr. It also arranges them into sets and collections that correspond to the directory structure of your photos directory.

The script is superior to other Flickr uploaders in several ways:

1. It preserves the layout of your photos directory by organizing the uploads into sets and collections, instead of just putting everything into one giant photo stream.
2. It can be safely interrupted and restarted and will avoid making duplicate uploads.
3. It can be automated to regularly back up your photos directory.

Instructions

The script in its current version is only tested on Mac OS X so all instructions are targeted to Mac users.

Step 0. Prerequisites

1. Python

Mac OS X 10.8 comes with Python preinstalled. To verfiy that, open a terminal and type python -V. If you don't have it, get it from here. You need Python 2.7 (Python 3 won't work).

2. Git

To see if you have Git installed, open a terminal and type git --version. If you don’t have it, you can get the latest version from here.

Step 1. Download the script

Go to your favorite hacking directory and clone the repo:

git clone https://github.com/tomov/uploadr.py.git

Then go to the script directory:

cd uploadr.py/uploadr

Step 2. Run the script

Make sure to have your Flickr API key and secret (if you don't, you can get them here). Then run the script:

python uploadr.py --dir=[photos directory] --api-key=[your api key] --api-secret=[your api secret]

Here is what this would look like for an example directory, API key and secret:

python uploadr.py --dir="/Users/tomov90/Downloads/My Photos/" --api-key=00954e229265b619362cb462da234100 --api-secret=4cf2baa933309b8e

You will be forwarded to a Flickr confirmation page in your browser. Click OK, I'LL AUTHORIZE IT at the bottom, go back to the terminal and type Y. You will get another prompt asking you if you are sure you want to continue. Type Y again and let python do the rest of the work!

Step 3. Check if everything is fine

Even as the script is running, you can watch photos being added if you go to your Flickr account photo stream. Once the script has finished, go there to make sure everything has been successfully uploaded. Check the photo count at the top of the photo stream and make sure it looks right. I also recommend checking the sets and collections in the organizer to make sure the photos are neatly organized like they were in your photos directory.

Step 4. Re-running the script

To back up the same folder to the same Flickr account, simply run:

python uploadr.py --dir=[photos directory] --no-prompt

And the upload should start immediately. You won't have to re-enter your API key and secret since the app saves them in your photos directory. The app also saves a history of all previously uploaded photos and unless you move stuff around or rename your files or directories, it will avoid uploading duplicate photos or creating duplicate sets and collections.

Step 5. Automate the script

The best part about a command-line script like this is that you can easily automate it. You can do this by creating a cron job through the command line:

crontab -e

This will open the crontab file. Simply add the line:

0  *  *  *  *  /full/path/to/uploadr.py/uploadr/uploadr.py --dir=[photos directory] --no-prompt > /dev/null 2>&1

Which will run the script in the background every hour. For example, for me the line would be:

0  *  *  *  * /Users/tomov90/Dev/uploadr.py/uploadr/uploadr.py --dir="/Users/tomov90/Downloads/My Photos/" --no-prompt > /dev/null 2>&1

Alternatively, you can use the Mac Automator by following this or this tutorial.

Advanced

The script works with relative paths, so if you move your photos directory to a different location or even if you upload it from a different computer, it should still work. Those relative paths are stored in the descriptions of the photos, sets, and collections in your Flickr account, so please avoid changing them. The script also never deletes uploaded photos.

Files

You will notice that the script creates a bunch of files with the prefix uploadr.* in your photos directory. Some of them will be hidden, namely:

.uploadr.flickrToken
.uploadr.apiKey
.uploadr.apiSecret

Those contain your Flickr account access information so you don't have to enter it every time. However, this also means that anyone who has access to those files can access your precious photos, so make sure to avoid sending them to random people. If you ever delete them, you will have to pass the API key and secret as command-line parameters as discussed in Step 2.

In addition, the script saves a history of all uploaded photos, sets, and collections in these files:

uploadr.uploaded_images.db
uploadr.created_sets.db
uploadr.created_collections.db

This helps the script avoid duplicate uploads. If you delete them, the script will still avoid duplicate uploads by first fetching a list of all images, sets, and collections from the Flickr account. In fact, if for some reason you upload photos to the same account from different directories, it might make sense to delete those files and let the script "refresh" them with the latest data in the Flickr account.

Finally, the script creates a log of failed uploads and ignored files:

uploadr.failed_uploads.log
uploadr.ignored_files.log

This is for debugging purposes and to make sure none of your important files were ignored or failed to upload for some reason. Feel free to remove them.

Future work

The script is far from perfect and there is plenty of room for improvement. Feel free to fork, change, improve, and distribute as you see fit! Some suggestions for improvements:

1. Windows and Linux compatibility

It would be awesome if someone tried to see if this works on other platforms. It will surely need some help to get it going under Windows since I've hardcoded a bunch of forward slashes here and there (sorry about that).

2. --dry-run option

It would be great to have the option to run the script without actually uploading or changing anything, just to see what will happen (which files will be uploaded, how many of them, etc)

3. Pause/resume script

Currently you can interrupt the script with Cmd+C and restart it. It would be nice if you could only pause it.

4. Subcollections

Currently the Flickr collections API is unofficial and I could not figure out how to create a collection within a collection. So if you have lots of nested directories, e.g. /path/to/some/album/, the script will create collections /path, /path/to, and /path/to/some, and a set album nested inside the last collection. Ideally, once Flickr releases their collections API, we would like instead to create a collection path and inside it a collection to and inside it a collection some and finally inside it a set album.

5. Intelligent deduplication

This is kind of advanced, but it would be awesome if the script can detect if you renamed a file or a directory and instead of reuploading the whole thing over again, it would simply rename the corresponding images, sets or collections in the Flickr account. This might require some sort of hashing but would make the script a lot more robust.

License

Uploadr.py consists of code by Cameron Mallory, Martin Kleppmann, Aaron Swartz and others. See COPYRIGHT for details. Latest modifications (integration with the sets and collections API) by Momchil Tomov.