Skip to content

Home

camelpunch edited this page Nov 13, 2011 · 14 revisions
Clone this wiki locally

Ungulate

Ungulate is a gem for handling file uploads using various Amazon Web Services. These are the main aims:

  • Reduction of time taken to upload a file to a given web application, especially when post-upload processing is required
  • Offloading work to Amazon services where possible (hence the work on direct-to-S3 uploading)
  • Easy integration with Rails and other Ruby web frameworks
  • Separation of concerns, with no necessity to use all of the gem’s functionality

Examples

Server

Why not use Paperclip / Carrierwave?

Here’s the standard problem: you have several different thumbnails to be generated, and you don’t want your customer to wait around while your servers manipulate and store each thumbnail. The customer should only have to wait for their image to be uploaded.

This problem has also been solved in a different way with delayed_paperclip etc. By using a separate queue service, Ungulate allows for jobs to be run on separate machines to the application.

Why not use an on-the-fly thumbnailing service?

Firstly, I want to store all my images on S3. I don’t want to rely on temporary EC2 storage in order to serve images that have already been processed.

In the worst-case scenario where all cached thumbnails are lost, I don’t want to have to re-process everything. Also, I don’t want to centralise an image processing service. Ungulate lets you run multiple queue runners easily on different boxes, and your client might like to share jobs out across these processes.

To simulate the benefits of having an on-fetch image processing service, I take advantage of the feature that PUTs to a notification URL when a job is complete: after the upload has been sent to Amazon, I poll a JSON representation of a record until a flag has been set by the action at the notification URL.

NB: none of this precludes you from writing an on-the-fly thumbnailing service using Ungulate! I’d appreciate hearing about it if you do.

What’s implemented

Ungulate currently provides the following:

  • A server that polls an SQS queue for image processing jobs
  • Thumbnailing and watermarking using RMagick
  • A library for handling direct-to-S3 file uploads
Something went wrong with that request. Please try again.