Skip to content

ngerakines/s3imagehost

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

README

This project, s3images, is an open source erlang image store that uses
the Amazon S3 service for its backend. It aims to provide an image storage
service that acts in a way similar to MogileFS. This project was inspired
by Daanga's MogileFS.

There are a few assumptions made to make things simple.
1. You have an active amazon s3 account.
2. All uploaded files are publicly accessible
3. The bucket you are using is corresponds to a domain that can access the
   via a CNAME record or the like. In other words, if you have the domain
   example.com then you've already setup the media.example.com bucket and
   set the media CNAME record to point to media.example.com.s3.amazonaws.com

After a file has been uploaded there are a few rules used to determine if
the file should be stored on s3 and on the local mnesia store.
1. Is the file under 5 megs?
2. Is the file an imaage (png, jpeg or gif)?
3. Is the image less than 4096 pixels in height or width?

NOTE: This application requires the ImageMagick `convert` executable to
create thumbnails.

API

* GET [HOST]/ -- Returns introductory HTML.
* GET [HOST]/upload -- Returns HTML that is used to upload an image.
* POST [HOST]/upload -- Processes multi-part form data to upload image
  content to a temp directory, assert that it is valid, send it to the
  provided Amazon S3 bucket and create a mnesia record describing the image.
* GET [HOST]/a/recent -- Returns a list of 25 of the most recent image
  records created.
* GET [HOST]/a/object:[NAME] -- Returns a list of image records that point
  to the object [NAME].
* GET [HOST]/a/owner:[NAME] -- Returns a list of image records that point
  to the owner [NAME].
* GET [HOST]/i/[NAME] -- Requests image keyed [NAME]. If the image exists
  either the user will be redirected to it or a perlbal-friendly header will
  be passed back indicating an internal redirect needs to be processed.

LICENSE

Copyright (c) 2008 Nick Gerakines <nick@gerakines.net>

This project has been open sourced under the MIT license. Please see the
LICENSE file for more information.

CONFIGURATION

The following configuration variables must be set in the ebin/s3images.app:
1. ip -- The IP address that yaws attempts to bind to.
2. port -- The port that yaws will attempt to bind to on the above IP.
3. domain -- The hostname that yaws will recognize requests on.
4. s3bucket -- The name of the Amazon S3 bucket used throughout the app.
5. s3key -- The Amazon S3 key used to access the above bucket.
6. s3secret -- The secret token used in conjunction with the key.
7. reproxy -- Set to 'true' to send the 'X-REPROXY-URL' header.
8. create_sq -- Set to 'true' to create/upload 100x100 thumbnails
9. delegate -- When this variable is set then the module this variable
   points to is used for a number of delegated actions. Please refer to the
   s3images_demo module for more information and documentation.
10. template -- This variable points to a template module that is used in
    place of the default s3imagestmpl_ahandler.et template.

BUILDING

  $ make

START-UP

1. Update the ebin/s3images.app file and set the domain, amazon s3 access
   information and s3 bucket.
2. Build the project (`make`)
3. Start the erlang node and application with `make start`.

Optional: Connect with a control erlang node via `make start-ctl`.

ELIB

The elib directory contains the erltl and s3 libraries used by this
application. Both of those libraries have been explicitly separated because
they have been released under a separate license.

The rfc4627 library is also used to encode image records into JSON.

CAVEATS / NOTES / KNOWN ISSUES

 * Files are temporarily stored in the /tmp dir.
 * Uploaded files are renamed. The names are long and should probably be
   shortened.
 * The system has to wait for files to be uploaded before it can reject it
   for being too big.

CHANGES

2008-08-17 ngerakines
 - Sprinkled a few try/catch calls to catch app failures (yay fault-tolerance!)
 - Added `make start` and `make start-ctl` Makefile commands.

2008-07-14 ngerakines
 - Added a config option to point to a different template delegate. 

2008-07-12 ngerakines
 - Added a config option to direct the app to set the 'X-REPROXY-URL' header
   instead of sending a redirect.
 - Added a config option to use the ImageMagick `convert` utility to create
   and upload 100x100 square thumbnail images.
 - Added the /a/object:XYZ123 URI to return a list of images where XYZ123 is
   an object id.
 - Added the rfc4627 library to support the above API method.
 - Added the /a/recent url to return a list of the 25 most recently added
   images.
 - Added the 'delegate' config variable and demo module (s3images_demo.erl).
 - Added more (and hopefully better) documentation.
 - Updated the default HTML.
 - Added a list of recently uploaded image thumbnails and links to the root
   page.
 - Bumped the project version to 0.2

2008-07-09 ngerakines
 - Initial development
 - Package and release of 0.1

TODO / ROADMAP

 * Set the /i/ handler to transparently serve content without issuing a
   redirect. 
 * Improve the image record.
 * Add hooks to an auth handler to prevent anonymous uploads (s3 ain't cheap!).
 * Document the delegate module and explain what does what and why.
 * Provide alternative delegate modules.
 * Add more documentation.
 * Add a mochiweb handler, making the application not yaws-specific.

About

An erlang powered image store using amazon s3.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages