HTTP image processing server based on imgflo
CoffeeScript JavaScript HTML Makefile
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bin
components
doc
examples
graphs
migrations
runtime @ b214c0a
spec
src
.buildpacks
.gitignore
.gitmodules
.travis.yml
.vendor_urls
Aptfile
CHANGES.md
Makefile
README.md
app.json
gitmodules
index.js
knexfile.coffee
newrelic.js
package.json
update-version-info.js
webpack.config.js
worker.js
worker.webpack.js

README.md

Build Status

imgflo-server

imgflo-server is an image-processing server with HTTP built using the imgflo dataflow runtime.

Deploy

Changelog

See ./CHANGES.md

License

MIT

Note: GEGL itself is under LGPLv3.

API

imgflo-server provides a HTTP API for processing images. The entire processing request is described using an URL, and the processing can be triggered by a HTTP GET. This means no special integration is needed in order to use an image processed by imgflo-server in an application.

Process image and get results

Request

GET /graphs/mygraph?input=http://example.com/input-image.png&attr1=value1&....

Response

HTTP 301
Location: https://s3.aws.com/bucket/er132123sder12.png

If the image has been processed before, it will come straight from cache (fast).

Process image without waiting for result

Request

POST /graphs/mygraph?input=http://example.com/input-image.png&attr1=value1&....

Response

HTTP 301
Location: https://s3.aws.com/bucket/er132123sder12.png

Since the image is not processed by the time the response arrives, accessing the Location immediately will likely fail (with a 404 or 403). If the image processing failed, it may fail forever. Use a GET with the same imgflo URL to get the error.

Get available image processing graphs

The inports describe which parameters are available for each graph.

Request

GET /graphs

Response

HTTP 200 application/json
{
  "graphs": {
    "customgrey": {
      "inports": {
        "input": {
        },
        "height": {
          "metadata": {
            "description": "Requested output height",
            "type": "int",
            "maximum": 2000,
            "minimum": 0
          }
        },
        "width": {
          "metadata": {
            "description": "Requested output width",
            "type": "int",
            "maximum": 2000,
            "minimum": 0
          }
        }
      },
      "outports": {
        "output": {
        }
      }
    }
}

Specifying output format

By specifying an extension on the graph, you can

/graph/gradientmap.png?width=300....

If the extension is omitted, a default image format is inferred. This is currently the same image format as the input.

Authentication

To prevent people from using your deployment to process their images, there exists a version of the processing URL scheme which uses encryption to secure it. Note that this only prevents people from forming new image processing requests. It does not prevent them from requesting an already processed image.

Each client has its own KEY and SECRET.

Given a request with a set of parameters (like gradientmap.png&width=300&input=...), a token is constructed by taking the md5sum of the SECRET concatenated with the parameters:

token=md5(parameters+secret)
GET /graph/$KEY/$token/$params

Upon getting such a request, the server will use the KEY to find the matching SECRET, do the same construction of token, and validate that there is a match. If not, request will fail with a 403 Permission Denied.

Adding new tokens for a client/application

Use the helper script

./bin/imgflo-server-application-add

It will generate client key and secret, insert it to the database, and output the details to stdout.

Changing quotas

If processing_quota field is 0, the client cannot process new images, but can still request existing cached/processed images.

Currently processing quota is not otherwise enforced.

To disallow clients from also accessing cache/processed images, set enabled to false/0.

There is currently no tools for manipulating quota, have to use SQL on the applications table.

Errors

  • 504: Unable to fetch the specified input URL
  • ...

API client libraries

These libraries make it it easier to use imgflo-server, by providing a programmatic API which handles the details of constructing authenticated URLs.

JavaScript / node.js / browser


  1. Forming valid imgflo urls: imgflo-url
  2. Creating responsive images using media-query: rig

Java / Android

imgflo-url-java

Swift / iOSs

ImgFlo.swift

Testing UI

imgflo-server ships with a simple testing UI served at /. It can be used to see the available graphs, and make test requests and see the results.

TODO: add picture

Use for debugging a request

Given a regular imgflo URL, you can add &debug=1 at the end to open it in the UI. It will extract the parameters, including removing the urlencoding on the input URL.

Creating new image processing graphs

imgflo-server can easily be extended with new image processing pipelines, either using a text-based DSL or Flowhub node-based visual IDE.

See Adding Graphs

System architecture

For an in-depth look at how the system is implemented, see system architecture

Hosted public instance

Currently our deployed instance is only for The Grid. If you are interested in access to hosted version, send us an email: support@thegrid.io

Deploying to Heroku

Server

Register/log-in with Heroku, and create a new app. First one is free.

After creating the app, login at Heroku:

heroku login

Clone imgflo-server:

git clone https://github.com/imgflo/imgflo-server.git
cd imgflo-server

Add YOURAPP as remote:

heroku git:remote -a YOURAPP

Specify the multi buildpack with build-env support, either at app creation time, in Heroku webui or using

heroku config:set BUILDPACK_URL=https://github.com/mojodna/heroku-buildpack-multi.git#build-env

Configure some environment variables (hostname, port and local image cache):

heroku config:set HOSTNAME=YOURAPP.herokuapp.com
heroku config:set PORT=80

Deploy, will take ~1 minute

git push heroku master

You should now see the imgflo server running at http://YOURAPP.herokuapp.com

If everything is OK you should be able to see a generative image at http://YOURAPP.herokuapp.com/graph/delaunay_triangles?seed=foobar&height=800&width=600

Developing and running locally

Note: imgflo-server is only tested on GNU/Linux systems. imgflo (the runtime) has experimental support for OSX. However, all underlying dependencies (node.js, RabbitMQ, GEGL etc) are commonly used also on other platforms, like Windows.

Root is not needed for any of the build.

Pre-requisites

imgflo requires git master of GEGL and BABL, as well as a custom version of libsoup. It is recommended to let make setup this for you, but you can use existing checkouts by customizing PREFIX. You only need to install the dependencies once, or when they have changed.

git submodule update --init --recursive
make dependencies

Install node.js dependencies

npm install

Build

Now you can build & install imgflo-server itself

make install

To verify that things are working, run the test suite

make check

Running server

node index.js

You should see your server running at http://localhost:8080