Optimises and resizes images
Clone or download
gvonkoss and rowanmanning add issue template (#345)
* add issue template

* add CODEOWNERS
Latest commit 18c57f8 Dec 7, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci Add a "fill" option to the "fit" property (#341) Nov 14, 2018
.github add issue template (#345) Dec 7, 2018
data Add layout (#337) Oct 5, 2018
incidents AWS S3 incident report (#226) Mar 6, 2017
lib Fix URLs (#342) Nov 14, 2018
public Add layout (#337) Oct 5, 2018
test Fix URLs (#342) Nov 14, 2018
views
.eslintignore Initial commit May 18, 2016
.eslintrc.js Add surrogate keys for content type and scheme type (#192) Jan 12, 2017
.gitignore Update origami service makefile and add whitesource config (#310) Nov 13, 2017
Makefile Add auto-versioning (#319) Dec 7, 2017
Procfile Update Origami Service and use Origami Service Makefile (#279) Jul 13, 2017
README.md Use correct service name in readme (#322) Jan 23, 2018
about.json Update Origami Service and use Origami Service Makefile (#279) Jul 13, 2017
index.js Cache DNS for 1 minute (#246) Mar 15, 2017
origami.json
package-lock.json Add layout (#337) Oct 5, 2018
package.json Migrate to Origami Service v4.0.0 (#336) Oct 1, 2018
whitesource.config.json Update origami service makefile and add whitesource config (#310) Nov 13, 2017

README.md

Origami Image Service

Optimises and resize images. See the production service for API information.

Build status MIT licensed

Table Of Contents

Requirements

Running Origami Image Service requires Node.js 6.x and npm.

Running Locally

Before we can run the application, we'll need to install dependencies:

npm install

Run the application in development mode with

make run-dev

Now you can access the app over HTTP on port 8080: http://localhost:8080/

Configuration

We configure Origami Image Service using environment variables. In development, configurations are set in a .env file. In production, these are set through Heroku config. Further documentation on the available options can be found in the Origami Service documentation.

Required everywhere

  • CLOUDINARY_ACCOUNT_NAME: The name of the Cloudinary account to use in image transforms.
  • CLOUDINARY_API_KEY: The Cloudinary API key corresponding to CLOUDINARY_ACCOUNT_NAME.
  • CLOUDINARY_API_SECRET: The Cloudinary API secret corresponding to CLOUDINARY_ACCOUNT_NAME.
  • CUSTOM_SCHEME_STORE: The location of the images used in custom schemes. This should be set to the base path under which images live.
  • CUSTOM_SCHEME_CACHE_BUST: A key used to manually cache-bust custom scheme images.
  • HOSTNAME: The hostname to use for tinting SVGs. This defaults to the hostname given in the request. See the trouble-shooting guide for more information.
  • NODE_ENV: The environment to run the application in. One of production, development (default), or test (for use in automated tests).
  • PORT: The port to run the application on.

Required in Heroku

  • CMDB_API_KEY: The API key to use when performing CMDB operations
  • FASTLY_PURGE_API_KEY: A Fastly API key which is used to purge URLs (when somebody POSTs to the /purge endpoint)
  • GRAPHITE_API_KEY: The FT's internal Graphite API key
  • PURGE_API_KEY: The API key to require when somebody POSTs to the /purge endpoint. This should be a non-memorable string, for example a UUID
  • REGION: The region the application is running in. One of QA, EU, or US
  • RELEASE_LOG_API_KEY: The change request API key to use when creating and closing release logs
  • RELEASE_LOG_ENVIRONMENT: The Salesforce environment to include in release logs. One of Test or Production
  • SENTRY_DSN: The Sentry URL to send error information to

TODO: The options below are required at the moment, but are duplicates of other options above. This will be addressed once all services are using Origami Makefile.

  • FASTLY_API_KEY: The Fastly API key to use when purging assets. If not set, purge endpoints are not registered. This should be the same value as FASTLY_PURGE_API_KEY
  • FASTLY_SERVICE_ID: The Fastly service to purge assets from
  • API_KEY: The API key to use when purging assets. If not set, endpoints which require an API key are not registered. This should be the same value as PURGE_API_KEY

Required in Heroku (QA only)

  • WHITESOURCE_API_KEY: The API key to use when testing dependencies with Whitesource

Required locally

  • GRAFANA_API_KEY: The API key to use when using Grafana push/pull

Headers

The service can also be configured by sending HTTP headers, these would normally be set in your CDN config:

  • FT-Origami-Service-Base-Path: The base path for the service, this gets prepended to all paths in the HTML and ensures that redirects work when the CDN rewrites URLs.
  • FT-Origami-Api-Key: The API key for the service, this is used when calling API endpoints which are restricted to FT Origami developers.

Testing

The tests are split into unit tests and integration tests. To run tests on your machine you'll need to install Node.js and run npm install. Then you can run the following commands:

make test              # run all the tests
make test-unit         # run the unit tests
make test-integration  # run the integration tests

You can run the unit tests with coverage reporting, which expects 90% coverage or more:

make test-unit-coverage verify-coverage

The code will also need to pass linting on CI, you can run the linter locally with:

make verify

We run the tests and linter on CI, you can view results on CircleCI. make test and make lint must pass before we merge a pull request.

Deployment

The production (EU/US) and QA applications run on Heroku. We deploy continuously to QA via CircleCI, you should never need to deploy to QA manually. We use a Heroku pipeline to promote QA deployments to production.

You can promote either through the Heroku interface, or by running the following command locally:

make promote

Monitoring

Trouble-Shooting

We've outlined some common issues that can occur in the running of the Image Service:

I need to purge an image

Please read the purging documentation on the website.

I need to purge all images, is this possible?

Please contact origami.support@ft.com - There is a way to purge all images, but this will incur a large cost.

What do I do if memory usage is high?

For now, restart the Heroku dynos:

heroku restart --app origami-image-service-eu
heroku restart --app origami-image-service-us

If this doesn't help, then a temporary measure could be to add more dynos to the production applications, or switch the existing ones to higher performance dynos.

What if I need to deploy manually?

If you really need to deploy manually, you should only do so to QA. Production deploys should always be a promotion from QA.

You'll need to provide an API key for change request logging. You can get this from the Origami LastPass folder in the note named Change Request API Keys. Now deploy to QA using the following:

make deploy

SVGs don't tint locally

SVG tinting is done in a way that makes it near-impossible to run locally. When an SVG image is requested and a tint query parameter is provided, then we rewrite the URL to go route back through the Image Service. It looks something like this:

  • User requests:
    http://imageservice/v2/images/raw/http://mysite/example.svg?tint=red
  • Image service rewrites to:
    http://imageservice/v2/images/raw/http://imageservice/images/svgtint/http://mysite/example.svg%3Fcolor=red
  • Cloudinary receives the image URL:
    http://imageservice/images/svgtint/http://mysite/example.svg?color=red

When you're running locally this won't work because Cloudinary cannot access your localhost. The flow would look like this:

  • User requests:
    http://localhost/v2/images/raw/http://mysite/example.svg?tint=red
  • Image service rewrites to:
    http://localhost/v2/images/raw/http://localhost/images/svgtint/http://mysite/example.svg%3Fcolor=red
  • Cloudinary receives the image URL:
    http://localhost/images/svgtint/http://mysite/example.svg?color=red

So Cloudinary responds with a 404. You can get around this by manually specifying a hostname in your configuration. You'll need to tell the service to rely on the QA instance for SVG tinting. Add the following to your .env file:

HOSTNAME=origami-image-service-qa.herokuapp.com

License

The Financial Times has published this software under the MIT license.