Kong Heroku app

Deploy Kong 0.11 Community Edition clusters to Heroku Common Runtime and Private Spaces using the Kong buildpack.

🔬 This is a community proof-of-concept: MIT license

• Purpose
• Usage
  • Deploy
  • Connect local to Heroku app
  • Admin console
  • Proxy & protect the Admin API
• Customization
  • Configuration
  • Kong plugins & additional Lua modules
• Demos
  • API Rate Limiting
  • Custom plugin: hello-world-header
  • Custom plugin: API translation, JSON→XML
• Development Notes
  • Learning the Language of Kong
  • Programming with Lua
  • Local Development
    • Requirements
    • Clone & connect
    • Setup
    • Running
    • Testing

Purpose

Kong is an extensible HTTP gateway/proxy application based on OpenResty, a web app framework built on the embedded Lua language capabilities of the Nginx web server.

With Heroku, Kong may be used for a variety of purposes. A few examples:

• implemenent unified authentication for a suite of apps
• enforce rate-limiting & request-size limits
• create a single management point for domains & hostnames of public APIs.

👓 See: main Kong site for more about this powerful API gateway.

Usage

Deploy

Use the deploy button to create a Kong app in your Heroku account:

Connect local to Heroku app

To use Admin console on a freshly-deployed app, clone and connect this repo (or your own fork) to the Heroku app:

git clone https://github.com/heroku/heroku-kong.git
cd heroku-kong

# Use the name of the Heroku app:
heroku git:remote --app $APP_NAME
heroku info

Admin console

Use Kong CLI and the Admin API in a one-off dyno:

heroku run bash

# Run Kong in the background of the one-off dyno:
~ $ bin/background-start

# Then, use `curl` to issue Admin API commands
# and `jq` to format the output:
~ $ curl http://$KONG_ADMIN_LISTEN | jq .

# Example CLI commands:
# (note some commands require the config file and others the prefix)
~ $ kong migrations list -c $KONG_CONF
~ $ kong health -p /app/.heroku

Proxy & protect the Admin API

Kong's Admin API has no built-in authentication. Its exposure must be limited to a restricted, private network. For Kong on Heroku, the Admin API listens privately on localhost:8001.

To make Kong Admin accessible from other locations, let's setup Kong itself to proxy its Admin API with key authentication, HTTPS-enforcement, and request rate & size limiting.

From the admin console:

# Create the authenticated `/kong-admin` API, targeting the localhost port:
curl http://localhost:8001/apis -i -X POST \
  --data name=kong-admin \
  --data uris=/kong-admin \
  --data upstream_url=http://localhost:8001 \
  --data https_only=true \
  --data http_if_terminated=true
curl http://localhost:8001/apis/kong-admin/plugins/ -i -X POST \
  --data 'name=request-size-limiting' \
  --data "config.allowed_payload_size=8"
curl http://localhost:8001/apis/kong-admin/plugins/ -i -X POST \
  --data 'name=rate-limiting' \
  --data "config.second=5"
curl http://localhost:8001/apis/kong-admin/plugins/ -i -X POST \
  --data 'name=key-auth' \
  --data "config.hide_credentials=true"
curl http://localhost:8001/apis/kong-admin/plugins/ -i -X POST \
  --data 'name=acl' \
  --data "config.whitelist=kong-admin"

# Create a consumer with username and authentication credentials:
curl http://localhost:8001/consumers/ -i -X POST \
  --data 'username=8th-wonder'
curl http://localhost:8001/consumers/8th-wonder/acls -i -X POST \
  --data 'group=kong-admin'
curl http://localhost:8001/consumers/8th-wonder/key-auth -i -X POST -d ''
# …this response contains the `"key"`, use it for `$ADMIN_KEY` below.

Now, access Kong's Admin API via the protected, public-facing proxy:

✏️ Replace variables such as $APP_NAME with values for your unique deployment.

# Set the request header:
curl -H 'apikey: $ADMIN_KEY' https://$APP_NAME.herokuapp.com/kong-admin/status
# or use query params:
curl https://$APP_NAME.herokuapp.com/kong-admin/status?apikey=$ADMIN_KEY

Customization

Kong may be customized through configuration and plugins.

Configuration

Kong is automatically configured at runtime with a .profile.d script:

• renders the config/kong.conf file based on:
  • the customizable config/kong.conf.etlua template
  • values of config vars, as defined in the buildpack
• exports environment variables
  • see: .profile.d/kong-env in a running dyno

All file-based config may be overridden by setting KONG_-prefixed config vars, e.g. heroku config:set KONG_LOG_LEVEL=debug

👓 See: Kong 0.11 Configuration Reference

Kong plugins & additional Lua modules

👓 See: buildpack usage

Demos

Usage examples and sample plugins are includes with this Heroku Kong app.

Demo: API Rate Limiting

Request this Bay Lights API more than five times in a minute, and you'll get HTTP Status 429: API rate limit exceeded, along with X-Ratelimit-Limit-Minute & X-Ratelimit-Remaining-Minute headers to help the API consumers regulate their usage.

Try it in your shell terminal:

curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights
# HTTP/1.1 200 OK
curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights
# HTTP/1.1 200 OK
curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights
# HTTP/1.1 200 OK
curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights
# HTTP/1.1 200 OK
curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights
# HTTP/1.1 200 OK
curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights
# HTTP/1.1 429

Here's the whole configuration for this API rate limiter:

curl http://localhost:8001/apis/ -i -X POST \
  --data 'name=bay-lights' \
  --data 'uris=/bay-lights' \
  --data 'upstream_url=https://bay-lights-api-production.herokuapp.com/'
curl http://localhost:8001/apis/bay-lights/plugins/ -i -X POST \
  --data 'name=request-size-limiting' \
  --data "config.allowed_payload_size=8"
curl http://localhost:8001/apis/bay-lights/plugins/ -i -X POST \
  --data 'name=rate-limiting' \
  --data "config.minute=5"

Demo: custom plugin: hello-world-header

Custom plugins allow you to observe and transform HTTP traffic using lightweight, high-performance Lua code in Nginx request processing contexts. Building on the previous example, let's add a simple plugin to Kong.

hello-world-header will add an HTTP response header X-Hello-World showing the date and a message from an environment variable.

Activate this plugin for the API:

curl http://localhost:8001/apis/bay-lights/plugins/ -i -X POST \
  --data 'name=hello-world-header'

Then, set a message through the Heroku config var:

heroku config:set HELLO_WORLD_MESSAGE='🌈🙈'
# …the app will restart.

Now, when fetching an API response, notice the X-Hello-World header:

curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights
# ↩︎
# HTTP/1.1 200 OK
# Connection: keep-alive
# Content-Type: application/json;charset=utf-8
# Content-Length: 9204
# X-Ratelimit-Limit-Minute: 5
# X-Ratelimit-Remaining-Minute: 4
# Server: Cowboy
# Date: Mon, 28 Aug 2017 23:14:47 GMT
# Strict-Transport-Security: max-age=31536000
# X-Content-Type-Options: nosniff
# Vary: Accept-Encoding
# Request-Id: 6c815aae-a5e9-496f-b731-dc72bbe2b63e
# Via: kong/0.11.0, 1.1 vegur
# X-Hello-World: Today is 2017-08-28. 🌈🙈  <--- The injected header
# X-Kong-Upstream-Latency: 49
# X-Kong-Proxy-Latency: 161

Demo: custom plugin: API translation, JSON→XML

JSON/REST has taken over as the internet API lingua franca, shedding the complexity of XML/SOAP. The National Digital Forecast Database [NDFD] is a legacy XML/SOAP service.

This app includes a sample, custom plugin ndfd-xml-as-json. This plugin exposes a JSON API that returns the maximum temperatures forecast for a location from the NDFD SOAP service. Using the single-resource concept of REST, the many variations of a SOAP or other legacy interfaces may be broken out into elegant, individual JSON APIs.

Try it in your shell terminal:

curl https://kong-proxy-public.herokuapp.com/ndfd-max-temps \
  --data '{"latitude":37.733795,"longitude":-122.446747}'
# Response contains max temperatures forecast for San Francisco, CA

curl https://kong-proxy-public.herokuapp.com/ndfd-max-temps \
 --data '{"latitude":27.964157,"longitude":-82.452606}'
# Response contains max temperatures forecast for Tampa, FL

curl https://kong-proxy-public.herokuapp.com/ndfd-max-temps \
  --data '{"latitude":41.696629,"longitude":-71.149994}'
# Response contains max temperatures forecast for Fall River, MA

Much more elegant than the legacy API. See the sample request body:

curl --data @spec/data/ndfd-request.xml -H 'Content-Type:text/xml' -X POST https://graphical.weather.gov/xml/SOAP_server/ndfdXMLserver.php
# Response contains wrapped XML data. Enjoy decoding that.

This technique may be used to create a suite of cohesive JSON APIs out of various legacy APIs.

Here's the configuration for this API translator:

curl http://localhost:8001/apis -i -X POST \
  --data 'name=ndfd-max-temps' \
  --data 'upstream_url=https://graphical.weather.gov/xml/SOAP_server/ndfdXMLserver.php' \
  --data 'uris=/ndfd-max-temps'
curl http://localhost:8001/apis/ndfd-max-temps/plugins/ -i -X POST \
  --data 'name=request-size-limiting' \
  --data "config.allowed_payload_size=8"
curl http://localhost:8001/apis/ndfd-max-temps/plugins/ -i -X POST \
  --data 'name=rate-limiting' \
  --data 'config.minute=5'
curl http://localhost:8001/apis/ndfd-max-temps/plugins/ -i -X POST \
  --data 'name=ndfd-xml-as-json'

👓 See the implementation of the custom plugin's Lua source code, unit tests, and integration tests.

Dev notes

Learning the language of Kong

• Definitely an openresty guide
• An Introduction To OpenResty - Part 1, 2, & 3
• Nginx API for Lua, ngx reference, for use in Kong plugins
  • Nginx variables, accessible through ngx.var

Programming with Lua

• Lua 5.1, Note: Kong is not compatible with the newest Lua version
• Classic Objects, the basis of Kong's plugins
• Moses, functional programming
• Lubyk, realtime programming (performance- & game-oriented)
• resty-http, Nginx-Lua co-routine based HTTP client
• Serpent, inspect values
• Busted, testing framework

Local development

To work with Kong locally on macOS X.

Requirements

• kong for macOS via Homebrew
• git
• Heroku
  • command-line tools (CLI)
  • a free account

Clone & connect

If you haven't already, clone and connect your own fork of this repo to the Heroku app:

# Replace the main repo with your own fork:
git clone https://github.com/heroku/heroku-kong.git
cd heroku-kong

# Use the name of the Heroku app:
heroku git:remote --app $APP_NAME
heroku info

Setup

1. Ensure requirements are met

2. Create the Postgres user & databases:

   createuser --pwprompt kong
   # set the password "kong"
   
   createdb --owner=kong kong_dev
   createdb --owner=kong kong_tests

3. Execute ./bin/setup

Running

bin/start

• Logs in /usr/local/var/kong/logs/
• Prefix is /usr/local/var/kong for commands like:
  • kong health -p /usr/local/var/kong
  • kong stop -p /usr/local/var/kong

Testing

Any test-specific Lua rocks should be specified in Rockfile_test file, so that they are not installed when the app is deployed.

Add tests in spec/:

• Uses the Busted testing framework
• Kong plugin testing guide
• buildpack requirements for testing

bin/test