Version 0.11 (#8)

* migrate to kong version 0.11

* Export 12f config to KONG_* env variables; add Admin SSL config

* Export 12f config to KONG_* env variables; add Admin SSL config

* Switch to stable default secondary ports

* 📚 revise for Kong 0.11.0

* Look for Aptfile in buildpack & build dirs

* Customizable Nginx config and autorun migrations in release phase

* 📚 clarify status

.gitignore

@@ -1,2 +1,4 @@
 config/kong.yml
 config/cassandra.cert
+
+config/kong.conf.etlua

.profile.d/kong-12f.sh

@@ -1,14 +1,5 @@
 #!/usr/bin/env bash
 
-# Fail immediately on non-zero exit code.
-set -e
-# Fail immediately on non-zero exit code within a pipeline.
-set -o pipefail
-# Fail on undeclared variables.
-set -u
-# Debug, echo every command
-#set -x
-
 echo "Starting Kong 12-factor config"
 
 SRC_DIR=$(pwd)
@@ -25,7 +16,7 @@ else
 fi
 echo "Kong cluster private IP: $KONG_CLUSTER_PRIVATE_IP"
 
-luajit $SRC_DIR/config/kong-12f.lua $SRC_DIR/config/kong.yml.etlua $SRC_DIR
+luajit $SRC_DIR/config/kong-12f.lua $SRC_DIR/config/kong.conf.etlua $SRC_DIR
 
 # export the variables generated by `kong-12f.lua`
 source $SRC_DIR/.profile.d/kong-env

.profile.d/kong-env

@@ -1,3 +1,5 @@
 #!/usr/bin/env bash
 
+export KONG_CONF=/app/config/kong.conf
+
 # `kong-12f.lua` appends environment variables here

LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2017 Heroku
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.markdown

@@ -1,79 +1,91 @@
-[Heroku Buildpack](https://devcenter.heroku.com/articles/buildpacks) for [Kong](https://getkong.org)
+[Heroku Buildpack](https://devcenter.heroku.com/articles/buildpacks) for [Kong](https://getkong.org/about/)
 =========================
-Based on [Kong version 0.7.0](http://blog.mashape.com/kong-0-7-0-released/) patched for compatibility with Heroku.
+Deploy [Kong 0.11.0 Community Edition](http://blog.mashape.com/kong-ce-0-11-0-released/) as a Heroku app.
 
+🌈 This buildpack now deploys genuine Mashape Kong, [built from source on Github](bin/compile#L226); patches are no longer required for compatibility with Heroku.
 
-🚨 **This Heroku buildpack is no longer in development. It uses an outdated version of Kong.** It remains here on Github only to support existing deployments.
+🔬👩‍💻 This software is a community proof-of-concept: [MIT license](LICENSE)
 
 
 Usage
 -----
 
-### Beginner
+### Example
 
 Deploy the [heroku-kong app](https://github.com/heroku/heroku-kong) to get started.
 
-### Expert
+### Custom
 
-* `kong.yml`
-  * config template in `config/kong.yml.etlua`
-    * buildpack detects this file in the app
-    * [sample config file](config/kong.yml.etlua.sample)
-* Lua source in the app
-  * [Kong plugins](https://getkong.org/docs/0.7.x/plugin-development/):
+Create a new git repo and Heroku app:
+
+```bash
+mkdir my-kong-app
+cd my-kong-app
+git init
+heroku create $APP_NAME
+heroku buildpacks:set https://github.com/heroku/heroku-buildpack-kong
+heroku addons:create heroku-postgresql:hobby-dev
+```
+
+Create the file `config/kong.conf.etlua` based on the [sample config file](config/kong.conf.etlua.sample). This is a config template which generates `config/kong.conf` at runtime.
+
+```bash
+git add config/kong.conf.etlua
+git commit -m '🐒'
+git push heroku master
+```
+
+🚀 Check `heroku logs` and `heroku open` to verify Kong launches.
+
+#### Plugins & other Lua source
+
+  * [Kong plugins](https://getkong.org/plugins/)
+    * [Development guide](https://getkong.org/docs/0.11.x/plugin-development/)
     * `lib/kong/plugins/{NAME}`
-    * Add each Kong plugin name to the `plugins_available` list in `config/kong.yml.etlua` 
-    * See: [Plugin File Structure](https://getkong.org/docs/0.7.x/plugin-development/file-structure/)
+    * Add each Kong plugin name to the `custom_plugins` comma-separated list in `config/kong.conf.etlua` 
+    * See: [Plugin File Structure](https://getkong.org/docs/0.11.x/plugin-development/file-structure/)
   * Lua rocks
     * specify in the app's `.luarocks` file
     * each line is `{NAME} {VERSION}`
   * Other Lua source modules
     * `lib/{NAME}.lua` or
     * `lib/{NAME}/init.lua`
 
-### Environment variables
+#### Environment variables
 
   * `PORT` exposed on the app/dyno
     * set automatically by the Heroku dyno manager
-  * `KONG_CLUSTER_SECRET` symmetric encryption key
-    * generate value with command `serf keygen`; requires [Serf](https://www.serfdom.io/downloads.html)
   * `KONG_GIT_URL` git repo URL for Kong source
-    * example `https://github.com/mars/kong.git`
+    * example `https://github.com/Mashape/kong.git`
   * `KONG_GIT_COMMITISH` git branch/tag/commit for Kong source
-    * example `0.7.0-external-supervisor.1` or `master`
-  * Cassandra datastore
-    * Heroku-style config vars
-      * `CASSANDRA_URL`
-
-        ```
-cassandra://username:password@x.x.x.x:port/keyspace,cassandra://username:password@y.y.y.y:port/keyspace
-        ```  
-        
-          `username:password` must be the same for all instances.
-      * `CASSANDRA_TRUSTED_CERT` (SSL is disabled unless provided)
-    * [Instaclustr add-on](https://elements.heroku.com/addons/instaclustr) config vars
-      * `IC_CONTACT_POINTS`
-        ```
-x.x.x.x,y.y.y.y
-        ```
-      * `IC_PORT`
-      * `IC_USER`
-      * `IC_PASSWORD`
-      * `IC_CERTIFICATE` (SSL is disabled unless provided)
+    * example `master`
+  * `DATABASE_URL`
+    * set automatically by [Heroku Postgres add-on](https://elements.heroku.com/addons/heroku-postgresql)#### Using Environment Variables in Plugins
+
+To use env vars within your own code.
+
+  1. Whitelist the variable name for use within Nginx 
+    * In a custom Nginx config file add `env MY_VARIABLE;`
+    * See: [Nginx config](#user-content-nginx-config) (below)
+  2. Access the variable in Lua plugins
+    * Use `os.getenv('MY_VARIABLE')` to retrieve the value.
+
+
+#### Nginx config
+
+Kong is an Nginx-based application. To customize the underlying Nginx configuration, commit the file `config/nginx.template` with contents based on [the docs](https://getkong.org/docs/0.11.x/configuration/#custom-nginx-configuration) or [this included sample](config/nginx.template.sample).
 
 Background
 ----------
 The first time this buildpack builds an app, the build time will be significantly longer as Kong and its dependencies are compiled from source. **The compiled artifacts are cached to speed up subsequent builds.**
 
 We vendor the sources for Lua, LuaRocks, & OpenResty/Nginx and compile them with a writable `/app/.heroku` prefix. Attempts to bootstrap Kong on Heroku using existing [Lua](https://github.com/leafo/heroku-buildpack-lua) & [apt](https://github.com/heroku/heroku-buildpack-apt) buildpacks failed due to their compile-time prefixes of `/usr/local` which is read-only in a dyno.
 
-OpenSSL 1.0.2 (required by OpenResty) is also compiled from source, as the versions included in the Cedar 14 stack & apt packages for Ubuntu/Trusty are too old.
+OpenSSL 1.0.2 (required by OpenResty) is also compiled from source.
 
-Kong is installed from a forked source repo that includes [minimal changes for compatibility with the Heroku runtime](https://github.com/Mashape/kong/compare/release/0.7.0...mars:0.7.0-external-supervisor).
 
+### Modification
 
-Modification
-------------
 This buildpack caches its compilation artifacts from the sources in `vendor/`. Changes to the sources in `vendor/` will be detected and the cache ignored.
 
 If you need to trigger a full rebuild without changing the source, use the [Heroku Repo CLI plugin](https://github.com/heroku/heroku-repo) to purge the cache:

bin/app/heroku-buildpack-kong-release

@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+set -eu
+
+KONG_CONF="${KONG_CONF:-config/kong.conf}"
+
+kong migrations up -c $KONG_CONF $@

bin/app/heroku-buildpack-kong-web

@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+set -eu
+
+# Customized Nginx setup
+# https://getkong.org/docs/0.11.x/configuration/#customized-nginx-setup
+
+APP_PREFIX="${APP_PREFIX:-/app/.heroku}"
+KONG_CONF="${KONG_CONF:-config/kong.conf}"
+NGINX_TEMPLATE="config/nginx.template"
+NGINX_TEMPLATE_PARAM=""
+
+if [ -e "$NGINX_TEMPLATE" ]
+  then
+  NGINX_TEMPLATE_PARAM="--nginx-conf $NGINX_TEMPLATE"
+fi
+
+kong prepare -p $APP_PREFIX -c $KONG_CONF $NGINX_TEMPLATE_PARAM
+eval "nginx -p $APP_PREFIX -c $APP_PREFIX/nginx.conf $@"

bin/compile

@@ -31,26 +31,25 @@ if [ -f "${ENV_DIR}/KONG_GIT_URL" ]
 then
   KONG_GIT_URL=`cat ${ENV_DIR}/KONG_GIT_URL`
 else
-  KONG_GIT_URL="https://github.com/mars/kong.git"
+  KONG_GIT_URL="https://github.com/Mashape/kong.git"
 fi
 # commit or tag to checkout
 if [ -f "${ENV_DIR}/KONG_GIT_COMMITISH" ]
 then
   KONG_GIT_COMMITISH=`cat ${ENV_DIR}/KONG_GIT_COMMITISH`
 else
-  KONG_GIT_COMMITISH="0.7.0-external-supervisor.1"
+  KONG_GIT_COMMITISH="0.11.0"
 fi
 
 KONG_SOURCE_DIR="${BP_CACHE_DIR}/kong-source"
 
+
 # Set dependency versions.
 # These correspond to the archives in `vendor/`.
 # If upgrading any those archives, then update the corresponding version here.
-LUAJIT_VERSION=2.1.0-beta1
-LUAROCKS_VERSION=2.3.0
-SERF_VERSION=0.7.0
-OPENSSL_VERSION=1.0.2f
-OPENRESTY_VERSION=1.9.7.2
+LUAROCKS_VERSION=2.4.2
+OPENSSL_VERSION=1.0.2l
+OPENRESTY_VERSION=1.11.2.4
 
 function error() {
   echo " !     $*" >&2
@@ -80,7 +79,7 @@ APT_OPTIONS="-o debug::nolocking=true -o dir::cache=$APT_CACHE_DIR -o dir::state
 topic "Updating apt caches"
 apt-get $APT_OPTIONS update | indent
 
-for PACKAGE in $(cat apt-packages); do
+for PACKAGE in $(cat $BP_DIR/Aptfile $BUILD_DIR/Aptfile); do
   if [[ $PACKAGE == *deb ]]; then
     PACKAGE_NAME=$(basename $PACKAGE .deb)
     PACKAGE_FILE=$APT_CACHE_DIR/archives/$PACKAGE_NAME.deb
@@ -103,7 +102,7 @@ done
 topic "Writing profile script"
 mkdir -p $BUILD_DIR/.profile.d
 cat <<EOF >$BUILD_DIR/.profile.d/000_apt.sh
-export PATH="\$HOME/.heroku/bin:\$HOME/.heroku/nginx/sbin:\$HOME/.apt/usr/local/bin:\$HOME/.apt/usr/bin:\$HOME/.apt/usr/sbin:\$PATH"
+export PATH="\$HOME/.heroku/nginx/sbin:\$HOME/.heroku/luajit/bin:\$HOME/.heroku/bin:\$HOME/.apt/usr/local/bin:\$HOME/.apt/usr/bin:\$HOME/.apt/usr/sbin:\$PATH"
 export LD_LIBRARY_PATH="\$HOME/.heroku/lib:\$HOME/.apt/usr/lib/x86_64-linux-gnu:\$HOME/.apt/usr/local/lib:\$HOME/.apt/usr/lib:/usr/lib/x86_64-linux-gnu:/lib/x86_64-linux-gnu:/usr/lib:\$LD_LIBRARY_PATH"
 export LIBRARY_PATH="\$HOME/.heroku/lib:\$HOME/.apt/usr/lib/x86_64-linux-gnu:\$HOME/.apt/usr/local/lib:\$HOME/.apt/usr/lib:/usr/lib/x86_64-linux-gnu:/lib/x86_64-linux-gnu:/usr/lib:\$LIBRARY_PATH"
 export INCLUDE_PATH="\$HOME/.heroku/include:\$HOME/.apt/usr/local/include:\$HOME/.apt/usr/include:/usr/include/x86_64-linux-gnu:/usr/include:\$INCLUDE_PATH"
@@ -114,7 +113,7 @@ export LUA_PATH="\$HOME/lib/?.lua;\$HOME/lib/?/init.lua;\$HOME/.luarocks/share/l
 export LUA_CPATH="\$HOME/lib/?.so;\$HOME/.luarocks/lib/lua/5.1/?.so;\$HOME/.heroku/lib/lua/5.1/?.so;./?.so"
 EOF
 
-export PATH="$APP_PREFIX/bin:$APP_PREFIX/nginx/sbin:$BUILD_DIR/.apt/usr/local/bin:$BUILD_DIR/.apt/usr/bin:$BUILD_DIR/.apt/usr/sbin:/sbin:$PATH"
+export PATH="$APP_PREFIX/nginx/sbin:$APP_PREFIX/luajit/bin:$APP_PREFIX/bin:$BUILD_DIR/.apt/usr/local/bin:$BUILD_DIR/.apt/usr/bin:$BUILD_DIR/.apt/usr/sbin:/sbin:$PATH"
 export LD_LIBRARY_PATH="$APP_PREFIX/lib:$BUILD_DIR/.apt/usr/lib/x86_64-linux-gnu:$BUILD_DIR/.apt/usr/local/lib:$BUILD_DIR/.apt/usr/lib:/usr/lib/x86_64-linux-gnu:/lib/x86_64-linux-gnu:/usr/lib"
 export LIBRARY_PATH="$APP_PREFIX/lib:$BUILD_DIR/.apt/usr/lib/x86_64-linux-gnu:$BUILD_DIR/.apt/usr/local/lib:$BUILD_DIR/.apt/usr/lib:/usr/lib/x86_64-linux-gnu:/lib/x86_64-linux-gnu:/usr/lib"
 export INCLUDE_PATH="$APP_PREFIX/include:$BUILD_DIR/.apt/usr/local/include:$BUILD_DIR/.apt/usr/include:/usr/include/x86_64-linux-gnu:/usr/include"
@@ -173,19 +172,19 @@ then
   mv ${VENDOR_DIR} ${VENDOR_CACHE_DIR}
   cd ${VENDOR_CACHE_DIR}
 
-  topic "Building LuaJIT"
-  tar -xf LuaJIT-${LUAJIT_VERSION}.tar.gz
-  cd LuaJIT-${LUAJIT_VERSION}
-  make PREFIX=${APP_PREFIX}
-  make install PREFIX=${APP_PREFIX}
-  ln -sf luajit-${LUAJIT_VERSION} ${APP_PREFIX}/bin/luajit
+  topic "Building OpenSSL"
+  tar -xf openssl-${OPENSSL_VERSION}.tar.gz
+  cd openssl-${OPENSSL_VERSION}
+  ./config --prefix=$APP_PREFIX -fPIC
+  make
+  make install
   cd ..
 
   topic "Building OpenResty"
   tar -xf openssl-${OPENSSL_VERSION}.tar.gz
-  tar -xf ngx_openresty-${OPENRESTY_VERSION}.tar.gz
-  cd ngx_openresty-${OPENRESTY_VERSION}
-  ./configure --prefix=$APP_PREFIX --with-pcre-jit --with-ipv6 --with-http_realip_module --with-http_ssl_module --with-http_stub_status_module --with-openssl=${VENDOR_CACHE_DIR}/openssl-${OPENSSL_VERSION} --with-luajit=${APP_PREFIX}
+  tar -xf openresty-${OPENRESTY_VERSION}.tar.gz
+  cd openresty-${OPENRESTY_VERSION}
+  ./configure --prefix=$APP_PREFIX --with-pcre-jit --with-ipv6 --with-http_realip_module --with-http_ssl_module --with-http_stub_status_module --with-http_v2_module --with-openssl=${VENDOR_CACHE_DIR}/openssl-${OPENSSL_VERSION}
   make
   make install
   cd ..
@@ -194,14 +193,11 @@ then
   topic "Building LuaRocks"
   tar -xf luarocks-${LUAROCKS_VERSION}.tar.gz
   cd luarocks-${LUAROCKS_VERSION}
-  ./configure --prefix=${APP_PREFIX} --lua-suffix=jit --with-lua=${APP_PREFIX} --with-lua-include=${APP_PREFIX}/include/luajit-2.1
+  ./configure --prefix=${APP_PREFIX} --lua-suffix=jit --with-lua=${APP_PREFIX}/luajit --with-lua-include=${APP_PREFIX}/luajit/include/luajit-2.1
   make build
   make install
   cd ..  
 
-  topic "Installing Serf"
-  unzip serf_${SERF_VERSION}_linux_amd64.zip
-  mv serf $APP_PREFIX/bin/
 
   topic "Caching build"
   mkdir -p $PREFIX_CACHE_DIR
@@ -225,16 +221,17 @@ else
   cd $KONG_SOURCE_DIR
   git checkout $KONG_GIT_COMMITISH
 fi
+export OPENSSL_DIR=${APP_PREFIX}
 # Install Kong itself from the cloned git repo
 luarocks make ./kong-*.rockspec
 # Install Kong's dependencies via LuaRocks
 # (`--local` is required for this to do anything)
-luarocks install --only-deps --local ./kong-*.rockspec
+luarocks install --local ./kong-*.rockspec
+mv bin/* ${APP_PREFIX}/bin/
 
 topic "Installing Lua rocks for kong-12f"
-luarocks install etlua 1.2.0
-luarocks install moses 1.4.0
-luarocks install serpent
+luarocks install etlua 1.3.0
+luarocks install luasocket
 
 topic "Installing Lua rocks specified in .luarocks"
 cd ${BUILD_DIR}
@@ -247,8 +244,8 @@ mv .profile.d/* $BUILD_DIR/.profile.d
 mkdir -p $BUILD_DIR/config
 mv config/* $BUILD_DIR/config
 
-topic "Creating NginX working directory"
-mkdir -p $BUILD_DIR/nginx-working
-
 topic "Making build artifacts available to app"
 mv $APP_PREFIX $BUILD_DIR/.heroku
+# Move executables for Procfile into place
+mkdir -p $BUILD_DIR/bin
+cp $BP_DIR/bin/app/heroku-* $BUILD_DIR/bin/

bin/detect

@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 # bin/detect <build-dir>
 
-if [ -f $1/config/kong.yml.etlua ]; then
+if [ -f $1/config/kong.conf.etlua ]; then
   echo "Kong" && exit 0
 else
   echo "no" && exit 1

bin/release

@@ -4,5 +4,6 @@ cat << EOF
 ---
 addons:
 default_process_types:
-  web: kong-12f && kong start -c config/kong.yml
+  web: bin/heroku-buildpack-kong-web
+  release: bin/heroku-buildpack-kong-release
 EOF