Permalink
Browse files

Removing stunnel in favor of native TLS in pgbouncer (#104)

  • Loading branch information...
binarycleric committed Jul 9, 2018
1 parent 5344e8d commit 75d70dd7daf3777849a711b520b57d00ea85c647
Showing with 195 additions and 236 deletions.
  1. +1 −1 CONTRIBUTING.md
  2. +16 −22 README.md
  3. +7 −12 bin/compile
  4. +1 −1 bin/detect
  5. +5 −33 bin/gen-pgbouncer-conf.sh
  6. +156 −0 bin/start-pgbouncer
  7. +9 −151 bin/start-pgbouncer-stunnel
  8. BIN stunnel-5.28.tgz
  9. +0 −1 support/docker-compose.yml
  10. +0 −15 support/pgbouncer-build
View
@@ -1,4 +1,4 @@
## Compiling new versions of pgbouncer and stunnel using Docker
## Compiling new versions of pgbouncer using Docker
Install [docker](https://www.docker.com/). Use the Get Started button at the
top of the page, which autodetects your OS and presents the appropriate
View
@@ -1,17 +1,15 @@
# Heroku buildpack: pgbouncer
This is a [Heroku buildpack](http://devcenter.heroku.com/articles/buildpacks) that
allows one to run pgbouncer and stunnel in a dyno alongside application code.
It is meant to be [used in conjunction with other buildpacks](https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app).
allows one to run pgbouncer in a dyno alongside application code. It is meant
to be [used in conjunction with other
buildpacks](https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app).
The primary use of this buildpack is to allow for transaction pooling of
PostgreSQL database connections among multiple workers in a dyno. For example,
10 unicorn workers would be able to share a single database connection, avoiding
connection limits and Out Of Memory errors on the Postgres server.
It uses [stunnel](http://stunnel.org/) and [pgbouncer](http://wiki.postgresql.org/wiki/PgBouncer).
## FAQ
- Q: Why should I use transaction pooling?
- A: You have many workers per dyno that hold open idle Postgres connections and
@@ -71,28 +69,27 @@ Example usage:
Run `git push heroku master` to create a new release using these buildpacks.
$ cat Procfile
web: bin/start-pgbouncer-stunnel bundle exec unicorn -p $PORT -c ./config/unicorn.rb -E $RACK_ENV
web: bin/start-pgbouncer bundle exec unicorn -p $PORT -c ./config/unicorn.rb -E $RACK_ENV
worker: bundle exec rake worker
$ git push heroku master
...
-----> Multipack app detected
-----> Fetching custom git buildpack... done
-----> pgbouncer-stunnel app detected
Using pgbouncer version: 1.5.4-heroku
Using stunnel version: 5.08
Using stack version: cedar-14
-----> pgbouncer app detected
Using pgbouncer version: 1.7-heroku
-----> Fetching and vendoring pgbouncer into slug
-----> Fetching and vendoring stunnel into slug
-----> Moving the configuration generation script into app/bin
-----> Moving the start-pgbouncer-stunnel script into app/bin
-----> pgbouncer/stunnel done
-----> Moving the start-pgbouncer script into app/bin
-----> pgbouncer done
-----> Fetching custom git buildpack... done
...
The buildpack will install and configure pgbouncer and stunnel to connect to
`DATABASE_URL` over a SSL connection. Prepend `bin/start-pgbouncer-stunnel`
to any process in the Procfile to run pgbouncer and stunnel alongside that process.
The buildpack will install and configure pgbouncer to connect to
`DATABASE_URL` over a TLS connection, where available. Prepend
`bin/start-pgbouncer` to any process in the Procfile to run pgbouncer alongside
that process.
## Multiple Databases
@@ -106,7 +103,7 @@ It is possible to connect to multiple databases through pgbouncer by setting
HEROKU_POSTGRESQL_ROSE_URL=postgres://u9dih9htu2t3ll:password@ec2-107-20-228-134.compute-1.amazonaws.com:5482/db6h3bkfuk5430
DATABASE_URL=postgres://uf2782hv7b3uqe:password@ec2-50-19-210-113.compute-1.amazonaws.com:5622/deamhhcj6q0d31
~ $ bin/start-pgbouncer-stunnel env # filtered for brevity
~ $ bin/start-pgbouncer env # filtered for brevity
HEROKU_POSTGRESQL_ROSE_URL=postgres://u9dih9htu2t3ll:password@127.0.0.1:6000/db2
DATABASE_URL=postgres://uf2782hv7b3uqe:password@127.0.0.1:6000/db1
@@ -123,9 +120,9 @@ a replica, make sure to include the color url of your leader in the
your leader as a read-only replica, potentially doubling your connection count.
## Tweak settings
Some settings are configurable through app config vars at runtime. Refer to the appropriate documentation for
[pgbouncer](https://pgbouncer.github.io/config.html)
and [stunnel](http://linux.die.net/man/8/stunnel) configurations to see what settings are right for you.
[pgbouncer](https://pgbouncer.github.io/config.html) configurations to see what settings are right for you.
- `PGBOUNCER_POOL_MODE` Default is transaction
- `PGBOUNCER_MAX_CLIENT_CONN` Default is 100
@@ -142,9 +139,6 @@ and [stunnel](http://linux.die.net/man/8/stunnel) configurations to see what set
- `PGBOUNCER_LOG_POOLER_ERRORS` Default is 1
- `PGBOUNCER_STATS_PERIOD` Default is 60
- `PGBOUNCER_SERVER_RESET_QUERY` Default is empty when pool mode is transaction, and "DISCARD ALL;" when session.
- `PGBOUNCER_STUNNEL_LOGLEVEL` Default is notice (5). Set this var to pass a syslog level name or number value to stunnel. This corresponds to the stunnel global configuration option called "debug".
- `ENABLE_STUNNEL_AMAZON_RDS_FIX` Default is unset. Set this var if you are connecting to an Amazon RDS instance of postgres.
Adds `options = NO_TICKET` which is documented to make stunnel work correctly after a dyno resumes from sleep. Otherwise, the dyno will lose connectivity to RDS.
- `PGBOUNCER_IGNORE_STARTUP_PARAMETERS` Adds parameters to ignore when pgbouncer is starting. Some postgres libraries, like Go's pq, append this parameter, making it impossible to use this buildpack. Default is empty and the most common ignored parameter is `extra_float_digits`. Multiple parameters can be seperated via commas. Example: `PGBOUNCER_IGNORE_STARTUP_PARAMETERS="extra_float_digits, some_other_param"`
For more info, see [CONTRIBUTING.md](CONTRIBUTING.md)
View
@@ -13,10 +13,8 @@ unset GIT_DIR
# config
if [ "$STACK" == "heroku-16" ]; then
PGBOUNCER_VERSION="1.7.2-heroku"
STUNNEL_VERSION="5.40"
else
PGBOUNCER_VERSION="1.7-heroku"
STUNNEL_VERSION="5.28"
fi
# parse and derive params
@@ -39,30 +37,27 @@ function indent() {
}
echo "Using pgbouncer version: ${PGBOUNCER_VERSION}" | indent
echo "Using stunnel version: ${STUNNEL_VERSION}" | indent
# vendor directories
VENDORED_PGBOUNCER="vendor/pgbouncer"
VENDORED_STUNNEL="vendor/stunnel"
# vendor pgbouncer into the slug
PATH="$BUILD_DIR/$VENDORED_PGBOUNCER/bin:$PATH"
echo "-----> Fetching and vendoring pgbouncer into slug"
mkdir -p "$BUILD_DIR/$VENDORED_PGBOUNCER"
tar xzf pgbouncer-${PGBOUNCER_VERSION}.tgz -C ${BUILD_DIR}/${VENDORED_PGBOUNCER}
# vendor stunnel into the slug
PATH="$BUILD_DIR/$VENDORED_STUNNEL/bin:$PATH"
echo "-----> Fetching and vendoring stunnel into slug"
mkdir -p "$BUILD_DIR/$VENDORED_STUNNEL"
tar xzf stunnel-${STUNNEL_VERSION}.tgz -C ${BUILD_DIR}/${VENDORED_STUNNEL}
echo "-----> Moving the configuration generation script into app/bin"
mkdir -p $BUILD_DIR/bin
cp "$BUILDPACK_DIR/bin/gen-pgbouncer-conf.sh" $BUILD_DIR/bin
chmod +x $BUILD_DIR/bin/gen-pgbouncer-conf.sh
echo "-----> Moving the start-pgbouncer-stunnel script into app/bin"
echo "-----> Moving the start-pgbouncer script into app/bin"
mkdir -p $BUILD_DIR/bin
cp "$BUILDPACK_DIR/bin/start-pgbouncer" $BUILD_DIR/bin/
chmod +x $BUILD_DIR/bin/start-pgbouncer
echo "-----> Moving the (legacy) start-pgbouncer-stunnel script into app/bin"
mkdir -p $BUILD_DIR/bin
cp "$BUILDPACK_DIR/bin/start-pgbouncer-stunnel" $BUILD_DIR/bin/
chmod +x $BUILD_DIR/bin/start-pgbouncer-stunnel
@@ -72,4 +67,4 @@ mkdir -p $BUILD_DIR/bin
cp "$BUILDPACK_DIR/bin/use-pgbouncer" $BUILD_DIR/bin/
chmod +x $BUILD_DIR/bin/use-pgbouncer
echo "-----> pgbouncer/stunnel done"
echo "-----> pgbouncer done"
View
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# bin/detect <build-dir>
echo 'pgbouncer-stunnel'
echo 'pgbouncer'
exit 0
View
@@ -11,33 +11,15 @@ if [ -z "${SERVER_RESET_QUERY}" ] && [ "$POOL_MODE" == "session" ]; then
SERVER_RESET_QUERY="DISCARD ALL;"
fi
# Enable this option to prevent stunnel failure with Amazon RDS when a dyno resumes after sleeping
if [ -z "${ENABLE_STUNNEL_AMAZON_RDS_FIX}" ]; then
AMAZON_RDS_STUNNEL_OPTION=""
else
AMAZON_RDS_STUNNEL_OPTION="options = NO_TICKET"
fi
mkdir -p /app/vendor/stunnel/var/run/stunnel/
cat > /app/vendor/stunnel/stunnel-pgbouncer.conf << EOFEOF
foreground = yes
options = NO_SSLv2
options = SINGLE_ECDH_USE
options = SINGLE_DH_USE
socket = r:TCP_NODELAY=1
options = NO_SSLv3
${AMAZON_RDS_STUNNEL_OPTION}
ciphers = HIGH:!ADH:!AECDH:!LOW:!EXP:!MD5:!3DES:!SRP:!PSK:@STRENGTH
debug = ${PGBOUNCER_STUNNEL_LOGLEVEL:-notice}
EOFEOF
cat > /app/vendor/pgbouncer/pgbouncer.ini << EOFEOF
cat >> /app/vendor/pgbouncer/pgbouncer.ini << EOFEOF
[pgbouncer]
listen_addr = 127.0.0.1
listen_port = 6000
auth_type = md5
auth_file = /app/vendor/pgbouncer/users.txt
server_tls_sslmode = prefer
server_tls_protocols = secure
server_tls_ciphers = HIGH:!ADH:!AECDH:!LOW:!EXP:!MD5:!3DES:!SRP:!PSK:@STRENGTH
; When server connection is released back to pool:
; session - after client disconnects
@@ -79,25 +61,15 @@ do
export ${POSTGRES_URL}_PGBOUNCER=postgres://$DB_USER:$DB_PASS@127.0.0.1:6000/$CLIENT_DB_NAME
fi
cat >> /app/vendor/stunnel/stunnel-pgbouncer.conf << EOFEOF
[$POSTGRES_URL]
client = yes
protocol = pgsql
accept = /tmp/.s.PGSQL.610${n}
connect = $DB_HOST:$DB_PORT
retry = ${PGBOUNCER_CONNECTION_RETRY:-"no"}
EOFEOF
cat >> /app/vendor/pgbouncer/users.txt << EOFEOF
"$DB_USER" "$DB_MD5_PASS"
EOFEOF
cat >> /app/vendor/pgbouncer/pgbouncer.ini << EOFEOF
$CLIENT_DB_NAME= dbname=$DB_NAME port=610${n}
$CLIENT_DB_NAME= host=$DB_HOST dbname=$DB_NAME port=$DB_PORT
EOFEOF
let "n += 1"
done
chmod go-rwx /app/vendor/pgbouncer/*
chmod go-rwx /app/vendor/stunnel/*
View
@@ -0,0 +1,156 @@
#!/usr/bin/env bash
# Adapted from https://github.com/ryandotsmith/nginx-buildpack/
main() {
if ! is-enabled "${PGBOUNCER_ENABLED:-1}"; then
at pgbouncer-disabled
exec "$@"
fi
at pgbouncer-enabled
run-pgbouncer "$@"
}
run-pgbouncer() {
declare psmgr=/tmp/pgbouncer-buildpack-wait
declare -A pids signals
config-gen
# Use named pipe to detect exit of any subprocess.
rm -f "$psmgr"
mkfifo "$psmgr"
# Start processes.
aux-start pgbouncer SIGINT vendor/pgbouncer/bin/pgbouncer vendor/pgbouncer/pgbouncer.ini
app-start SIGTERM "$@"
pid=$!
pgid=$(ps -o pgid= $pid | xargs)
# Don't exit top script until all subprocesses are done.
trap '' SIGTERM
# This read will block the process waiting on a msg to be put into the
# fifo. If any of the processes defined above should exit, a msg will be
# put into the fifo causing the read operation to un-block. The process
# putting the msg into the fifo will use it's process name as a msg so that
# we can print the offending process to stdout.
declare exit_process
read exit_process < "$psmgr"
at "exit process=$exit_process"
# Remove the FIFO. This allows following writes to simply create a file,
# rather than blocking because there's nothing reading the other end.
rm -f "$psmgr"
# Clean up any running processes.
# SIGTERM the application's process group (hence the negative PID), just in
# case something else crashed. If the dyno is shutting down, then SIGTERM
# has already been sent.
at "kill-app pid=$pid"
kill -SIGTERM -$pgid
# Wait for the app to finish.
at "wait-app pid=$pid"
wait $pid
# Kill the auxiliary processes.
# Send each one SIGHUP which will be translated by the trap in aux-start.
declare name
for name in "${!pids[@]}"; do
at "kill-aux name=$name pid=${pids[$name]} signal=${signals[$name]}"
kill -SIGHUP "${pids[$name]}"
done
}
config-gen() {
# Generate config files
at config-gen-start
source bin/gen-pgbouncer-conf.sh
at config-gen-end
# Overwrite config vars with pgbouncer targets
POSTGRES_URLS=${PGBOUNCER_URLS:-DATABASE_URL}
for POSTGRES_URL in $POSTGRES_URLS; do
at "config-gen-override $POSTGRES_URL"
eval "$POSTGRES_URL=\$${POSTGRES_URL}_PGBOUNCER"
done
}
aux-start() {
declare name=$1 signal=$2
shift 2
(
at "$name-start"
# Ignore SIGTERM; this is inherited by the child process.
trap '' SIGTERM
# Start child in the background.
"$@" &
# Translate SIGHUP to the appropriate signal to stop the child (anything
# except SIGTERM which is ignored). This *will* cancel the wait and may
# lead to the outer subshell exiting before the aux process
trap "kill -$signal $!" SIGHUP
# Wait for child to finish, either by crash or by $signal
wait
# Notify FIFO if this finishes first
echo "$name" > "$psmgr"
at "$name-end"
) &
pids[$name]=$!
signals[$name]=$signal
at "$name-launched pid=$! signal=$signal"
}
app-start() {
declare name=app signal=$1
shift
(
at "$name-start"
# Start child in the background. This is before the trap because
# the app needs to be able to receive when the dyno broadcasts
# SIGTERM on shutdown.
"$@" &
# Translate SIGHUP to the appropriate signal to stop the child
# (probably SIGTERM in this case). This *will* cancel the wait and may
# lead to the outer subshell exiting before the app.
trap "kill -$signal $!" SIGHUP
# Ignore SIGTERM because the dyno will broadcast it to all children --
# there is no need to translate it.
trap "" SIGTERM
# Wait for the app to finish, either by crash or by $signal
wait
# Notify FIFO if this finishes first
echo "$name" > $psmgr
at "$name-end"
) &
at "$name-launched pid=$!"
}
at() {
echo "buildpack=pgbouncer at=$*"
}
is-enabled() {
( shopt -s extglob nocasematch
[[ $1 == @(1|true|yes|on) ]]
)
}
[[ "$0" != "$BASH_SOURCE" ]] || main "$@"
Oops, something went wrong.

0 comments on commit 75d70dd

Please sign in to comment.