Permalink
Browse files

Switch to Buildkite beta agent

* Optimizes CI Dockerfile for cacheability
* Upload testing Cog container to Docker Hub
  Subsequent build steps will pick this up automatically. For now,
  we'll send images to the `cog-testing` repository, to give a clear
  separation between official and testing images.
* Dynamically generate pipeline specification
  This allows to DRY up a bit of repetition, and also eliminate the
  stand-alone build triggering script by folding that logic into the
  `pipeline.sh` script. By using Buildkite's native triggering
  mechanism, we only now need to make a single Git call to figure out
  what branch we want to build on.
* Removed `test.sh` script
  We can just use `wait-for-it.sh` directly, since it can run a command
  for you when it's done waiting.

Fixes #1280
  • Loading branch information...
christophermaier committed Jan 3, 2017
1 parent ae2608a commit 7954c6459c39b21cff1e8e668159a40bc5733109
View
@@ -0,0 +1,121 @@
#!/bin/bash
set -euo pipefail
# Time to wait for the database to come up in seconds
DB_TIMEOUT=30
cat <<EOF
steps:
- label: ":docker: Build Test Image"
agents:
queue: beta # TODO: dedicated builder queue
plugins:
docker-compose:
build: test
image_repository: "index.docker.io/operable/cog-testing"
config: docker-compose.ci.yml
- wait
EOF
########################################################################
# Self-contained Tests
for TEST in unit integration
do
cat <<EOF
- command: ./scripts/wait-for-it.sh postgres:5432 -s -t ${DB_TIMEOUT} -- make test-${TEST}
label: ":cogops: ${TEST}"
agents:
queue: beta
plugins:
docker-compose:
run: test
config: docker-compose.ci.yml
EOF
done
########################################################################
# Live Chat Provider Tests
for PLATFORM in slack hipchat
do
cat <<EOF
- command: ./scripts/wait-for-it.sh postgres:5432 -s -t ${DB_TIMEOUT} -- make test-${PLATFORM}
label: ":${PLATFORM}: Integration"
agents:
queue: beta
plugins:
docker-compose:
run: test
config: docker-compose.ci.yml
concurrency_group: "cog_${PLATFORM}_integration"
concurrency: 1
EOF
done
########################################################################
# "Real" Image build
#
# The docker-compose image above is for a testing build of Cog, not
# for the real images we ultimately distribute. In the future, we may
# converge the two, but for now, we'll build the real images
# separately. This'll allow us to provide up-to-date images if we so
# choose, as well as easily provide images to dependent pipelines
# (like the cogctl we trigger below).
#
# When / if Buildkite gets a plain Docker plugin (as opposed to the
# docker-compose one), we might be able to replace some of this custom
# code.)
# Not sure if both build number and commit SHA is overkill here or
# not. Both are probably useful right now; build number can help us
# (Operable) track things down, but commit provides extra clarity (and
# can be meaningfully used by non-Operable folks until Buildkite opens
# up builds for public view).
#
#
# For now, we'll push this to `cog-testing`, instead of `cog`, even
# though it's a "real" image. Once we start actually promoting
# official images from CI, though, we can send it to `cog`.
COG_IMAGE="operable/cog-testing:ci-build-${BUILDKITE_BUILD_NUMBER}-${BUILDKITE_COMMIT::8}"
cat <<EOF
- wait
- command: .buildkite/scripts/build_and_push_docker_image.sh $COG_IMAGE
label: ":docker: Build Real Image"
agents:
queue: beta # TODO future builder queue
- wait
EOF
########################################################################
# Triggered Builds
# If there's a branch in cogctl with the same name as the branch we're
# building here in Cog, use that branch. Otherwise, build on master.
if git ls-remote --exit-code --heads https://github.com/operable/cogctl refs/heads/${BUILDKITE_BRANCH} > /dev/null 2>&1
then
TRIGGER_BRANCH=${BUILDKITE_BRANCH}
else
TRIGGER_BRANCH='master'
fi
cat <<EOF
- trigger: "cogctl"
label: ":cogops: Triggered cogctl build"
async: true
build:
branch: "${TRIGGER_BRANCH}"
commit: "HEAD"
env:
COG_IMAGE: ${COG_IMAGE}
EOF
View

This file was deleted.

Oops, something went wrong.
@@ -0,0 +1,11 @@
#!/bin/bash
set -euo pipefail
export DOCKER_IMAGE="$1"
echo "--- :docker: Building ${DOCKER_IMAGE}"
make docker
echo "--- :docker: Pushing ${DOCKER_IMAGE}"
docker push ${DOCKER_IMAGE}
View

This file was deleted.

Oops, something went wrong.

This file was deleted.

Oops, something went wrong.
View
@@ -9,5 +9,14 @@ WORKDIR /code
RUN mix deps.get
RUN MIX_ENV=test mix deps.compile
COPY . /code
COPY emqttd_plugins/ /code/emqttd_plugins/
COPY priv/ /code/priv/
COPY test/ /code/test/
COPY web/ /code/web/
COPY lib/ /code/lib/
RUN MIX_ENV=test mix compile
COPY .buildkite/ /code/.buildkite
COPY scripts/ /code/scripts/
COPY Makefile /code
View
@@ -8,7 +8,7 @@ services:
- POSTGRES_PASSWORD=cog
expose:
- 5432
cog:
test:
build:
context: .
dockerfile: Dockerfile.ci

0 comments on commit 7954c64

Please sign in to comment.